About  Posts  Feed

  1. Effects, functionally

    What are functional effects, really?

    Functional effects can be used to model side effects in a purely functional program. When we think about what side effects are in a non-pure program we can say that they are all the other things that can happen besides passing around a value. Functional effects are a functional way to keep this separation of the effect and passing around a value.

    Functional effects consist of:

    • a polymorphic data type that separates the value from the effect
    • combinators on that data type that allow you to work with the value without having to deal with the effect

    Let’s look at some examples (from PureScript):

    Partiality

    data Maybe a = Nothing | Just a
    
    • Separates the value a from the effect of partiality enabled by the Nothing constructor
    • Effect: partiality, being able to short circuit a computation, nondeterministic choice
    • Useful combinators: bind, apply, alt

    Read-only context

    data Reader r a = Reader (r -> a)
    
    • Separates the value a from the effect of reading a value r from the context
    • Effect: access to a value inside the computation, a “global value” so to speak
    • Useful combinators: bind, apply, ask, local

    Read-write context

    data State s a = State (s -> Tuple a s)
    
    • Separates the value a from the effect of state s
    • Effect: access to a read-write context within the computation
    • Useful combinators: bind, apply, put, modify

    No effects

    data Identity a = Identity a
    
    • Separates the value a from the “null” effect. A surprisingly useful data type that can be used in places where no effects are needed.
    • Effect: none
    • Useful combinators: bind, apply, alt, and many others

  2. Start refactoring from the leaves

    To reduce duplication in your code, there usually are two ways to approach it:

    1. Start from the top and extract the biggest common block of code to a function
    2. Start from the bottom (leaves) and extract the common expression(s) to a function

    It’s best to always aim for doing the latter and start by extracting smaller things rather than bigger things. Here’s why:

    • There’s a much bigger chance that you will extract a function that is small enough to not require any further refactoring. You will have extracted a part of code that will have more lasting value.
    • Smaller parts are easier to compose, so you will end up with a better overall design. In some cases you might discover a new abstraction hiding underneath, making it possible to express the code’s intent more clearly.
    • Extracting smaller (pure) functions has more chance of being able to add tests around the new code.
    • If you start from the top you will most likely make a mistake anyway because it’s really difficult to see the commonalities in a big block of code.

    Starting small seems counterintuitive because it feels like nothing is getting done. We all like to removing code. It’s better to start small, it always works out better.


  3. The CMS problem

    Every CMS project needs to tackle a subset of what I like to call the “CMS problem”. A CMS is like a programming language where every feature has been lifted to the runtime.

    • Data types = content schemas
    • Type definitions = OpenAPI specs
    • Type checking = content validation
    • Data structures = content (defined by the schemas)
    • Logic = end-user programmability, which results in extensive use of defunctionalization and scripting
    • Linking = Dynamic linking over HTTP APIs

  4. Static analysis using functional abstraction

    A look at some functional abstractions that enable static analysis.

    This is a common problem: you want to write a program that can explain itself without running it with real data:

    • A CLI argument parser that can generate the --help output without providing it with any actual inputs
    • A build system that can show you the dependencies between build steps without running the actual build
    • A formula that can visualize itself without providing it with any data

    In other words, we can perform runtime static analysis on the code. Runtime static analysis means that we must run the program to analyze it, but we don’t have to provide any user inputs to it.

    Some programming languages can use reflection to accomplish some of this, and it’s a bit similar in the sense that you are not running the program with user inputs. But it’s also very different from what I’m talking about here because with reflection you need to create a special program to analyse the code. It also requires support from the runtime which many languages simply don’t have.

    The problem

    Let’s first zoom into the issue to understand what the problem is. If we take a normal imperative program like the following:

    foo :: Effect Number
    foo = do
      x <- baz
      bar x
    

    which is the same as this:

    foo = baz `bind` \x -> bar x
    

    we can observe that in order to evaluate bar we need the value x from baz. If we want to analyze this program without running it we are out of luck. In order to feed an x to bar x we must run baz.

    We can also just look at the type signature of bind for Monad:

    bind :: m a -> (a -> m b) -> m b
                         ^^^
    

    and observe that in the second argument the term m b is inside a lambda, and these terms of type m b are the ones that we want to analyze and see statically without running the program. To make program analysis possible we must use other abstractions which don’t have the terms inside a lambda.

    Applicative functors

    One such abstraction is captured by the Apply/Applicative type class in Haskell/PureScript:

    class Functor f => Apply f where
      apply :: f (a -> b) -> f a -> f b
    

    In neither of the arguments f is inside a lambda, which is what we need in order to make static analysis on f possible. The f in the first argument is parameterized by a pure function a -> b which we don’t need to apply in order to know what the f is.

    There are several ways we can design our Applicative data type so that it supports static analysis. The paper Free Applicative Functors (PDF) explains one approach using a “Free structure”. The Haskell package optparse-applicative uses a similar encoding to implement an argument parser that can generate a --help page from the program definition. The PureScript package argparse-basic on the other hand builds both the help and the parser in one go. Both libraries provide a similar API relying on Applicative.

    Selective applicative functors

    Selective Applicative Functors (PDF) are interesting.

    class Applicative f <= Selective f where
      select :: forall a b. f (Either a b) -> f (a -> b) -> f b
    

    First of all we can see that no term f x is inside a lambda, so that’s good in terms of static analysis. The point of Selective is that the second effect, f (a -> b), is optional. We can’t apply the callback a -> b if the first effect doesn’t provide an a. When the first effect returns a b, we can skip the second effect and just return the b.

    We can see that Selective provides a static structure much like Applicative but it also allows for rudimentary branching logic – all the branches are “visible” statically. The branchS combinator is a good example of this:

    branchS :: forall f a. Selective f => f Boolean -> f a -> f a -> f a
    branchS i t e = select (map (bool (Left unit) (Right unit)) i) (map const t) (map const e)
      where
      bool x y test = if test then x else y
    

    There are two possible branches that the computation might take. The one that is taken depends on the Boolean we get from the first effect. Since we can enumerate all the values of a Boolean, we can statically also enumerate all the possible consequences because neither of the branches are under a lambda term.

    In the paper they explain how Selective Applicative Functors can be used in build systems to calculate dependencies statically without running the build.

    Lifting to Category

    There’s one other way to make static analysis possible even in the presense of dependencies between terms. Basically, you can take the callback from bind:

    bind :: m a -> (a -> m b) -> m b
                   ^^^^^^^^^^
    

    and lift it to a data type:

    data Star m a b = Star (a -> m b)
                           ^^^^^^^^^^
    

    This eliminates the lambda term from the type you are composing. Instead of composing terms of m a you are now composing terms of the shape a -> m b. Essentially here we are composing “boxes” of inputs and outputs, without the possibility of knowing what the inputs and outputs are exactly. This forces you to work with more high level APIs like Category and Profunctor.

    The most basic example is the compose function in the Category type class:

    compose :: Category cat => cat b c -> cat a b -> cat a c
    
    a2c :: Category cat => cat a c
    a2c = compose b2c a2b
    

    We can statically analyze the expression a2c and say that both a2b and b2c are used to compute the result. The same cannot be said if we write the composition “by hand” using plain function composition:

    compose :: (b -> c) -> (a -> b) -> (a -> c)
    
    a2c :: a -> c
    a2c = compose b2c a2b 
    

    The reason, once again, is that the terms we are composing include lambda terms, and we can’t look inside them without applying them to some value.

    Now, it might not be useful to analyze the program at this higher Category level but it could be useful in situtations where applicative composition is not possible. You could say that with this approach we can see what boxes are being composed, but we can’t see what those boxes contain. But remember, boxes can be composed of other boxes! There are combinators at your disposal from Category and the Profunctor hierarchy which provide ways to do this.

    The tradeoff with this kind API is that it is much more high-level and those combinators can be quite difficult to work with, at least it takes some time getting used to. Because it’s so high-level, the analysis you can do on the terms is limited to be on a higher level as well.

    Conclusion

    I’ve shown a few ways we can approach runtime static analysis using functional abstraction. We’ve seen how Monad is doesn’t have the necessary properties while Applicative Functors do. There is also Selective Applicative Functors that sit between Applicative and Monad, which do have similar qualities for doing limited runtime static analysis. Sometimes we can even reach for a higher level abstraction with Category and Profunctor to be able to analyze programs on a higher level. I didn’t even mention Monoids or Semirings which have good properties for static analysis since they are even simpler than the ones mentioned in this post. That will have to be another post!


  5. Learning the fundamentals

    Something I’ve been thinking lately is where to focus my learning. Should my learning be focused on learning more practical things like new programming tools and languages or should it be mainly focused on learning theory and more fundamental things? For some reason I’m somewhat lacking motivation to learn new tools and I’m much more keen on learning new “fundamental” skills. But what are fundamental skills? I think fundamental skills are those that age well. That of course also depends on your circumstances and what kind of work you do as a programmer. For me as someone who works as a consultant it means that the skills should be applicable to many tools and programming languages. There’s an obvious tradeoff though - theory is not always practical and practice often doesn’t age well. So we must find a balance and make learning a fun process at the same time.

    Practice

    On the one hand I could spend the little freetime I have just writing code using the tools I already know. This is fun at times but in my experience quickly turns into a grind resembling real work. I’ve never been able to sustain the development of bigger OSS software projects because of this. It’s just not super fun doing the same thing for a longer period of time. With bigger projects the work also consists of maintanance tasks like fixing bugs, making releases and writing documentation, which can be hard work.

    On the contrary, picking up new tools and libraries can act as vehicles for much deeper learning than it might first seem like. Maybe you find a bug in a compiler or a library and you start digging deeper, which requires you to learn things that are not related to the original problem but that challenge your current understanding in some way.

    There are also differences in the tools you decide to spend your time on. Should you pick up Go or PureScript? I’d argue that PureScript stands on a much firmer theoretical ground and thus provides deeper things to learn. How much of skills involved in knowing Go are transferable to other languages? To be honest I don’t think much of it is. On the contrary, learning a pure functional languages will make you a better Java programmer, I’m sure of it.

    Theory

    On the theory side of things we have programming paradigms, algorithms, formal methods, mathematics, and maybe some programming laguages with deep theory behind them. These have more potential to age better than just learning the new hipster devstack and be a good investment in your career. For me the difficult part about learning theory is that it’s …difficult, and takes a lot of time. I find that it’s also often difficult to find good learning material which might be in part because of the things I’ve chosen to learn.

    Learning theory and fundamentals also seems like a good way to secure our jobs when AI replaces some of the grind from software development.

    Combining the theory learning with some programming seems like the winning combination for me. This is obviously the case when learning programming related theory like type systems and category theory, but I’ve been wondering whether this method would work for learning mathematics as well since we have better tools now.


  6. Squash merge all the things

    I’m a huge fan of the squash merge strategy in GitHub and with git in general. Squash merging is a merging strategy where the commits of branch are combined (squashed) to a single commit. You should use it in your projects, here’s why.

    Clean history

    The most obvious and immediate benefit of this approach is that the history of the main branch will be kept very clean and linear - no more “merge bubbles”. This happens even when your feature branch is not up to date with the main branch, because a single new commit will be added on top of whatever was last merged to main. When your project grows, keeping the main branch history clean is important because it’s vastly easier then to keep track of the changes between releases. Github adds the PR number by default to the squash merge commit message which also helps finding the associated PR.

    Feature branches and whether to rebase or merge

    The argument I’ve had most about using git in general is how to keep the feature branches up-to-date with the main branch. Either you want to integrate a feature from main in order to finish your PR or you want to ensure your branch works with main before it’s merged. You could do this either with rebasing or merging in the main to your feature branch, and I’ve noticed that in general devs prefer rebasing. Their argument usually is that they want to keep the history clean, without merge commits. It’s understandable, because if you merge multiple times, by the time you merge your feature branch to main you might have multiple merge commits, which is not nice.

    The problem with rebasing is that it rewrites history. This is especially bad with PRs because code reviewing requires that someone else keeps track of the changes you are making. The reviewer usually has to come back and look at the fixes the PR author made after comments. Github even has this feature where it will track the changes the reviewer has seen. If you force-push to your branch this feature doesn’t work. It will lose track of the history (obviously, because you rewrote it) and it won’t show new stuff anymore.

    I was once in a project where some devs kept browsing through the list of PRs and just manually rebased all the branches every now and then without asking the author. It was a big team, with a monorepo kind of a setup so the person doing the rebase might have been someone that didn’t know anything about what you were working on - let alone the project. It’s obviously ok to rebase and force-push to your own feature branch when it’s not public yet (no PR created). But that you, or let alone other people, rewrite the history when it’s being used by other devs just causes problems.

    All of these problems go away when using squash merging. You can add as many “fixup” commits as you want, you can merge in the main branch as many times as you want. Go nuts with your feature branch, it doesn’t matter. Do whatever you can to make reviewing easier by not removing or editing commits and merge in the main branch if you have to. Squash merging solves this argument once and for all: never rebase!

    Best practices with squash merging

    There are some things that a team needs to pay attention when using squash merging.

    When you do the squash-merge, make sure the commit message is edited a bit so that it’s not just a list of all the commit messages that are going to be squashed. Often it’s enough to copy and paste a part of the PR description if you have a habit of writing those (you should!).

    Another important thing is to also run your full test suite and build in the main branch after merging in a feature branch. I’ve found out that it’s much better to do this instead of keeping the feature branches up-to-date with the main branch all the time. Even in a medium-sized team doing small PRs this becomes really difficult in practice. Better to just accept that the build can fail in the main branch and just fix it if this happens.

    Lastly, squash merging only makes sense if you do small PRs that only do one thing. This basically means you can’t have long-lived branches and should practice trunk-based development.

    Conclusion

    You should consider using squash merging in your project because it keeps your git history clean and still allows for hygienic PR review practices by using merging instead of rebasing to keep feature branches up to date.


  7. Purely functional dependency injection in TypeScript

    A very common pattern I see in web applications on the server side is a database connection module that exposes a bunch of functions. These functions run queries to the database and may call other functions defined in the module to abstract some of the common database operations. Let’s look at one possible solution.

    function query(client: pg.Client, sql: Query, params: Param[]): Task<pg.QueryResult> {
      return client.query(sql, params)
    }
    
    function fetchUser(client: pg.Client, id: number): Task<User> {
      return query(client, "SELECT first_name, last_name from users where id = ?", [id])
        .map(rows => rows[0])
    }
    

    This is the most obvious solution. We just pass around the client to every function. But that is also the problem with this approach. The database functions quickly become infectious: every function that uses one of those needs to also receive the pg.Client as argument.

    This is a problem we want to solve with dependency injection. In object oriented programming we usually create a class and inject the DB client to it. The functions become methods in that class and have access to the client.

    class Database {
      constructor(client: pg.Client) { }
    
      query(sql: Query, params: Param[]): Task<pg.QueryResult> {
        return this.client.query(sql, params)
      }
    
      fetchUser<T>(id: number): Task<User> {
        return this.query("SELECT first_name, last_name from users where id = ?", [id])
          .map(rows => rows[0])
      }
    }
    

    This somewhat solves the problem and we no longer need to pass the pg.Client around. However, we then need to pass that class around we just created. We also need to write code in a more object oriented style: we can no longer just pass functions around because we have an object instead the plain functions. And I like functions, they are easy to work with.

    How can we keep the simplicity of functions without having to pass that client around?

    The essence of the database layer

    The trick is to identify the essence of the database abstraction. Looking at the functions we have, what are the parts that are common to both?

    function query(client: pg.Client, sql: Query, params: Param[]): Task<pg.QueryResultRow[]> {
    }
    
    function fetchUser(client: pg.Client, id: number): Task<User> {
    }
    

    Both of them take the raw database client as input and return a Task. Ok, so extracting the common parts out we are left with a function type:

    function query<A>(client: pg.Client): Task<A>
    

    Note the introduction of a type variable A for the return value inside the Task.

    So, with the common parts extracted out into a type, what do we do with it?

    What we can do is, instead of returning a Task from our fetchUser function, we return the query function itself.

    function fetchUser(id: number): (client: pg.Client) => Task<User> {
      // Returning the `query` instead of calling it!
      return function query(client: pg.Client): Task<User> {
        return client.query("SELECT first_name, last_name from users where id = ?", [id])
          .map(result => result.rows[0])
      }
    }
    

    The return type now has the same type as the query function had! In fact, let’s extract that type out so it becomes a bit clearer, and let’s call it DatabaseFn because it represents our database abstraction.

    type DatabaseFn<A> = (client: pg.Client) => Task<A>
    
    function fetchUser(id: number): DatabaseFn<User> {
      return function (client: pg.Client): Task<User> {
        return client.query("SELECT first_name, last_name from users where id = ?", [id])
          .map(result => result.rows[0])
      }
    }
    

    So now our fetchUser itself returns a function. This also means that calling the fetchUser function doesn’t require us to have the client available.

    To actually run our query (really just to give us a Task) we must call the database function we returned:

    fetchUser(42)(client)
    

    What all this has now given us is a pure function which we can just import from anywhere and call without needing to provide the client value. We can rely on the fact that the client will be passed in later at the top level of the call stack when we actually want to run the query.

    More complex domain logic

    Now, what if we want to further abstract things and call fetchUser from another DB function? Do we then need to have the client available? No. We can just keep returning those database functions.

    // Fetch one row, fail if no rows were found
    function queryOne(sql: Query, params: Param[]): DatabaseFn<QueryResultRow> {
      return (client: pg.Client): Task<QueryResultRow> {
        client.query(query, params)
          .chain(rows => {
            return rows.length > 0
              ? Task.of(rows[0])
              : Task.rejected(new Error('Query returned no rows'))
          })
      }
    }
    
    function authenticate(username: string, password: string): DatabaseFn {
      return function (client: pg.Client): Task<User> {
        const query = "SELECT id from users where username = ? and password = ?"
        // `queryOne` returns a `DatabaseFn` so we need to pass it a `client`
        // in order to give us a `Task` so we can `chain`.
        return queryOne(query, [username, password])(client)
          .chain(id => {
            // Same here, need to unwrap our `DatabaseFn`.
            return fetchUser(id)(client)
          })
      }
    }
    

    We are still returning a DatabaseFn. Note that chain is a method from Task, and since we are inside a Task we must also return a Task. Each time we call a query function we need to unwrap the DatabaseFn to be able to chain with another call.

    All that DatabaseFn unwrapping creates too much boilerplate and is error prone to write. It doesn’t compose well. Can we do better?

    Make it a datatype

    In order to combine our database functions more easily we need some kind of helper functions or something to make it simpler. There are several solutions to this, but I’m proposing a solution here that uses classes.

    Instead of passing around the raw DatabaseFn function around, let’s wrap it in a class.

    class DatabaseFn<A> {
      constructor(readonly fn: (client: pg.Client) => Task<A>) {}
    }
    

    We can now add methods to this class to work with the wrapped function. Let’s add chain which allows call chaining like we did in authenticate.

    class DatabaseFn<A> {
      constructor(readonly fn: (client: pg.Client) => Task<A>) {}
    
      chain<B>(f: (a: A => DatabaseFn<B>)): DatabaseFn<B> {
        return new DatabaseFn((client: pg.Client) => {
          this.fn(client).chain(v => f(v)(client))
        })
      }
    }
    

    This is nice but we don’t have any way of running queries, we can only chain them! Let’s write a query combinator for this.

    function query(sql: Query, params: Param[]): DatabaseFn<QueryResultRow[]> {
      return new DatabaseFn((client: pg.Client) => {
        return client.query(sql, params)
      })
    }
    

    Now we can rewrite queryOne and authenticate with this.

    // Fetch one row, fail if no rows were found
    function queryOne(sql: Query, params: Param[]): DatabaseFn<QueryResultRow> {
      return query(query, params)
        .chain(rows => {
          return rows.length > 0
            ? DatabaseFn.of(rows[0])
            : DatabaseFn.throw(new Error('Query returned no rows'))
        })
    }
    
    function authenticate(username: string, password: string): DatabaseFn<User> {
      return queryOne("SELECT id from users where username = ? and password = ?", [username, password])
        .chain(row => fetchUser(row.id))
    }
    

    Much nicer! (The implementation of DatabaseFn.of and DatabaseFn.throw are left as an exercise for the reader :)

    We can now write pure functions to run queries against a database and we are not required to pass in the client argument around. What’s more, by wrapping the database function into it’s own datatype we can attach methods to it for easier chaining. Contrasting this to the object oriented approach to dependency injection, we don’t pass in the client to any object beforehand. Instead, we combine our database interactions using chain and then at the end pass in the client to run the program.

    The reader pattern

    When you see that you have many functions that receive the same input, like a resource of some kind, I propose the following:

    • Identify the common input to your functions.
    • Identify the return type and make it polymorphic by adding a type variable.
    • Wrap the function you get out of these into a class.
    • Add helper methods to that class: map and chain are good candidates.
    • Write your domain logic using pure functions that return instances of your class.

    The “pattern” I’m describing here can itself be generalized into a datatype. In Haskell and PureScript this is called the reader monad transformer, ReaderT. While it is possible to implement ReaderT in Typescript and Javascript, I think this simple pattern will get you a long way. There is an implementation of ReaderT for TypeScript in fp-ts if you want to dig deeper.


  8. Timeboxing

    I’ve been experimenting recently with timeboxing for personal work. I don’t remember what triggered the interest this time, but it probably was the chronic procrastination that I suffer from.

    Pomodoro

    I googled “time boxing for personal tasks” or something like that and all the links seemed to lead to the pomodoro technique. I was already familiar with the pomodoro technique so I wasn’t that interested in that right now. I was mainly looking for tips on hard timeboxing of personal tasks. Like, set a time-box, work that amount, ship. Pomodoro is for splitting your work into manageable chunks, but I was looking for advice on how to really squeeze out the important stuff and ship.

    That turned out to be a little too hard core of a time management tactic but that got me into pomodoro yet again.

    I’m not going to explain what the pomodoro technique is, you can find a lot of stuff about it elsewhere. But basically what I now do is I start a timer and work for 25 minutes, try to have a break of 5 minutes after that, repeat.

    I’m quite happy how this has improved my productivity and focus. It has helped with procrastination a lot, which is most important. Here are some thoughts and insights I’ve had with the pomodoro technique:

    • The barrier to start working is now way lower since all you have to commit to is 25 minutes of work. You say to yourself that you can just stop after 25 minutes if nothing comes out of it. This never happens. I always want to keep on working. This has the greatest impact on procrastination that I’ve experienced.

    • When you set aside 25 minutes for some task, it automatically makes you think about the task beforehand a bit. I often have a task in OmniFocus that I dedicate to a task. It makes you plan your todo items so that they fit better into 25 minute chunks.

    • The break after 25 minutes makes you stop and think about what you are doing to contemplate how you are progressing. I often feel like I don’t want to have a break, especially when programming. Then I just keep on going. Even if I continue with the next pomodoro without a break I at least recognize that 25 minutes have now gone by and I briefly think about if what I’m doing is the right thing. I ask myself: should I be doing this in the first place? Am I doing the right thing? How many pomodoros have I worked on this? How many more before I should reconsider the whole task?

    • When the pomodoro is on and the clock is ticking I’m much more aware of interruptions. I open Twitter or Slack out of old habits and quickly notice that I should be working and switch back to work mode. This is actually much like mindfulness meditation, where instead of noticing your thoughts, you notice the interruptions, both in your head but also the external ones. Listening to something calming like rain sounds or white noise helps here as well, because I find it helps me to remember that the clock is ticking, that I’m in work mode.

    • I’m much more aware of the amount of time that goes into actual work (in my case programming and writing) than other tasks. The app that I use keeps track of the completed pomodoros for today and it’s set to count until 11 pomodoros every day. I’ve never completed those 11. And that’s probably ok. It doesn’t mean I’m not working, but it keeps me aware of how much time goes into other stuff.

    • Sometimes the ring of the pomodoro timer telling you to continue working can feel a bit stressful. You just don’t feel like working. At that point I always take a small step back and try to think why the thing I’m doing feels so boring and tedious. It’s ok if work sometimes feels like… work, but I usually enjoy what I do, and stopping for a second to think what I’m doing when I don’t feel good might tell me I should be doing something else or take a different approach.

    In conclusion, I would say the pomodoro technique, or timeboxing in general, is really about feedback. And it’s the best procrastination hack there is.

    Here are a few related posts I liked:


  9. Promise is the wrong abstraction

    Swallowed Stop
    "Swallowed Stop" by Theen Moy

    If you are using Promises in JavaScript or about to, I’d suggest you reconsider. Here’s why:

    • The Promise API is bad
    • it’s the wrong abstraction to use for asynchronous side effects.

    It’s a bad API

    I almost didn’t have to write this article because just as I was writing my first draft Aldwin Vlasblom wrote an excellent article on broken promises. That article summarizes well what’s wrong with the Promises API.

    While the API is bad I think it’s the abstraction itself which causes more problems. The abstraction leaks, and just isn’t suitable in many cases. In particular, it’s the eagerness or non-purity about Promises that make them bad.

    The alternative I’m going to talk about here is a similar abstraction for asynchronous values called a Task. There are many libraries like Task, such as Fluture, and what I’m about to say applies to both of those and to many other pure alternatives to Promises.

    The wrong abstraction

    There’s nothing inherently wrong with the Promise abstraction, but I think it’s the wrong abstraction in many places where it is used.

    Promises in JavaScript represent processes which are already happening, which can be chained with callback functions. - MDN

    A Promise is like a box that has a value or will have a value at some point in the future. When we create a Promise it is immediately run:

    const twelve = new Promise((resolve, reject) => {
      resolve(12)
    })
    

    twelve is now a Promise containing the number 12. The function we pass into the Promise, called an executor, is immeditaly run so that before the call to Promise returns the executor has been called.

    A Task looks similar to a Promise except it is run only when we call fork.

    const twelve = new Task((reject, resolve) => {
      resolve(12)
    })
    .fork(
      err => console.log(“Cannot compute 12”),
      twelve => console.log(“We got “ + twelve)
    )
    

    Tasks are thus pure and lazy in nature, and this has huge consequences.

    A Promise can be thought of as the result of the computation. You either already have the result (the Promise has been resolved) or the value is about to be delivered. When you have a reference to a Promise the side effect has already been run and we are just waiting for the result to arrive.

    A Task on the other hand represents the computation itself. We can combine Tasks in interesting ways and pass them around, yet no side effects have been run until we call fork. In other words: Task is a side effect we can pass around. It’s side effects as data.

    Side effects as data

    Since Tasks are computations represented as values, we can pass them around, combine and sequence them. We can wrap computations in other computations and we can pass computations into other functions and decide later if and when to run them. I’ll explain this with a few examples.

    Let’s say we are writing some code to interact with a SQL database and we have a Promise which makes database updates that we want to run in a transaction. We need a function like this:

    wrapInTransaction :: Promise a -> Promise a
    

    It’s a function that takes a Promise and returns a new Promise. That is, we want that function to wrap our update statements in a transaction and give us back a Promise which resolves if the transaction is committed. If the transaction fails, the Promise is rejected.

    However, there’s no way to implement that function. Why? The Promise that we pass into the function, the one that makes the updates to the database, could already have been resolved before we try to wrap it in a transaction. Remember, Promise represents the possible result of the computation that is already “in flight”. We cannot wrap it in a transaction because it’s already running the query.

    We have no problem implementing that function using a Task instead. The implementation looks something like this:

    const withTransaction = task =>
      db.query(‘BEGIN TRANSACTION’)
      .chain(const(task))
      .chain(const(db.query(‘COMMIT’)))
      .orElse(const(db.query(‘ROLLBACK’)))
    

    Notice how we are taking the task as a parameter and injecting it into the chain of other computations, yet it has not been run until the whole returned computation is forked.

    Controlling the order of computations

    A problem that I (and many others) often face is having an array of items and for each of them you want to perform an asynchronous operation. This can be for example an ajax request. Often you want to do these requests in sequence, so that the next request starts only after the previous one has finished. What we know about promises by now is that creating a Promise will also trigger the request, right then and there. That means the looping of the items and making the requests are tied with the promise creation. What does that mean?

    It means we need a special helper function specialized to Promises to simultaneously loop and fire the requests. The standard Promises API doesn’t even have such a function, but for example Bluebird does.

    Tasks can deal with this much more elegantly. We can first map each id to a Task and then later use sequence from ramda to collect the results from each Task into an array. We can also further map over the Tasks and later decide to run them in parallel instead if we prefer.

    Side note: traverse would work here as well but I find myself using sequence more often, and it’s a bit simpler.

    The important thing to note is that both map and sequence know nothing about Tasks and are pure functions that just operate on data. We have solved the problem using nothing but very simple and pure functions both found in ramda (could have used Array.prototype.map for mapping as well). We have also been able to divide the problem into two composable pieces — mapping and sequencing — which makes reasoning about the code much easier. The implementation of Bluebirds each in turn delegates to Bluebird’s reduce whose implementation is 178 lines long, and is very specific to Promises.

    Because Tasks are data, just like any other data structures, the same rules apply to them. We can use the same powerful concept of manipulating data with pure functions and apply it to side effects.

    Conclusions

    • Promises are the go-to solution for getting rid of callback hell, but there are serious limitations to the abstraction.
    • Tasks are pure and lazy, you have to explicitly run them to execute any side effects. This means that you can have side effects in your code but keep using referentially transparent functions and reason about your code much more easily.
    • Tasks allow you to control the execution order of computations. Promises are always “in flight” and extra code is needed to defer their execution.
    • Because Tasks are pure, it’s much easier to write utilities and libraries for them.

    A note about data.task v2: There is a new version of data.task which has better support for cancellation and resource handling. I haven’t used it yet but you should definitely check it out if you are wondering which library to use.