Value Space Decoding for Aeson Implementation of combinators for JSON decoding in Haskell

I’ve learned to like the way JSON decoding works in Elm¹. I see some advantages of using combinators to decode values from JSONs. In some situations, I would very much prefer this style of decoding over type class instances used by Aeson.

Conceptual Model of Type Classes

Type classes allow us to define that type is a member of larger groups of types (classes). Or the other way around, to define classes of data types. The important aspect, though, is that given specific type there is exactly one way in which it can be made an instance of the class. Even type classes like Applicative “suffer” from this limitation. List [a] data type, for instance, has two completely valid, but different, implementations of Applicative instance. Let me be clear, I’m not saying this is a huge problem in cases like [a]. But this aspect will come back to haunt us, even more, when we start using type classes for JSON decoding where it is even less clear to know what we mean by “instance of FromJSON”. Edward Kmett had a briliant talk about type classes in Boston Haskell Meetup so don’t forget to check it out if you haven’t.

I hope I’ve managed to establish that with classes (like Aeson’s FromJSON) we’re essentially closing the JSON decoding implementation over types. I think that this, in many cases, is not a good model for translating data between different representations. A much better way would be to allow for an arbitrary number of functions from JSON values to the same type. It’s fair to assume that the same data type can be decoded from different JSONs structures (problematic with type classes) as it is to assume that the same JSON structure can be decoded to multiple data types (no problem with classes).

Because of this, it’s a common practice to define type wrappers around existing data types just to make it possible to define “another” instance of FromJSON around the type. While this solution works, I personally don’t find it satisfying. To me, all these types are just unnecessary noise. I don’t like to have types which are, in my opinion at least, not important in the conceptual model of my domain at all.

Solution

What I would like to have is not a complete replacement for FromJSON class. There are in fact cases where it makes sense to have a single definition of a mapping between JSON and data type. So I still want to be able to define FromJSON instance but I also want an option to define “decoders” not as an instance of a class of some type but as a value.

What I mean I that is to be able to do something like this (ACD being our library):

data Person = Person {
      name :: Text
    , age  :: Int
    } deriving (Generic, Show)

instance FromJSON Person

decodeEmbeded :: FromJSON a => [Text] -> ByteString -> Maybe a
decodeEmbeded path json =
    ACD.decode (ACD.at path ACD.auto) json

Which can be used to extract Person embedded in any JSON structure as following:

>>> decodeEmbeded ["data", "person"] "{\"data\":{\"person\":{\"name\":\"Joe\",\"age\":12}}}"
Just (Person {name = "Joe", age = 12})

Or I want to be able to define anonymous product (tuple) decoder:

type Token = Text -- using alias for simplicity

decodePersonWithToken :: ByteString -> Maybe (Token, Person)
decodePersonWithToken json = ACD.decode decoder json
    where decoder =
            (,) <$> ACD.field "token" ACD.auto
                <*> ACD.field "person" ACD.auto

which works as following:

>>> decodePersonWithToken "{\"person\":{\"name\":\"Joe\",\"age\":12},\"token\":\"foo\"}"
Just ("foo",Person {name = "Joe", age = 12})

Implementation

The idea is to be able to define Decoder a type which is essentially just parseJSON method from FromJSON class. Since we want to make this work with Aeson’s type classes without introducing more than necessary overhead, we simply just wrap its member function parseJSON to a newtype:

newtype Decoder a =
  Decoder (Value -> Parser a)

The simplest constructor of this type is from types which are member of FromJSON class:

auto :: FromJSON a => Decoder a
auto = Decoder parseJSON

I don’t want to spent much time defining other constructors as this alone provides enough but as an example this is how we can easily define constructor which turns any Decoder a to Decoder [a]:

list :: Decoder a -> Decoder [a]
list (Decoder d) = Decoder $ listParser d

where listParser is a function provided by Aeson itself.

To make Decoder more useful we’re going to define instances of Functor, Applicative and Monad which should be enough for providing most of important functionality.

instance Functor Decoder where
  fmap f (Decoder d) = Decoder $ fmap f . d

instance Applicative Decoder where
  pure val = Decoder $ \_ -> pure val
  (Decoder f') <*> (Decoder d) = Decoder $
    \val ->
        (\f -> fmap f (d val)) =<< f' val

instance Monad Decoder where
  (Decoder a) >>= f = Decoder $
    \val -> case parse a val of
      Success v -> let (Decoder res) = f v
                   in res val
      _ -> unexpected val

And finally some combinators specific for JSON. We need a function then can extract value from JSON filed. And then we can define another function for “drilling” a few fields deep into JSON Object.

field :: Text -> Decoder a -> Decoder a
field t (Decoder d) = Decoder $
  \val -> case val of
    Object v -> d =<< v .: t
    _        -> typeMismatch "Object" val

at :: [Text] -> Decoder a -> Decoder a
at path d =
  foldr field d path

Once again this is using functions already provided by Aeson.

Last step is to define new decode which will work with Decoder.

decode :: Decoder a -> LB.ByteString -> Maybe a
decode (Decoder d) =
  Parser.decodeWith ParserI.jsonEOF (parse d)

All functions used here are provided by Aeson. LB is a lazy version of ByteString.

And this is all we need to make examples from section above working.

Elm Style Decoding

It’s of course possible to use this Decoder type exclusively. This makes writing Aeson decoders feel like elm/json.

data Person = Person {
      name :: Text
    , age  :: Int
    } deriving (Show)

personDecoder :: Decoder Person
personDecoder =
    Person
        <$> field "name" auto
        <*> field "age" auto

Additional Resources

Full implementation of this Idea can be found in this GitHub repository. Feel free to provide any feedback including criticism. Just be aware that this is not published and is still missing some important bits.

Waargonaut package is Aeson alternative which has API similar to our Decoder approach if you want to avoid whole Aeson.

While writing this post I’ve discovered that Chris Martin had a similar idea implemented in aeson-decode package but the actual implementation uses Value -> Maybe a function similar to my original prototype.