Yet Another JSON Library? But Why?

View articles in the go-codec series, source at http://github.com/ugorji/go

And let me answer …

The encoding/json library bundled with the standard library is a GREAT piece of software.

The reasons why I decided to write a json library are:

1. decode from a multi-part stream.
   encoding/json would read many bytes from the stream into an internal buffer, and then incrementally decode from that internal buffer.

   This makes it hard to parse some json string from a stream, and then continue reading other values in other formats from the stream. A potential use-case here is a multipart stream which contains some text in json, some images in gif format, etc. I had a usecase similar to this.

   encoding/json has a kludge where it exposes a Buffered() method so users can gain access to the bytes it read but didn’t use. That is hard to work with.

   The go-codec json library would only read the data it needs from the stream.

2. Encode/Decode from/to chan type (for iterative streaming support).
   A use-case involves a json array containing millions of values. For efficiency, they should be processed concurrently as the values are being decoded. The idiomatic way is to send the values into a chan, and process them one-by-one in a different goroutine.

   encoding/json does not support chan types. go-codec allows you easily encode a chan as an array in json, or decode from an array in json into a chan natively, elegantly solving this use-case.

3. ability to decode a number as either a float64, int64 or uint64.
   Currently, encoding/json only decodes as float64, and you need to use the hatch “UseNumber()” to keep it as json.Number.

   This made things hard to work with, as you have to do a second parse to convert the json.Number to an appropriate value. It would be nice if the decoder could determine the best “type” to use e.g. decode 789 as uint64, -789 as int64, and -78.9 as float64.

4. avoid allocation when parsing numbers.
   encoding/json uses strconv.ParseFloat when decoding a number.

   go-codec parses the number while reading the stream, and not as a subsequent out-of-band action.

5. leverage all the performance enhancements of the go-codec library.
   The go-codec library has some very impressive performance attributes. We wanted to leverage it for json encoding and decoding.

   See why go-codec has stellar performance

6. Support for fully-featured code generation for more performance.
   encoding/json only works with runtime introspection (reflection). However, it is shown that you could get over a 2X performance improvement by static encoding/decoding when the structure of the types are known at compile time.

   See code generation support from go-codec

7. Drop-in replacement for encoding/json. json: key in struct tag supported.
   encoding/json uses the json: key in the struct tag value to configure how the struct is encoded. go-codec will use the json key as a fallback, if the codec key is unavailable in the struct tag value.

   This allows go-codec to be used as a drop-in replacement for encoding/json without having to make changes to the structs.

The performance numbers speak for themselves. When we introduce code generation support, the numbers are even more stellar.

Raw Data:

ENCODE (RUNTIME)
Benchmark__Json_______Encode-8         	    6051	    188551 ns/op	    3256 B/op	      44 allocs/op
Benchmark__Std_Json___Encode-8         	    5514	    218973 ns/op	   74474 B/op	     444 allocs/op

ENCODE (CODE GENERATION)
Benchmark__Json_______Encode-8         	    8064	    141844 ns/op	     352 B/op	       2 allocs/op

DECODE (RUNTIME)
Benchmark__Json_______Decode-8         	    3105	    390624 ns/op	   89300 B/op	    1041 allocs/op
Benchmark__Std_Json___Decode-8         	    1365	    855218 ns/op	  138558 B/op	    3032 allocs/op

DECODE (CODE GENERATION)
Benchmark__Json_______Decode-8         	    4257	    283405 ns/op	   87471 B/op	    1002 allocs/op

The table below shows how much more time, memory allocated and number of memory allocations is used by encoding/json compared to the go-codec json library. The improvements are clear:

The beauty of this is that encoding and decoding is just as feature-rich as the encoding/json package. go-codec builds more features and performance beyond what is provided by the encoding/json package.