By Ugorji Nwoke   16 Dec 2014 (updated 01 Jul 2019)   /blog   technology go-codec

Yet Another JSON Library? But Why?

View articles in the go-codec series, source at

And let me answer …

The encoding/json library bundled with the standard library is a GREAT piece of software.

The reasons why I decided to write a json library are:

  1. decode from a multi-part stream.
    encoding/json would read many bytes from the stream into an internal buffer, and then incrementally decode from that internal buffer.

    This makes it hard to parse some json string from a stream, and then continue reading other values in other formats from the stream. A potential use-case here is a multipart stream which contains some text in json, some images in gif format, etc. I had a usecase similar to this.

    encoding/json has a kludge where it exposes a Buffered() method so users can gain access to the bytes it read but didn’t use. That is hard to work with.

    The go-codec json library would only read the data it needs from the stream.

  2. Encode/Decode from/to chan type (for iterative streaming support).
    A use-case involves a json array containing millions of values. For efficiency, they should be processed concurrently as the values are being decoded. The idiomatic way is to send the values into a chan, and process them one-by-one in a different goroutine.

    encoding/json does not support chan types. go-codec allows you easily encode a chan as an array in json, or decode from an array in json into a chan natively, elegantly solving this use-case.

  3. ability to decode a number as either a float64, int64 or uint64.
    Currently, encoding/json only decodes as float64, and you need to use the hatch “UseNumber()” to keep it as json.Number.

    This made things hard to work with, as you have to do a second parse to convert the json.Number to an appropriate value. It would be nice if the decoder could determine the best “type” to use e.g. decode 789 as uint64, -789 as int64, and -78.9 as float64.

  4. avoid allocation when parsing numbers.
    encoding/json uses strconv.ParseFloat when decoding a number.

    go-codec parses the number while reading the stream, and not as a subsequent out-of-band action.

  5. leverage all the performance enhancements of the go-codec library.
    The go-codec library has some very impressive performance attributes. We wanted to leverage it for json encoding and decoding.

    See why go-codec has stellar performance

  6. Support for fully-featured code generation for more performance.
    encoding/json only works with runtime introspection (reflection). However, it is shown that you could get over a 2X performance improvement by static encoding/decoding when the structure of the types are known at compile time.

    See code generation support from go-codec

  7. Drop-in replacement for encoding/json. json: key in struct tag supported.
    encoding/json uses the json: key in the struct tag value to configure how the struct is encoded. go-codec will use the json key as a fallback, if the codec key is unavailable in the struct tag value.

    This allows go-codec to be used as a drop-in replacement for encoding/json without having to make changes to the structs.

The performance numbers speak for themselves. When we introduce code generation support, the numbers are even more stellar.

Raw Data:

Benchmark__Json_______Encode-8         	    6051	    188551 ns/op	    3256 B/op	      44 allocs/op
Benchmark__Std_Json___Encode-8         	    5514	    218973 ns/op	   74474 B/op	     444 allocs/op

Benchmark__Json_______Encode-8         	    8064	    141844 ns/op	     352 B/op	       2 allocs/op

Benchmark__Json_______Decode-8         	    3105	    390624 ns/op	   89300 B/op	    1041 allocs/op
Benchmark__Std_Json___Decode-8         	    1365	    855218 ns/op	  138558 B/op	    3032 allocs/op

Benchmark__Json_______Decode-8         	    4257	    283405 ns/op	   87471 B/op	    1002 allocs/op

The table below shows how much more time, memory allocated and number of memory allocations is used by encoding/json compared to the go-codec json library. The improvements are clear:

Time Memory Allocations
Encoding (runtime) 1.16 X 22.87 X 10.1 X
Encoding (code generation) 1.55 X 211.6 X 222 X
Decoding (runtime) 2.18 X 1.55 X 2.9 X
Decoding (code generation) 3.02 X 1.58 X 3.03 X

The beauty of this is that encoding and decoding is just as feature-rich as the encoding/json package. go-codec builds more features and performance beyond what is provided by the encoding/json package.

Tags: technology go-codec

Subscribe: Technology
© Ugorji Nwoke