Go JSON (Un)Marshalling, Missing Fields and Omitempty

When working with JSON in Go recently, I wondered: “What the heck does json:omitempty do?”. I was looking for ways to handle missing fields while unmarshalling some JSON into a struct, and got confused for a while.

Here are a few examples, as a reminder how most of it works.

Those are not fancy edgecases. Those are self-contained, completely functional code snippets which should be easy to follow. You’re welcome future-me.

Just parse some JSON

You provide field names as annotations. If the JSON payload matches the struct 1:1, it doesn’t get more complicated than this:

package main

import (
        "encoding/json"
        "fmt"
        "os"
)

type Dummy struct {
        Name    string  `json:"name"`
        Number  int64   `json:"number"`
        Pointer *string `json:"pointer"`
}

func main() {
        data := []byte(`
            {
                "name": "Mr Dummy",
                "number": 4,
                "pointer": "yes"
            }
        `)

        var dummy Dummy
        err := json.Unmarshal(data, &dummy)
        if err != nil {
                fmt.Println("An error occured: %v", err)
                os.Exit(1)
        }

	// we want to print the field names as well
        fmt.Printf("%+v\n", dummy)
}

The output is a bit ugly, but it’s good enough to see what’s going on:

{Name:Mr Dummy Number:4 Pointer:0xc00000e280}

All fields got used. The pointer is non-nil, it points to a string value (you can’t see here, but the pointed-to string contains “yes”).

So far so good.

Omit fields when parsing JSON

You got way too much stuff in the JSON data. You don’t need all of it.

Nothing fancy needed here. Just don’t specify those extra fields in your struct. Go won’t mind.

In fact, you could use the same JSON data with multiple small structs to get out parts of the data one after the other.

Cool trick: you can parse the same JSON using multiple structs, to adjust to different contents dynamically this way.

package main

import (
        "encoding/json"
        "fmt"
        "os"
)

// let's change the struct:
type Dummy struct {
	// we only care about the name now
        Name    string  `json:"name"`
	// all other fields are gone
}

func main() {
        data := []byte(`
            {
                "name": "Mr Dummy",
                "number": 4,
                "pointer": "yes"
            }
        `)

        var dummy Dummy
        err := json.Unmarshal(data, &dummy)
        if err != nil {
                fmt.Println("An error occured: %v", err)
                os.Exit(1)
        }

	// we want to print the field names as well
        fmt.Printf("%+v\n", dummy)
}

Output:

{Name:Mr Dummy}

Just one field. Nothing else got used. Neat!

Ignore JSON fields when parsing it, even though your struct has them

Here’s where the json:"-" annotation comes in handy. You tell Go to leave this particular struct field alone, when pumping values from JSON into it.

package main

import (
        "encoding/json"
        "fmt"
        "os"
)

type Dummy struct {
        Name    string  `json:"name"`
        Number  int64   `json:"number"`
        Pointer *string `json:"-"` // we want to ignore JSON for this one
}

func main() {
        data := []byte(`
            {
                "name": "Mr Dummy",
                "number": 4,
                "pointer": "yes"
            }
        `)

        var dummy Dummy
        err := json.Unmarshal(data, &dummy)
        if err != nil {
                fmt.Println("An error occured: %v", err)
                os.Exit(1)
        }

	// we want to print the field names as well
        fmt.Printf("%+v\n", dummy)
}

Output:

{Name:Mr Dummy Number:4 Pointer:<nil>}

Even though the JSON specifies something for the pointer field, our structs ignores the value and leaves its Pointer unchanged (at nil).

Don’t output fields when writing out JSON

json:"-" once again! It works both ways.

Even though the struct has the field, we won’t see it in the JSON output.

package main

import (
        "encoding/json"
        "fmt"
        "os"
)

type Dummy struct {
        Name    string  `json:"name"`
        Number  int64   `json:"number"`
        Pointer *string `json:"-"`
}

func main() {
        pointer := "yes"

        dummy := Dummy{
                Name: "Mr Dummy",
                Number: 4,
                Pointer: &pointer,
        }

        data, err := json.Marshal(dummy)
        if err != nil {
                fmt.Println("An error occured: %v", err)
                os.Exit(1)
        }

        fmt.Println(string(data))
}

Output:

{"name":"Mr Dummy","number":4}

See ma? No pointer field.

Leave out fields if they are empty when writing out JSON

json:omitempty to the rescue. We only want to ignore the field of the struct if it’s empty.

Now there’s a gotcha here: you gotta be pretty sure what Go takes as empty.

package main

import (
        "encoding/json"
        "fmt"
        "os"
)

type Dummy struct {
        Name    string  `json:"name,omitempty"`
        Number  int64   `json:"number,omitempty"`
        Pointer *string `json:"pointer,omitempty"`
}

func main() {
        pointer := "yes"

        dummyComplete := Dummy{
                Name:    "Mr Dummy",
                Number:  4,
                Pointer: &pointer,
        }

        data, err := json.Marshal(dummyComplete)
        if err != nil {
                fmt.Println("An error occured: %v", err)
                os.Exit(1)
        }

        fmt.Println(string(data))

	// ALL of those are considered empty by Go
        dummyNilPointer := Dummy{
                Name:    "",
                Number:  0,
                Pointer: nil,
        }

        dataNil, err := json.Marshal(dummyNilPointer)
        if err != nil {
                fmt.Println("An error occured: %v", err)
                os.Exit(1)
        }

        fmt.Println(string(dataNil))
}

Output

{"name":"Mr Dummy","number":4,"pointer":"yes"}
{}

The second struct is completely empty. All fields are left out due to their values thanks to the omitempty annotation.

Zero-values and nil pointers are considered empty, and this can feel counterintuitive sometimes. If you want to output a 0, you better work with something like *int64, if you really want to use omitempty.

But there’s no need to make it trickier than it needs to be. Everything else is pretty much straightforward.

Additional Resources

If you want to read up more on this topic, and get details for more tricky edgecases, you can check out those links:

Happy (Un)Marshalling!

Newsletter

It's the really early of this site! There is no automated newsletter signup yet. But if you want to stay posted already (thanks!), send a mail to mail@vsupalov.com. Just say hi and that you'd like to get notified about new Gophed Dojo articles. There's also an RSS link in the footer if you prefer!

gopherdojo.com

© 2024 gopherdojo.com. All rights reserved.