Unmarshal unknown types

Unmarshal unknown types

Published

A simple custom unmarshaler example.

So we have this small dog struct:

type dog struct {
Name string
}

With this we are gonna get random dog names from a dog name api. Unfortunately, this random dog name api is not a good one, and the response we get can come as a string, object or an array.

So let us create a new type:

type dogs []dog

This is the type we are going to use when we are unmarsheling the response from the api.

First we create our custom UnmarshalJSON function:

func (d *dogs) UnmarshalJSON(b []byte) error {}

Now, whenever we call a unmarshal function on the dogs type, it will call our function.

Next, let us create a test function for our new custom unmarshal function:

NB! If you use VS Code and the golang extension this test can be generated for you. The only difference is I have added what struct we expect in return.

func TestDogs_UnmarshalJSON(t *testing.T) {
type args struct {
b []byte
}

type want struct {
err bool
dd dogs
}

tests := []struct {
name string
dd *dogs
args args
want want
}
}

Breakdown:

  • name is just to identify which test it is that is running.
  • args is the []byte slice we provide to our function.
  • wantis what we want in return
  • tests is all the different kind of input types we wanna test.

Our first test looks like this:

{
name: "array",
dd: &dogs{},
args: args{
b: []byte(`[{"name": "Stan"},{"name": "Elliot"}]`),
},
want: want{
err: false,
dd: dogs{
{
Name: "Stan",
},
{
Name: "Elliot",
},
},
},
}

To make this test pass, our unmarshal function can be quite straight forward:

func (d *dogs) UnmarshalJSON(b []byte) error {
var dd []dog

err := json.Unmarshal(b, &dd)
if err != nil {
return err
}

// here we point to our newly unmarshaled []dog variable.
*d = dd
return nil
}

Now we can add our second test:

{
name: "string",
dd: &dogs{},
args: args{
b: []byte(`"[{"name": "Stan"},{"name": "Elliot"}]"`),
},
want: want{
err: false,
dd: dogs{
{
Name: "Stan",
},
{
Name: "Elliot",
},
},
},
},

When we run our tests again we will see this this error:

--- FAIL: TestDogs_UnmarshalJSON (0.00s)
--- FAIL: TestDogs_UnmarshalJSON/string (0.00s)
main_test.go:80: Dogs.UnmarshalJSON() error = invalid character 'n' after top-level value, wantErr false

This means our response is in the shape of a string.

One way around this is to change the byte slice before we try to unmarshal it.

switch b[0] {
case '"':
b = b[1 : len(b)-1]

Here we check if the first character in the slice is a quote, if it is we remove the first and the last character in the string. To make our code still work for the first test, we add a case for that as well.

switch b[0] {
case '"':
b = b[1 : len(b)-1]
if b[0] == '[' {
err := json.Unmarshal(b, &dd)
if err != nil {
return err
}
} else if b[0] == '{' {
var doggie dog
err := json.Unmarshal(b, &doggie)
if err != nil {
return err
}

dd = append(dd, doggie)
}
case '[':
err := json.Unmarshal(b, &dd)
if err != nil {
return err
}
}
*d = dd
return nil
}

Now we can our third test:

{
name: "object",
dd: &dogs{},
args: args{
b: []byte(`{"name": "Stan"}`),
},
want: want{
err: false,
dd: dogs{
{
Name: "Stan",
},
},
},
}

When we run that we get a new error:

--- FAIL: TestDogs_UnmarshalJSON (0.00s)
--- FAIL: TestDogs_UnmarshalJSON/object (0.00s)
main_test.go:95: Dogs.UnmarshalJSON() error = json: cannot unmarshal object into Go value of type []main.dog, wantErr false

This fails because we are trying to unmarshal an object as an array. To pass this test we add our third case:

case '{':
var d dog
err := json.Unmarshal(b, &dd)
if err != nil {
return err
}

dd = append(dd, d)

Now we are able to handle a single object as well.

All that is left now is to handle the "correct" response from the api:

case '[':
err := json.Unmarshal(b, &dd)
if err != nil {
return err
}

Now our Unmarshal function is able to handle pretty much everything that the api can throw at it.

Here is a complete example of our code:

type dog struct {
Name string
}

type dogs []dog

func (d *dogs) UnmarshalJSON(b []byte) error {
var dd []dog

switch b[0] {
case '"':
// remove first and last "
b = b[1 : len(b)-1]
if b[0] == '[' {
err := json.Unmarshal(b, &dd)
if err != nil {
return err
}
} else if b[0] == '{' {
var doggie dog
err := json.Unmarshal(b, &doggie)
if err != nil {
return err
}

dd = append(dd, doggie)
}
case '{':
var d dog
err := json.Unmarshal(b, &d)
if err != nil {
return err
}

dd = append(dd, d)
case '[':
err := json.Unmarshal(b, &dd)
if err != nil {
return err
}
}

*d = dd
return nil
}

And our tests:

func TestDogs_UnmarshalJSON(t *testing.T) {
type args struct {
b []byte
}

type want struct {
err bool
dd dogs
}

tests := []struct {
name string
dd *dogs
args args
want want
}{
{
name: "array",
dd: &dogs{},
args: args{
b: []byte(`[{"name": "Stan"},{"name": "Elliot"}]`),
},
want: want{
err: false,
dd: dogs{
{
Name: "Stan",
},
{
Name: "Elliot",
},
},
},
},
{
name: "string array",
dd: &dogs{},
args: args{
b: []byte(`"[{"name": "Stan"},{"name": "Elliot"}]"`),
},
want: want{
err: false,
dd: dogs{
{
Name: "Stan",
},
{
Name: "Elliot",
},
},
},
},
{
name: "object",
dd: &dogs{},
args: args{
b: []byte(`{"name": "Stan"}`),
},
want: want{
err: false,
dd: dogs{
{
Name: "Stan",
},
},
},
},
{
name: "string object",
dd: &dogs{},
args: args{
b: []byte(`"{"name": "Stan"}"`),
},
want: want{
err: false,
dd: dogs{
{
Name: "Stan",
},
},
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := tt.dd.UnmarshalJSON(tt.args.b)
if (err != nil) != tt.want.err {
t.Errorf("Dogs.UnmarshalJSON() error = %v, wantErr %v", err, tt.want.err)
} else if got := *tt.dd; !reflect.DeepEqual(*tt.dd, got) {
t.Errorf("Dogs.UnmarshalJSON() got = %v, want %v", got, tt.want.dd)
}
})
}

}

Thank you for reading.