Unmarshal unknown types

Unmarshal unknown types

Posted

So we have this small dog struct:

type dog struct {
	Name string
}

With this we are gonna get random dog names from a dog name api. Unfortunately, this random dog name api is not a good one, and the response we get can come as a string, object or an array.

So let us create a new type:

type dogs []dog

This is the type we are going to use when we are unmarsheling the response from the api.

First we create our custom UnmarshalJSON function:

func (d *dogs) UnmarshalJSON(b []byte) error {}

Now, whenever we call a unmarshal function on the dogs type, it will call our function.

Next, let us create a test function for our new custom unmarshal function:

NB! If you use VS Code and the golang extension this test can be generated for you. The only difference is I have added what struct we expect in return.

func TestDogs_UnmarshalJSON(t *testing.T) {
	type args struct {
		b []byte
	}

	type want struct {
		err bool
		dd  dogs
	}

	tests := []struct {
		name string
		dd   *dogs
		args args
		want want
	}
}

Breakdown:

  • name is just to identify which test it is that is running.
  • args is the []byte slice we provide to our function.
  • wantis what we want in return
  • tests is all the different kind of input types we wanna test.

Our first test looks like this:

{
	name: "array",
	dd:   &dogs{},
	args: args{
		b: []byte(`[{"name": "Stan"},{"name": "Elliot"}]`),
	},
	want: want{
		err: false,
		dd: dogs{
			{
				Name: "Stan",
			},
			{
				Name: "Elliot",
			},
		},
	},
}

To make this test pass, our unmarshal function can be quite straight forward:

func (d *dogs) UnmarshalJSON(b []byte) error {
	var dd []dog

	err := json.Unmarshal(b, &dd)
	if err != nil {
		return err
	}

	// here we point to our newly unmarshaled []dog variable.
	*d = dd
	return nil
}

Now we can add our second test:

{
	name: "string",
	dd:   &dogs{},
	args: args{
		b: []byte(`"[{"name": "Stan"},{"name": "Elliot"}]"`),
	},
	want: want{
		err: false,
		dd: dogs{
			{
				Name: "Stan",
			},
			{
				Name: "Elliot",
			},
		},
	},
},

When we run our tests again we will see this this error:

--- FAIL: TestDogs_UnmarshalJSON (0.00s)
	--- FAIL: TestDogs_UnmarshalJSON/string (0.00s)
			main_test.go:80: Dogs.UnmarshalJSON() error = invalid character 'n' after top-level value, wantErr false

This means our response is in the shape of a string.

One way around this is to change the byte slice before we try to unmarshal it.

switch b[0] {
	case '"':
		b = b[1 : len(b)-1]

Here we check if the first character in the slice is a quote, if it is we remove the first and the last character in the string. To make our code still work for the first test, we add a case for that as well.

switch b[0] {
	case '"':
		b = b[1 : len(b)-1]
		if b[0] == '[' {
			err := json.Unmarshal(b, &dd)
			if err != nil {
				return err
			}
		} else if b[0] == '{' {
			var doggie dog
			err := json.Unmarshal(b, &doggie)
			if err != nil {
				return err
			}

			dd = append(dd, doggie)
		}
	case '[':
		err := json.Unmarshal(b, &dd)
		if err != nil {
			return err
		}
	}
*d = dd
return nil
}

Now we can our third test:

{
	name: "object",
	dd:   &dogs{},
	args: args{
		b: []byte(`{"name": "Stan"}`),
	},
	want: want{
		err: false,
		dd: dogs{
			{
				Name: "Stan",
			},
		},
	},
}

When we run that we get a new error:

--- FAIL: TestDogs_UnmarshalJSON (0.00s)
    --- FAIL: TestDogs_UnmarshalJSON/object (0.00s)
        main_test.go:95: Dogs.UnmarshalJSON() error = json: cannot unmarshal object into Go value of type []main.dog, wantErr false

This fails because we are trying to unmarshal an object as an array. To pass this test we add our third case:

case '{':
	var d dog
	err := json.Unmarshal(b, &dd)
	if err != nil {
		return err
	}

	dd = append(dd, d)

Now we are able to handle a single object as well.

All that is left now is to handle the “correct” response from the api:

case '[':
		err := json.Unmarshal(b, &dd)
		if err != nil {
			return err
		}

Now our Unmarshal function is able to handle pretty much everything that the api can throw at it.

Here is a complete example of our code:

type dog struct {
	Name string
}

type dogs []dog

func (d *dogs) UnmarshalJSON(b []byte) error {
	var dd []dog

	switch b[0] {
	case '"':
		// remove first and last "
		b = b[1 : len(b)-1]
		if b[0] == '[' {
			err := json.Unmarshal(b, &dd)
			if err != nil {
				return err
			}
		} else if b[0] == '{' {
			var doggie dog
			err := json.Unmarshal(b, &doggie)
			if err != nil {
				return err
			}

			dd = append(dd, doggie)
		}
	case '{':
		var d dog
		err := json.Unmarshal(b, &d)
		if err != nil {
			return err
		}

		dd = append(dd, d)
	case '[':
		err := json.Unmarshal(b, &dd)
		if err != nil {
			return err
		}
	}

	*d = dd
	return nil
}

And our tests:

func TestDogs_UnmarshalJSON(t *testing.T) {
type args struct {
b []byte
}

    type want struct {
    	err bool
    	dd  dogs
    }

    tests := []struct {
    	name string
    	dd   *dogs
    	args args
    	want want
    }{
    	{
    		name: "array",
    		dd:   &dogs{},
    		args: args{
    			b: []byte(`[{"name": "Stan"},{"name": "Elliot"}]`),
    		},
    		want: want{
    			err: false,
    			dd: dogs{
    				{
    					Name: "Stan",
    				},
    				{
    					Name: "Elliot",
    				},
    			},
    		},
    	},
    	{
    		name: "string array",
    		dd:   &dogs{},
    		args: args{
    			b: []byte(`"[{"name": "Stan"},{"name": "Elliot"}]"`),
    		},
    		want: want{
    			err: false,
    			dd: dogs{
    				{
    					Name: "Stan",
    				},
    				{
    					Name: "Elliot",
    				},
    			},
    		},
    	},
    	{
    		name: "object",
    		dd:   &dogs{},
    		args: args{
    			b: []byte(`{"name": "Stan"}`),
    		},
    		want: want{
    			err: false,
    			dd: dogs{
    				{
    					Name: "Stan",
    				},
    			},
    		},
    	},
    	{
    		name: "string object",
    		dd:   &dogs{},
    		args: args{
    			b: []byte(`"{"name": "Stan"}"`),
    		},
    		want: want{
    			err: false,
    			dd: dogs{
    				{
    					Name: "Stan",
    				},
    			},
    		},
    	},
    }
    for _, tt := range tests {
    	t.Run(tt.name, func(t *testing.T) {
    		err := tt.dd.UnmarshalJSON(tt.args.b)
    		if (err != nil) != tt.want.err {
    			t.Errorf("Dogs.UnmarshalJSON() error = %v, wantErr %v", err, tt.want.err)
    		} else if got := *tt.dd; !reflect.DeepEqual(*tt.dd, got) {
    			t.Errorf("Dogs.UnmarshalJSON() got = %v, want %v", got, tt.want.dd)
    		}
    	})
    }

}

Thank you for reading.