Example for nested structures and rendering only a nested struct

[skandragon]

My use case is the Square API, where I want to retrieve a list of items, and iterate over some internal data, updating only the internal objects, not the main one.

The data is structured like this:

type ApiWrapper struct {
    Cursor       string                 `json:"cursor,omitempty"`
    Objects      []*Object              `json:"objects,omitempty"`
}

type Object struct {
    Type              string                 `json:"type,omitempty"`
    Id                string                 `json:"id,omitempty"`
    UpdatedAt         time.Time              `json:"updated_at,omitempty"`
    CreatedAt         time.Time              `json:"created_at,omitempty"`
    Version           int64                  `json:"version,omitempty"`
    IsDeleted         bool                   `json:"is_deleted,omitempty"`
    ItemData          *Item                  `json:"item_data,omitempty"`
    ItemVariationData *ItemVariation         `json:"item_variation_data,omitempty"`
}

type Item struct {
    Name         string                 `json:"name,omitempty"`
    Description  string                 `json:"description,omitempty"`
    Visibility   string                 `json:"visibility,omitempty"`
    Variations   []*Object              `json:"variations,omitempty"`
}

type ItemVariation struct {
    ItemId       string                 `json:"item_id,omitempty"`
    SKU          string                 `json:"sku,omitempty"`
}

In this case, I want to fetch a list of ITEMs from the API, each of which will be of Type ITEM, and have ItemData set. In the ItemData, it will have a list of variations, each of which is an Object of type ITEM_VARIATION, with ItemVariationData set.

What I want to do here is modify each of the ItemVariationData items, and then render the Object holding that variation's data.

I've tried implementing HandleJSONData() such as:

func (e *Item) HandleJSONData(data map[string]interface{}) {
    e.InternalData = data
}

and then using something like this to render the Object out:

func spewJSON(m map[string]any, obj any) {
    structOut, err := json.Marshal(obj)
    if err != nil {
        panic(err)
    }

    err = json.Unmarshal(structOut, &m)
    if err != nil {
        panic(err)
    }

    out, err := json.MarshalIndent(m, "", "\t")
    if err != nil {
        panic(err)
    }

    fmt.Println(string(out))
}

func main() {
    for _, itemObject := range result.Objects {
        for _, variationObject := range itemObject.ItemData.Variations {
            if variationObject.ItemVariationData.SKU == "" {
                sku := makeSku()
                log.Printf("item %s, variation %s, sku was empty, is now %s", variationObject.ItemVariationData.ItemId, variationObject.Id, sku)
                variationObject.ItemVariationData.SKU = sku
            } else {
                log.Printf("item %s, variation %s, sku set to %s", variationObject.ItemVariationData.ItemId, variationObject.Id, variationObject.ItemVariationData.SKU)
            }
            spewJSON(variationObject.InternalData, variationObject)
        }
    }
}

However, while all the "unknown" fields in the Object render, the sub-object does not.

Any hints on where to go from here?

[avivpxi]

@skandragon for us to be able to reproduce and fully understand the ask, can you please provide an example input JSON and the output you expect to get? Also the code you've attached is not full (for instance, I'm missing the internalData field on the Item struct), if you're able to attach a full main file or a playground link that reproduces the problem you experience we will be able to pin point it faster.

[skandragon]

After playing around a bit, I found I had to make a custom MarshalJSON() function that basically decodes the current data in struct form on top of the interface catch-all, and now things seem to work as I'd expect.

type ApiWrapper struct {
    internalData map[string]interface{} `json:"-"`
    Cursor       string                 `json:"cursor,omitempty"`
    Objects      []*Object              `json:"objects,omitempty"`
}

func (e *ApiWrapper) HandleJSONData(data map[string]interface{}) {
    e.internalData = data
}

func (e *ApiWrapper) MarshalJSON() ([]byte, error) {
    type foo ApiWrapper
    structOut, err := json.Marshal((*foo)(e))
    if err != nil {
        return nil, err
    }
    if err := json.Unmarshal(structOut, &e.internalData); err != nil {
        return nil, err
    }
    return json.Marshal(e.internalData)
}

type Object struct {
    internalData      map[string]interface{} `json:"-"`
    Type              string                 `json:"type,omitempty"`
    Id                string                 `json:"id,omitempty"`
    UpdatedAt         time.Time              `json:"updated_at,omitempty"`
    CreatedAt         time.Time              `json:"created_at,omitempty"`
    Version           int64                  `json:"version,omitempty"`
    IsDeleted         bool                   `json:"is_deleted,omitempty"`
    ItemData          *Item                  `json:"item_data,omitempty"`
    ItemVariationData *ItemVariation         `json:"item_variation_data,omitempty"`
}

func (e *Object) HandleJSONData(data map[string]interface{}) {
    e.internalData = data
}

func (e *Object) MarshalJSON() ([]byte, error) {
    type foo Object
    structOut, err := json.Marshal((*foo)(e))
    if err != nil {
        return nil, err
    }
    if err := json.Unmarshal(structOut, &e.internalData); err != nil {
        return nil, err
    }
    return json.Marshal(e.internalData)
}

type Item struct {
    internalData map[string]interface{} `json:"-"`
    Name         string                 `json:"name,omitempty"`
    Description  string                 `json:"description,omitempty"`
    Visibility   string                 `json:"visibility,omitempty"`
    Variations   []*Object              `json:"variations,omitempty"`
}

func (e *Item) HandleJSONData(data map[string]interface{}) {
    e.internalData = data
}

func (e *Item) MarshalJSON() ([]byte, error) {
    type foo Item
    structOut, err := json.Marshal((*foo)(e))
    if err != nil {
        return nil, err
    }
    if err := json.Unmarshal(structOut, &e.internalData); err != nil {
        return nil, err
    }
    return json.Marshal(e.internalData)
}

type ItemVariation struct {
    internalData map[string]interface{} `json:"-"`
    ItemId       string                 `json:"item_id,omitempty"`
    SKU          string                 `json:"sku,omitempty"`
}

func (e *ItemVariation) HandleJSONData(data map[string]interface{}) {
    e.internalData = data
}

func (e *ItemVariation) MarshalJSON() ([]byte, error) {
    type foo ItemVariation
    structOut, err := json.Marshal((*foo)(e))
    if err != nil {
        return nil, err
    }
    if err := json.Unmarshal(structOut, &e.internalData); err != nil {
        return nil, err
    }
    return json.Marshal(e.internalData)
}

I can now recode the top level struct, modify anything inside any struct, and then marshal the top-level struct. Without this custom marshaller, while I did save the data and could render any specific single level, it would not do this on sub-objects.

It might be cool if this could be automatic, but I'm not sure how to do this generically yet.