bytedance / sonic

A blazingly fast JSON serializing & deserializing library
Apache License 2.0
6.59k stars 327 forks source link

Is there a more elegant implementation of re-serialization after partial modification of the contents of ast.Node? #588

Closed Bisstocuz closed 4 months ago

Bisstocuz commented 5 months ago

Example data:

{
  "hits": [
    {
      "title": "title1",
      "moduleId": 1,
      "category": 2,
      "subcategory": 6,
      "source": 4,
      "thumbnail": "https://domain.ltd/131e18d17384f6a09098a1978a01690a.png",
      "updateAt": "2024-01-16T22:17:14+08:00"
    },
    {
      "title": "title2",
      "moduleId": 63,
      "category": 1,
      "subcategory": 1,
      "source": 2,
      "thumbnail": "https://domain.ltd/0e752aacbe708ea305135f2fc1d2eb02.png",
      "updateAt": "2024-01-21T13:52:48+08:00"
    }
  ],
  "query": "search text",
  "processingTimeMs": 0,
  "hitsPerPage": 24,
  "page": 1,
  "totalPages": 1,
  "totalHits": 2
}

This is my traveling code:

func handleRaw(src json.RawMessage) (json.RawMessage, error) {
    root, getRootErr := sonic.Get(src)
    if getRootErr != nil {
        return nil, getRootErr
    }

    hits := root.IndexOrGet(1, "hits")
    _ = hits.ForEach(func(_ ast.Sequence, node *ast.Node) bool {
        thumbnail := node.Index(5)
        objectKey, _ := thumbnail.String()
        fmt.Println(thumbnail.String())
        _, _ = node.SetByIndex(5, ast.NewString("new url"))
        return true
    })

    return root.MarshalJSON()
}

If use ListIterator, the copy of node will not change the source 'root' json content, only ForEach works but I only to particially update the key thumbnail, no need to load another content.

  1. Is there a more elegant implementation of traveling V_ARRAY and edit its object's member?
  2. Is it possible to get the right index after calling IndexOrGet? For example, I wanna know thumbnail's index after calling IndexOrGet(5, "thumbnail").
AsterDY commented 5 months ago
  1. Currently not for ast.Node, otherwise you can try ast.Visitor
AsterDY commented 5 months ago
  1. Possible. But what's the point of it?
Bisstocuz commented 5 months ago
  1. Possible. But what's the point of it?

No another points, just I wanna know this: how to get thumbnail's right index after calling IndexOrGet(5, "thumbnail")

AsterDY commented 5 months ago

I mean its practical meaning, as a reference to support or not

Bisstocuz commented 5 months ago

I mean its practical meaning, as a reference to support or not

Tip:

since Index() uses offset to locate data, which is much faster than scanning like Get()

So I guess SetByIndex() should be much faster than Set(), If I wanna change some fields from their original value, I hope to use index to set new value.

AsterDY commented 5 months ago

OK. Looks like it's worth doing that. Let me add a new API to support it -- or can you try to submit a PR for this?

Bisstocuz commented 5 months ago

OK. Looks like it's worth doing that. Let me add a new API to support it -- or can you try to submit a PR for this?

It looks like only one or two lines need to be changed to support this feature.

// IndexOrGet firstly use idx to index a value and check if its key matches
// If not, then use the key to search value
func (self *Node) IndexOrGet(idx int, key string) *Node {
    if err := self.should(types.V_OBJECT, "an object"); err != nil {
        return unwrapError(err)
    }

    pr := self.skipIndexPair(idx)
    if pr != nil && pr.Key == key {
        return &pr.Value
    }
    n, _ := self.skipKey(key)
    return n
}

skipKey returned the right index in the second returned value.

Should we directly add a return value to this method? Or is it better to create a new method to handle this logic? If we are to create a new method, what should the method name be?

AsterDY commented 5 months ago

A new method, to keep compatible with elder version. Give any name you like, just make it easy to understand

Bisstocuz commented 4 months ago

close via #594