Interface for travsersing json pointers in go types

[b5]

In order to make json schema references work, we need an interface for traversing json pointers in decoded go types. I've seen this pattern pop up in a few places, and would love to think about how to broaden the applicability of such an interface(s).

Problem

Let's say we decode this json to a generic map[string]interface{}:

{
    "foo" : ["bar"],
    "references": {
        "a": {"$ref": "#/foo/0"},
        "b": {"$ref": "/foo/0"}
    }
}

According to the json pointer spec, Both "a" and "b" in the above example should resolve to "bar". By decoding to map[string]interface{}, this is no problem, and the current jsonpointer.Eval can handle this for you using a type switch. This breaks, however, when anything that isn't map[string]interface{} or []interface{} are provided to jsonpointer.Eval. This problem gets worse when we do exotic things with go types.

After experimenting a bunch with the "just marshal back to generic structs & traverse", it's just way too much overhead and complexity. So:

We need an interface that lets go types declare how to traverse json pointer paths

Suggested Solution:

Instead of having go structs be in charge of resolving full paths, I think each struct should only be in charge of resolving it's immediate children (if any). There are only two "container" types in JSON: objects and arrays. The thornier of the two types is an object. So I'm suggesting two interfaces.

The first interface provides fast lookup for traversing expected paths (eg, json pointers):

// JSONContainer returns any existing child value for a given JSON property string
type JSONContainer interface {
    // JSONProp takes a string reference for a given JSON property.
    // implementations must return any matching property of that name,
    // nil if no such subproperty exists.
    // Note that implementations on slice-types are expected to convert
    // prop to an integer value 
    JSONProp(prop string) interface{}
}

The second interface provides comprehensive child listing for full-tree traversal:

// JSONParent is an interface that enables tree traversal by listing
// all immediate children of an object
type JSONParent interface {
    // JSONChildren should return all immidiate children of this element
    // with json property names as keys, go types as values
    // Note that implementations on slice-types are expected to convert
    // integers to string keys
    JSONProps() map[string]interface{}
}

Looking up a path should be done by breaking up the pointer into tokens and evaluating the components, as the current jsonpointer.Eval() does, but now with a check for the JSONContainer interface, and reflection as a fallback.

The work of traversing the entire tree should be a generic function:

// WalkJSON calls visit on all elements in a tree of decoded json
jsonpointer.WalkJSON(tree interface{}, visit func(elem interface{}) error) error

This follows the spirit of encoding/json and filepath.Walk from the standard lib. The filepath.Walk() part allows users to BYO visit function to call on each element in a tree.

Both functions would follow the spirit of the way encoding/json handles mapping JSON to go types:

• Checks for implementation of the above interfaces, using those if present.
• Has fast-paths for map[string]interface{} and []interface{}
• Falls back reflection, examining the json tag when present for property naming, and case-insensitive matching on exported field names otherwise

My expectation is that most implementations on slice-types would not use the JSONContainer or JSONParent at all, and instead rely on reflection. Slice-types would still have the JSONContainer interface for odd circumstances or edge cases.

@Stebalien, I'd love to see if we can't make the distance between resolving json pointers and IPLD paths as short as possible, and would love your (or really anyone from the IPLD project's) input on this interface as we go forward. I'm assuming IPLD's needs will be a superset of these problems, but it'd be great if the two could play nicely.

I've come across this problem a bunch in the wild, and would love other's feedback on the subject. Thanks!

[dolmen]

According to the json pointer spec, Both "a" and "b" in the above example should resolve to "bar"

I disagree:

1. This is not relevant to the JSON pointer spec (which this project is about). This is a JSON reference resolution issue
2. Following JSON reference rules, /foo/0 refers to the root element (JSON pointer "") of file /foo/0. So this is not the same JSON pointer.

Note: I'm not the author of this project, but an author of another JSON Pointer parser for Go : github.com/dolmen-go/jsonptr

[chanced]

this is an old issue but i agree with @dolmen. /foo/0 and #/foo/0 are two different pointers entirely.

Note: I am not the author of this project, but an author of yet another json pointer parser in go: github.com/chanced/jsonpointer