Frame processing of lists appears to be broken

kazarena / json-gold

A JSON-LD processor for Go

Apache License 2.0

114 stars 17 forks source link

Frame processing of lists appears to be broken #4

Closed hapnermw closed 9 years ago

hapnermw commented 9 years ago

Here's a program that reproduces the problem. It also lists a few more general issues with ld that I've come across.

/* This program provides examples of two cases where frame fails to properly process a graph.

The example contains a list that is broken by frame. The content of the list is an array of node references to nodes that don't exist.

See below for a description of other ld package issues ... */ package main

import

/* ld package issues:

    * NewJsonLdApi does not accept a JsonLdOptions parameter as it documents. Instead it appears that JsonLdOptions is given to only
    subset of JsonLdApi functions. This implies that only these functions make use of it.
    For instance, only these use a Document Loader to resolve remote context/document references.
    "NewJsonLdApi creates a new instance of JsonLdApi and initialises it with the given JsonLdOptions structure."

    * Frame does not process lists correctly. It appears it loses their content after they have been flattened and then does
    not later embed the content. Instead it results in a hanging node reference to their content.

    * The output of the Node jsonld module wraps a graph object around Frame output - this package does not.

    * It also does not do the 'empty context' compact as specified by the framing spec.

    * The spec is very unclear about how to construct the input frame and exactly what features it provides.
    The ld package doesn't provide any additional description.

*/ ( "encoding/json" "fmt"

"github.com/kazarena/json-gold/ld"

)

/* Canonicalize filters and transforms an unmarshalled JSON LD graph int o a consistent form for processing. This is done in three steps:

Expand removes all context if any has been included.
Frame extracts only the node(s) or value object(s) of the types specified. The outgoing edges of these nodes are fully 'unrolled'. If the unrolled edges include a node with multiple incoming edges, a copy of the node is attached to each edge.
Compact is used to remove array 'wrappers' from singleton arrays.

The input must be unmarshalled JSON. If only one node matches the typeFilter, it is returned; if no nodes are matched, the result is nil; otherwise an array of the matched nodes are returned. */ func Canonicalize(input interface{}, types []interface{}) (interface{}, error) { var ( emptyCtx = ld.NewContext(nil, ld.NewJsonLdOptions("")) ldapi = ld.NewJsonLdApi() err error frame []interface{} obj map[string]interface{} expanded interface{} framed interface{} framedArray []interface{} compactInput interface{} compacted interface{} )

obj = map[string]interface{}{"@type": types}
frame = []interface{}{obj}

expanded, err = ldapi.Expand(emptyCtx, "", input)
if err != nil {
    return nil, err
}

framed, err = ldapi.Frame(expanded, frame, nil)
if err != nil {
    return nil, err
}
switch framed.(type) {
case map[string]interface{}:
    compactInput = framed
case []interface{}:
    framedArray = framed.([]interface{})
    switch len(framedArray) {
    case 0:
        return nil, nil
    case 1:
        compactInput = framedArray[0]
    default:
        compactInput = framed
    }
default:
    return nil, nil
}

compacted, err = ldapi.Compact(emptyCtx, "", compactInput, true)
if err != nil {
    return nil, err
}
return compacted, nil

}

func main() {

var (
    examples      = make([]string, 1)
    unmarshalled  interface{}
    canonicalized interface{}
    err           error
)

examples[0] = `{
    "@context": {
        "p_type": "http://tn.resilient-networks.com/policy/type/#",
        "p_prop": "http://tn.resilient-networks.com/policy/property/#"
    },
    "@id": "_:per",
    "@type": ["p_type:policy_evaluation_request"],
    "p_prop:policy": {
        "@id": "_:1",
        "@type": "p_type:policy_def",
        "p_prop:px": {
            "@id": "_:1.1",
            "p_prop:px": {
                "@id": "_:1.1",
                "@type": "p_type:phase",
                "p_prop:step_list": {
                    "@list": [{
                        "@id": "_:1.1.1",
                        "@type": "p_type:authority_ref",
                        "p_prop:authority_uri": "http://localhost:9080/request",
                        "p_prop:pc_request_interface_ref": {
                            "@id": "_:1.1.1.1",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": [{
                                "@id": "_:1.1.1.1.1",
                                "@type": "p_type:pc_prop_ref",
                                "p_prop:pc_prop_rqstor_id": "foo",
                                "p_prop:pc_prop_rqstee_id": "foo"
                               }]
                        },
                        "p_prop:pc_result_interface_ref": {
                            "@id": "_:1.1.1.2",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": null
                        }
                    }, {
                        "@id": "_:1.1.2",
                        "@type": "p_type:authority_ref",
                        "p_prop:authority_uri": "http://localhost:9080/request",
                        "p_prop:pc_request_interface_ref": {
                            "@id": "_:1.1.2.1",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": [{
                                "@id": "_:1.1.2.1.1",
                                "@type": "p_type:pc_prop_ref",
                                "p_prop:pc_prop_rqstor_id": "foo",
                                "p_prop:pc_prop_rqstee_id": "foo"
                               }]
                        },
                        "p_prop:pc_result_interface_ref": {
                            "@id": "_:1.1.2.2",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": null
                        }
                    }, {
                        "@id": "_:1.1.3",
                        "@type": "p_type:authority_ref",
                        "p_prop:authority_uri": "http://localhost:9080/request",
                        "p_prop:pc_request_interface_ref": {
                            "@id": "_:1.1.3.1",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": [{
                                "@id": "_:1.1.3.1.1",
                                "@type": "p_type:pc_prop_ref",
                                "p_prop:pc_prop_rqstor_id": "foo",
                                "p_prop:pc_prop_rqstee_id": "foo"
                               }]
                        },
                        "p_prop:pc_result_interface_ref": {
                            "@id": "_:1.1.3.2",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": null
                        }
                    }, {
                        "@id": "_:1.1.4",
                        "@type": "p_type:authority_ref",
                        "p_prop:authority_uri": "http://localhost:9180/request",
                        "p_prop:pc_request_interface_ref": {
                            "@id": "_:1.1.4.1",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": [{
                                "@id": "_:1.1.4.1.1",
                                "@type": "p_type:pc_prop_ref",
                                "p_prop:pc_prop_rqstor_id": "foo",
                                "p_prop:pc_prop_rqstee_id": "foo"
                               }]
                        },
                        "p_prop:pc_result_interface_ref": {
                            "@id": "_:1.1.4.2",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": null
                        }
                    }, {
                        "@id": "_:1.1.5",
                        "@type": "p_type:authority_ref",
                        "p_prop:authority_uri": "http://localhost:9180/request",
                        "p_prop:pc_request_interface_ref": {
                            "@id": "_:1.1.5.1",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": [{
                                "@id": "_:1.1.5.1.1",
                                "@type": "p_type:pc_prop_ref",
                                "p_prop:pc_prop_rqstor_id": "foo",
                                "p_prop:pc_prop_rqstee_id": "foo"
                               }]
                        },
                        "p_prop:pc_result_interface_ref": {
                            "@id": "_:1.1.5.2",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": null
                        }
                    }, {
                        "@id": "_:1.1.6",
                        "@type": "p_type:authority_ref",
                        "p_prop:authority_uri": "http://localhost:9180/request",
                        "p_prop:pc_request_interface_ref": {
                            "@id": "_:1.1.6.1",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": [{
                                "@id": "_:1.1.6.1.1",
                                "@type": "p_type:pc_prop_ref",
                                "p_prop:pc_prop_rqstor_id": "foo",
                                "p_prop:pc_prop_rqstee_id": "foo"
                               }]
                        },
                        "p_prop:pc_result_interface_ref": {
                            "@id": "_:1.1.6.2",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": null
                        }
                    }]
                }
            }
        }
    }
}`

for _, example := range examples {
    err = json.Unmarshal([]byte(example), &unmarshalled)
    if err != nil {
        fmt.Printf("Unmarshal Error: %v", err)
    }

    canonicalized, err = Canonicalize(unmarshalled, []interface{}{"http://tn.resilient-networks.com/policy/type/#policy_evaluation_request"})
    if err != nil {
        fmt.Printf("Canonicalize Error: %v", err)
    }
    ld.PrintDocument("Bad Graph", canonicalized)
}

}

kazarena commented 9 years ago

Thanks for your very detailed report @hapnermw, I'll take a look.

kazarena commented 9 years ago

Hi @hapnermw,

Please see my comments below.

* NewJsonLdApi does not accept a JsonLdOptions parameter as it documents. Instead it appears that JsonLdOptions is given to only
subset of JsonLdApi functions. This implies that only these functions make use of it.
For instance, only these use a Document Loader to resolve remote context/document references.
"NewJsonLdApi creates a new instance of JsonLdApi and initialises it with the given JsonLdOptions structure."

The doc string is outdated. Thank you for reporting it, will fix. This is a legacy of the Java JSON-LD library: it initialises the API object with JsonLdOptions, even though, as you noticed, it's only used by some functions.

Then, as per spec http://www.w3.org/TR/json-ld-api/#the-jsonldprocessor-interface, JSON-LD operations should be invoked through JsonLdProcessor, and not through the internal JsonLdApi interface. I wouldn't recommend using JsonLdApi, unless you really need to do something advanced. I'm going to change the docstrings accordingly to avoid confusion, apologies for that!

Frame does not process lists correctly. It appears it loses their content after they have been flattened and then does not later embed the content. Instead it results in a hanging node reference to their content.

Ok, there are 3 factors here:

1) I found 2 bugs in the framing code, one is mine, another was inherited from the Java implementation of JSON-LD: embedValues function didn't process lists, as specified in the spec: http://json-ld.org/spec/latest/json-ld-framing/#embed-values. This is now corrected, see the changeset (to follow shortly).

2) when using the correct interface JsonLdProcessor, your frame definition should be:

frame = map[string]interface{}{
        "@context": map[string]interface{}{
            "p_type": "http://tn.resilient-networks.com/policy/type/#",
            "p_prop": "http://tn.resilient-networks.com/policy/property/#",
        },
        "@type": types,
    }

(context definition will be used for compaction)

3) your input document seems to have a duplicate embedded property _pprop:px:

            "@type": "p_type:policy_def",
            "p_prop:px": {
                "@id": "_:1.1",
                "p_prop:px": {
                    "@id": "_:1.1",
                    "@type": "p_type:phase",

The output of the Node jsonld module wraps a graph object around Frame output - this package does not.

It also does not do the 'empty context' compact as specified by the framing spec.

If you use JsonLdProcessor, these issues won't appear.

The spec is very unclear about how to construct the input frame and exactly what features it provides. The ld package doesn't provide any additional description.

Yes, you are right. I also had great difficulty validating this implementation without this information. However, there's an abundant source of practical examples available in the test suite https://github.com/kazarena/json-gold/tree/master/ld/testdata, look for frame-XXXX-YYY.jsonld files.

kazarena commented 9 years ago

@hapnermw, please see the working code with comments below:

package main

import (
    "encoding/json"
    "fmt"

    "github.com/kazarena/json-gold/ld"
)

/*
Canonicalize filters and transforms an unmarshalled JSON LD graph int o a consistent form for processing. This is done in three steps:

1. Expand removes all context if any has been included.

2. Frame extracts only the node(s) or value object(s) of the types specified. The outgoing edges of these nodes are
fully 'unrolled'. If the unrolled edges include a node with multiple incoming edges, a copy of the node is attached to each edge.

3. Compact is used to remove array 'wrappers' from singleton arrays.

The input must be unmarshalled JSON.
If only one node matches the typeFilter, it is returned; if no nodes are matched, the result is nil; otherwise an array of the matched nodes are returned.
*/
func Canonicalize(input interface{}, types []interface{}) (interface{}, error) {
    var (
        err          error
        frame        map[string]interface{}
        expanded     interface{}
        framed       interface{}
        framedArray  []interface{}
        compactInput interface{}
        compacted    interface{}
    )

    opts := ld.NewJsonLdOptions("")
    proc := ld.NewJsonLdProcessor()

    frame = map[string]interface{}{
        "@context": map[string]interface{}{
            "p_type": "http://tn.resilient-networks.com/policy/type/#",
            "p_prop": "http://tn.resilient-networks.com/policy/property/#",
        },
        "@type": types,
    }

    framed, err = proc.Frame(expanded, frame, opts)
    if err != nil {
        return nil, err
    }

    // not sure what this code is meant to do with corrected inputs
    switch framed.(type) {
    case map[string]interface{}:
        compactInput = framed
    case []interface{}:
        framedArray = framed.([]interface{})
        switch len(framedArray) {
        case 0:
            return nil, nil
        case 1:
            compactInput = framedArray[0]
        default:
            compactInput = framed
        }
    default:
        return nil, nil
    }

    // there should be no need for compaction - Frame operation compacts the document
    compacted, err = proc.Compact(compactInput, frame["@context"], opts)
    if err != nil {
        return nil, err
    }
    return compacted, nil
}

func main() {

    var (
        examples      = make([]string, 1)
        unmarshalled  interface{}
        canonicalized interface{}
        err           error
    )

    examples[0] = `{
        "@context": {
            "p_type": "http://tn.resilient-networks.com/policy/type/#",
            "p_prop": "http://tn.resilient-networks.com/policy/property/#"
        },
        "@id": "_:per",
        "@type": ["p_type:policy_evaluation_request"],
        "p_prop:policy": {
            "@id": "_:1",
            "@type": "p_type:policy_def",
            "p_prop:px": {
                "@id": "_:1.1",
                "@type": "p_type:phase",
                "p_prop:step_list": {
                    "@list": [{
                        "@id": "_:1.1.1",
                        "@type": "p_type:authority_ref",
                        "p_prop:authority_uri": "http://localhost:9080/request",
                        "p_prop:pc_request_interface_ref": {
                            "@id": "_:1.1.1.1",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": [{
                                "@id": "_:1.1.1.1.1",
                                "@type": "p_type:pc_prop_ref",
                                "p_prop:pc_prop_rqstor_id": "foo",
                                "p_prop:pc_prop_rqstee_id": "foo"
                               }]
                        },
                        "p_prop:pc_result_interface_ref": {
                            "@id": "_:1.1.1.2",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": null
                        }
                    }, {
                        "@id": "_:1.1.2",
                        "@type": "p_type:authority_ref",
                        "p_prop:authority_uri": "http://localhost:9080/request",
                        "p_prop:pc_request_interface_ref": {
                            "@id": "_:1.1.2.1",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": [{
                                "@id": "_:1.1.2.1.1",
                                "@type": "p_type:pc_prop_ref",
                                "p_prop:pc_prop_rqstor_id": "foo",
                                "p_prop:pc_prop_rqstee_id": "foo"
                               }]
                        },
                        "p_prop:pc_result_interface_ref": {
                            "@id": "_:1.1.2.2",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": null
                        }
                    }, {
                        "@id": "_:1.1.3",
                        "@type": "p_type:authority_ref",
                        "p_prop:authority_uri": "http://localhost:9080/request",
                        "p_prop:pc_request_interface_ref": {
                            "@id": "_:1.1.3.1",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": [{
                                "@id": "_:1.1.3.1.1",
                                "@type": "p_type:pc_prop_ref",
                                "p_prop:pc_prop_rqstor_id": "foo",
                                "p_prop:pc_prop_rqstee_id": "foo"
                               }]
                        },
                        "p_prop:pc_result_interface_ref": {
                            "@id": "_:1.1.3.2",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": null
                        }
                    }, {
                        "@id": "_:1.1.4",
                        "@type": "p_type:authority_ref",
                        "p_prop:authority_uri": "http://localhost:9180/request",
                        "p_prop:pc_request_interface_ref": {
                            "@id": "_:1.1.4.1",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": [{
                                "@id": "_:1.1.4.1.1",
                                "@type": "p_type:pc_prop_ref",
                                "p_prop:pc_prop_rqstor_id": "foo",
                                "p_prop:pc_prop_rqstee_id": "foo"
                               }]
                        },
                        "p_prop:pc_result_interface_ref": {
                            "@id": "_:1.1.4.2",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": null
                        }
                    }, {
                        "@id": "_:1.1.5",
                        "@type": "p_type:authority_ref",
                        "p_prop:authority_uri": "http://localhost:9180/request",
                        "p_prop:pc_request_interface_ref": {
                            "@id": "_:1.1.5.1",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": [{
                                "@id": "_:1.1.5.1.1",
                                "@type": "p_type:pc_prop_ref",
                                "p_prop:pc_prop_rqstor_id": "foo",
                                "p_prop:pc_prop_rqstee_id": "foo"
                               }]
                        },
                        "p_prop:pc_result_interface_ref": {
                            "@id": "_:1.1.5.2",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": null
                        }
                    }, {
                        "@id": "_:1.1.6",
                        "@type": "p_type:authority_ref",
                        "p_prop:authority_uri": "http://localhost:9180/request",
                        "p_prop:pc_request_interface_ref": {
                            "@id": "_:1.1.6.1",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": [{
                                "@id": "_:1.1.6.1.1",
                                "@type": "p_type:pc_prop_ref",
                                "p_prop:pc_prop_rqstor_id": "foo",
                                "p_prop:pc_prop_rqstee_id": "foo"
                               }]
                        },
                        "p_prop:pc_result_interface_ref": {
                            "@id": "_:1.1.6.2",
                            "@type": "p_type:pc_interface_ref",
                            "p_prop:pc_prop_refs": null
                        }
                    }]
                }
            }
        }
    }`

    for _, example := range examples {
        err = json.Unmarshal([]byte(example), &unmarshalled)
        if err != nil {
            fmt.Printf("Unmarshal Error: %v", err)
        }

        canonicalized, err = Canonicalize(unmarshalled, []interface{}{"http://tn.resilient-networks.com/policy/type/#policy_evaluation_request"})
        if err != nil {
            fmt.Printf("Canonicalize Error: %v", err)
        }
        ld.PrintDocument("Good Graph", canonicalized)
    }

}

kazarena commented 9 years ago

Reopening for comments as it was closed by mistake

hapnermw commented 9 years ago

Hi Kazarena,

Thanks for the effort you have put into providing a JSON LD processor for the go community.

It was not clear to me that JsonLdProcessor is supposed the be the top level abstraction. Thanks for clarifying this. I'll convert my code to use it.

Thanks also for fixing Frame's processing of lists. Now I can put them back into my schema instead of relying on the side effect of sets retaining their order.

The example I provided did duplicate the p_prop:px property in an embedded node. This is allowed in the JSON LD schema I've designed. I'm assuming there isn't any issue processing it.

It's too bad that JSON LD is such a well-kept secret. It's excellent for the design of extensible metadata and data.

-- Mark.

On Wed, Sep 23, 2015 at 4:59 PM, kazarena notifications@github.com wrote:

Hi @hapnermw https://github.com/hapnermw,

Please see my comments below.

NewJsonLdApi does not accept a JsonLdOptions parameter as it documents. Instead it appears that JsonLdOptions is given to only subset of JsonLdApi functions. This implies that only these functions make use of it. For instance, only these use a Document Loader to resolve remote context/document references. "NewJsonLdApi creates a new instance of JsonLdApi and initialises it with the given JsonLdOptions structure."

The doc string is outdated. Thank you for reporting it, will fix. This is a legacy of the Java JSON-LD library: it initialises the API object with JsonLdOptions, even though, as you noticed, it's only used by some functions.

Then, as per spec http://www.w3.org/TR/json-ld-api/#the-jsonldprocessor-interface, JSON-LD operations should be invoked through JsonLdProcessor, and not through the internal JsonLdApi interface. I wouldn't recommend using JsonLdApi, unless you really need to do something advanced.

Frame does not process lists correctly. It appears it loses their content after they have been flattened and then does not later embed the content. Instead it results in a hanging node reference to their content.

Ok, there are 3 factors here:

1) I found 2 bugs in the framing code, one is mine, another was inherited from the Java implementation of JSON-LD: embedValues function didn't process lists, as specified in the spec: http://json-ld.org/spec/latest/json-ld-framing/#embed-values. This is now corrected, see the changeset (to follow shortly).

2) when using the correct interface JsonLdProcessor, your frame definition should be:

frame = map[string]interface{}{ "@context": map[string]interface{}{ "p_type": "http://tn.resilient-networks.com/policy/type/#", "p_prop": "http://tn.resilient-networks.com/policy/property/#", }, "@type": types, }

(context definition will be used for compaction)

3) your input document seems to have a duplicate embedded property _pprop:px:
        "@type": "p_type:policy_def",
        "p_prop:px": {
            "@id": "_:1.1",
            "p_prop:px": {
                "@id": "_:1.1",
                "@type": "p_type:phase",
The output of the Node jsonld module wraps a graph object around Frame output - this package does not.

It also does not do the 'empty context' compact as specified by the framing spec.

If you use JsonLdProcessor, these issues won't appear.

The spec is very unclear about how to construct the input frame and exactly what features it provides. The ld package doesn't provide any additional description.

Yes, you are right. I also had great difficulty validating this implementation without this information. However, there's an abundant source of practical examples available in the test suite https://github.com/kazarena/json-gold/tree/master/ld/testdata, look for frame-XXXX-YYY.jsonld files.

— Reply to this email directly or view it on GitHub https://github.com/kazarena/json-gold/issues/4#issuecomment-142764119.

kazarena commented 9 years ago

@hapnermw, thanks for your feedback. I'm planning to do some refactoring to make the main interface more apparent. It's also clear to me that there need to be more examples how to use this library.

hapnermw commented 9 years ago

Hi Kazarena,

I finally switched over to using JsonLdProcessor and ran into a few minor issues.

JsonLdProcessor JsonLdOptions parameter is supposed to be nullable for all functions.

When Expand is given a nil JsonLdOptions it fails at processor.go +116.

When Frame is given a nil JsonLdOptions it fails at utils.go +236

It also appears that the default values for an empty JsonLdOptions are not correct. The default value for CompactArrays is true; however, it currently defaults to false.

Frame will take a frame with an @type of a string but fails with an @type of []string.

These are all reproduced with this program.

-- Mark

package main

import (

"encoding/json"

"fmt"

"github.com/kazarena/json-gold/ld"

)

func main() {

*var* (

    uploadRsp = `{

"@id": "f006db08-e735-432d-a90f-fba9b99ac211",

"@type": [

"http://api.webshield.io/type#SvcResponse",

"http://api.webshield.io/type#Assertion"

],

"http://api.webshield.io/prop#assertions": {

    "@id": "

http://assertion.id.webshield.io/a6df144d-f76c-4afe-b81f-171cb0a5c64c",

    "@type": [

    "http://api.webshield.io/type#Assertion",

    "http://api.webshield.io/type#Upload2SIM"

    ],

    "http://api.webshield.io/prop#issue_time":

"2015-10-05T17:21:39.043Z",

    "http://api.webshield.io/prop#issuer": {

        "@type": "http://api.webshield.io/type#x509CN",

        "@value": "pb.webshield.io"

    },

    "http://api.webshield.io/prop#items": [{

            "@id": "_:14c75cf4-6602-4761-a3cc-591fb56befaa",

            "@type": "http://api.webshield.io/type#UploadResultItem",

            "http://api.webshield.io/prop#entity_ref": {

                "@id": "

http://id.webshield.io/uuid/ba07b5af-e533-4700-891d-c5103730b0fd",

                "@type": "http://api.webshield.io/type#EntityRef",

                "http://api.webshield.io/prop#dataset_ref": {

                    "http://api.webshield.io/prop#dataset_id":

"e2e_upload2Aetna_dataset_f34707a0-78a7-4ed8-8e58-7c12f6aa62a1",

                    "http://api.webshield.io/prop#sim_id":

"e2e_upload2Aetna_simid"

                }

            },

            "http://api.webshield.io/prop#source_id": "

http://id.webshield.io/com/aetna#012-345-67801"

    }],

    "http://api.webshield.io/prop#responding_to":

"3eb33db5-9592-4ccb-b954-64c411ee975e_e2e_upload_data_aetna_dataset_svcrq",

    "http://api.webshield.io/prop#responding_to_policy":

"77928dc9-1a7a-4eaf-8907-90e4d25163bb"

},

"http://api.webshield.io/prop#issue_time": "2015-10-05T17:21:39.043Z",

"http://api.webshield.io/prop#issuer": "add",

"http://api.webshield.io/prop#responding_to":

"3eb33db5-9592-4ccb-b954-64c411ee975e_e2e_upload_data_aetna_dataset_svcrq"

    unmarshalled    *interface*{}

    expanded        []*interface*{}

    framed          *map*[string]*interface*{}

    jsonLdProcessor = ld.NewJsonLdProcessor()

    err             error

)

err = json.Unmarshal([]byte(uploadRsp), &unmarshalled)

*if* err != nil {

    fmt.Printf("Unmarshal Error: %v", err)

}

expanded, err = jsonLdProcessor.Expand(unmarshalled, nil)

*if* err != nil {

    fmt.Printf("Expand Error: %v\n", err)

    *return*

}

ld.PrintDocument("Expanded Upload Response", expanded)

framed, err = jsonLdProcessor.Frame(expanded, *map*[string]*interface*

{}{"@type": []string{"http://api.webshield.io/type#SvcResponse"}}, nil)

*if* err != nil {

    fmt.Printf("Frame Error: %v\n", err)

    *return*

}

ld.PrintDocument("Framed Upload Response", framed)

}

On Fri, Sep 25, 2015 at 9:36 PM, Mark Hapner hapnermw@gmail.com wrote:

Hi Kazarena,

Thanks for the effort you have put into providing a JSON LD processor for the go community.

It was not clear to me that JsonLdProcessor is supposed the be the top level abstraction. Thanks for clarifying this. I'll convert my code to use it.

Thanks also for fixing Frame's processing of lists. Now I can put them back into my schema instead of relying on the side effect of sets retaining their order.

The example I provided did duplicate the p_prop:px property in an embedded node. This is allowed in the JSON LD schema I've designed. I'm assuming there isn't any issue processing it.

It's too bad that JSON LD is such a well-kept secret. It's excellent for the design of extensible metadata and data.

-- Mark.

On Wed, Sep 23, 2015 at 4:59 PM, kazarena notifications@github.com wrote:
Hi @hapnermw https://github.com/hapnermw,

Please see my comments below.

NewJsonLdApi does not accept a JsonLdOptions parameter as it documents. Instead it appears that JsonLdOptions is given to only subset of JsonLdApi functions. This implies that only these functions make use of it. For instance, only these use a Document Loader to resolve remote context/document references. "NewJsonLdApi creates a new instance of JsonLdApi and initialises it with the given JsonLdOptions structure."

The doc string is outdated. Thank you for reporting it, will fix. This is a legacy of the Java JSON-LD library: it initialises the API object with JsonLdOptions, even though, as you noticed, it's only used by some functions.

Then, as per spec http://www.w3.org/TR/json-ld-api/#the-jsonldprocessor-interface, JSON-LD operations should be invoked through JsonLdProcessor, and not through the internal JsonLdApi interface. I wouldn't recommend using JsonLdApi, unless you really need to do something advanced.

Frame does not process lists correctly. It appears it loses their content after they have been flattened and then does not later embed the content. Instead it results in a hanging node reference to their content.

Ok, there are 3 factors here:

1) I found 2 bugs in the framing code, one is mine, another was inherited from the Java implementation of JSON-LD: embedValues function didn't process lists, as specified in the spec: http://json-ld.org/spec/latest/json-ld-framing/#embed-values. This is now corrected, see the changeset (to follow shortly).

2) when using the correct interface JsonLdProcessor, your frame definition should be:

frame = map[string]interface{}{ "@context": map[string]interface{}{ "p_type": "http://tn.resilient-networks.com/policy/type/#", "p_prop": "http://tn.resilient-networks.com/policy/property/#", }, "@type": types, }

(context definition will be used for compaction)

3) your input document seems to have a duplicate embedded property _pprop:px:
        "@type": "p_type:policy_def",
        "p_prop:px": {
            "@id": "_:1.1",
            "p_prop:px": {
                "@id": "_:1.1",
                "@type": "p_type:phase",
The output of the Node jsonld module wraps a graph object around Frame output - this package does not.

It also does not do the 'empty context' compact as specified by the framing spec.

If you use JsonLdProcessor, these issues won't appear.

The spec is very unclear about how to construct the input frame and exactly what features it provides. The ld package doesn't provide any additional description.

Yes, you are right. I also had great difficulty validating this implementation without this information. However, there's an abundant source of practical examples available in the test suite https://github.com/kazarena/json-gold/tree/master/ld/testdata, look for frame-XXXX-YYY.jsonld files.

— Reply to this email directly or view it on GitHub https://github.com/kazarena/json-gold/issues/4#issuecomment-142764119.

kazarena commented 9 years ago

Hi @hapnermw, I'll take a look.

hapnermw commented 9 years ago

There also seems to be a more serious issue. Frame is duplicating nodes.

Here’s a program to replicate this issue. One node goes in and two come out ...

package main

import (

"encoding/json"

"fmt"

"github.com/kazarena/json-gold/ld"

)

func Canonicalize(input interface{}, typeID map[string]interface{}) (interface{}, error) {

*var* (

    jsonLdProcessor = ld.NewJsonLdProcessor()

    err             error

    expanded        []*interface*{}

    framed          *map*[string]*interface*{}

)

expanded, err = jsonLdProcessor.Expand(input, nil)

*if* err != nil {

    *return* nil, err

}

ld.PrintDocument("expanded", expanded)

framed, err = jsonLdProcessor.Frame(expanded, typeID, nil)

*if* err != nil {

    *return* nil, err

}

ld.PrintDocument("expanded", framed)

*return* framed, nil

}

func main() {

*var* (

    qjResult = `{

"@id": "_:1",

"@type": [

"http://api.webshield.io/type#SvcResponse",

"http://api.webshield.io/type#Assertion"

"http://api.webshield.io/prop#assertions": {

"@id": "

http://assertion.id.webshield.io/68c2fe63-cec6-4d61-a8d8-5ea38d98d513",

"@type": [

  "http://api.webshield.io/type#Assertion",

  "http://api.webshield.io/type#Query",

  "http://api.webshield.io/type#Dataset",

  "http://api.webshield.io/type#Syndicated"

],

"http://api.webshield.io/prop#dataset_data": [{

    "@id": "_:uploaded_results",

    "@type": "http://api.webshield.io/type#DatasetData",

    "http://api.webshield.io/prop#dataset_ref": {

      "http://api.webshield.io/prop#dataset_id":

"e2e_aetna_synd_query_job_22b99638-5174-4827-a8ad-ff7385fb8eb7",

      "http://api.webshield.io/prop#sim_id":

"e2e_aetna_synd_query_job_simid"

    },

    "http://api.webshield.io/prop#domain_data": [{

"@id": "_:2",

"@type": [

"http://api.webshield.io/type#SvcResponse",

"http://api.webshield.io/type#Assertion"

"http://api.webshield.io/prop#assertions": {

"@id": "

http://assertion.id.webshield.io/68c2fe63-cec6-4d61-a8d8-5ea38d98d513",

"@type": [

  "http://api.webshield.io/type#Assertion",

  "http://api.webshield.io/type#Query",

  "http://api.webshield.io/type#Dataset",

  "http://api.webshield.io/type#Syndicated"

],

"http://api.webshield.io/prop#dataset_data": [{

    "@id": "_:uploaded_results",

    "@type": "http://api.webshield.io/type#DatasetData",

    "http://api.webshield.io/prop#dataset_ref": {

      "http://api.webshield.io/prop#dataset_id":

"e2e_aetna_synd_query_job_22b99638-5174-4827-a8ad-ff7385fb8eb7",

      "http://api.webshield.io/prop#sim_id":

"e2e_aetna_synd_query_job_simid"

    },

    "http://api.webshield.io/prop#domain_data": [{

        "@id": "http://id.webshield.io/com/aetna#012-345-67801",

        "@type": "http://aetna.schema.webshield.io/type#Subject",

        "http://aetna.schema.webshield.io/prop#Address": {

          "@id": "_:b1531d34-fd23-4143-b28e-4c401e90b15c",

          "@type": "http://aetna.schema.webshield.io/type#Address",

          "http://aetna.schema.webshield.io/prop#Address1stLine": "123 A

Blvd",

          "http://aetna.schema.webshield.io/prop#Address2ndLine": "",

          "http://aetna.schema.webshield.io/prop#City": "Ex City",

          "http://aetna.schema.webshield.io/prop#State": "CA",

          "http://aetna.schema.webshield.io/prop#Zip5": "12345"

        },

        "http://aetna.schema.webshield.io/prop#DOB": "01/01/1999",

        "http://aetna.schema.webshield.io/prop#FirstName": "John",

        "http://aetna.schema.webshield.io/prop#Gender": "Male",

        "http://aetna.schema.webshield.io/prop#LastName": "Smith",

        "http://aetna.schema.webshield.io/prop#SSN": "987654321",

        "http://aetna.schema.webshield.io/prop#SSNValid": "Good",

        "http://aetna.schema.webshield.io/prop#SourceID":

"012-345-67801"

      }]

  }],

"http://api.webshield.io/prop#dataset_ref": {

  "http://api.webshield.io/prop#dataset_id":

"e2e_aetna_synd_query_job_22b99638-5174-4827-a8ad-ff7385fb8eb7",

  "http://api.webshield.io/prop#sim_id":

"e2e_aetna_synd_query_job_simid"

},

"http://api.webshield.io/prop#issue_time": "2015-10-03T03:29:04.756Z",

"http://api.webshield.io/prop#issuer": "https://pb.webshield.io",

"http://api.webshield.io/prop#responding_to":

"svcRequest_aetna_syndicated_query_61b267c5-f642-47a3-863b-013cbcefec30",

"http://api.webshield.io/prop#responding_to_policy":

"af35270e-03fc-4b3b-832b-7d34e416ee9e"

"http://api.webshield.io/prop#issue_time": "2015-10-03T03:29:04.757Z",

"http://api.webshield.io/prop#issuer": "add",

"http://api.webshield.io/prop#responding_to": "svcRequest_aetna_syndicated_query_61b267c5-f642-47a3-863b-013cbcefec30"

}]

  }],

"http://api.webshield.io/prop#dataset_ref": {

  "http://api.webshield.io/prop#dataset_id":

"e2e_aetna_synd_query_job_22b99638-5174-4827-a8ad-ff7385fb8eb7",

  "http://api.webshield.io/prop#sim_id":

"e2e_aetna_synd_query_job_simid"

},

"http://api.webshield.io/prop#issue_time": "2015-10-03T03:29:04.756Z",

"http://api.webshield.io/prop#issuer": "https://pb.webshield.io",

"http://api.webshield.io/prop#responding_to":

"svcRequest_aetna_syndicated_query_61b267c5-f642-47a3-863b-013cbcefec30",

"http://api.webshield.io/prop#responding_to_policy":

"af35270e-03fc-4b3b-832b-7d34e416ee9e"

"http://api.webshield.io/prop#issue_time": "2015-10-03T03:29:04.757Z",

"http://api.webshield.io/prop#issuer": "add",

"http://api.webshield.io/prop#responding_to": "svcRequest_aetna_syndicated_query_61b267c5-f642-47a3-863b-013cbcefec30"

    unmarshalled *interface*{}

    err          error

)

err = json.Unmarshal([]byte(qjResult), &unmarshalled)

*if* err != nil {

    fmt.Printf("Unmarshal Error: %v", err)

}

_, err = Canonicalize(unmarshalled, *map*[string]*interface*{}{"@type":

"http://api.webshield.io/type#SvcResponse"})

*if* err != nil {

    fmt.Printf("Canonicalize Error: %v\n", err)

    *return*

}

}

On Fri, Oct 9, 2015 at 3:08 AM, kazarena notifications@github.com wrote:

Hi @hapnermw https://github.com/hapnermw, I'll take a look.

— Reply to this email directly or view it on GitHub https://github.com/kazarena/json-gold/issues/4#issuecomment-146822395.

hapnermw commented 9 years ago

There are some duplicate @id's in the last example that when removed eliminated the overall duplication. I'm not clear what the correct Frame of the graph would be with the dups so the result of Frame may be correct for it; or, it might be a pathogenic case that causes Frame to do the wrong thing - I can't tell which.

On Fri, Oct 9, 2015 at 4:09 AM, Mark Hapner hapnermw@gmail.com wrote:

There also seems to be a more serious issue. Frame is duplicating nodes.

Here’s a program to replicate this issue. One node goes in and two come out ...

package main

import (
"encoding/json"

"fmt"

"github.com/kazarena/json-gold/ld"
)

func Canonicalize(input interface{}, typeID map[string]interface {}) (interface{}, error) {
*var* (

    jsonLdProcessor = ld.NewJsonLdProcessor()

    err             error

    expanded        []*interface*{}

    framed          *map*[string]*interface*{}

)

expanded, err = jsonLdProcessor.Expand(input, nil)

*if* err != nil {

    *return* nil, err

}

ld.PrintDocument("expanded", expanded)

framed, err = jsonLdProcessor.Frame(expanded, typeID, nil)

*if* err != nil {

    *return* nil, err

}

ld.PrintDocument("expanded", framed)

*return* framed, nil
}

func main() {
*var* (

    qjResult = `{
"@id": "_:1",

"@type": [
"http://api.webshield.io/type#SvcResponse",

"http://api.webshield.io/type#Assertion"
],

"http://api.webshield.io/prop#assertions": {
"@id": "
http://assertion.id.webshield.io/68c2fe63-cec6-4d61-a8d8-5ea38d98d513",
"@type": [

  "http://api.webshield.io/type#Assertion",

  "http://api.webshield.io/type#Query",

  "http://api.webshield.io/type#Dataset",

  "http://api.webshield.io/type#Syndicated"

],

"http://api.webshield.io/prop#dataset_data": [{

    "@id": "_:uploaded_results",

    "@type": "http://api.webshield.io/type#DatasetData",

    "http://api.webshield.io/prop#dataset_ref": {

      "http://api.webshield.io/prop#dataset_id":
"e2e_aetna_synd_query_job_22b99638-5174-4827-a8ad-ff7385fb8eb7",
      "http://api.webshield.io/prop#sim_id":
"e2e_aetna_synd_query_job_simid"
    },

    "http://api.webshield.io/prop#domain_data": [{
"@id": "_:2",

"@type": [
"http://api.webshield.io/type#SvcResponse",

"http://api.webshield.io/type#Assertion"
],

"http://api.webshield.io/prop#assertions": {
"@id": "
http://assertion.id.webshield.io/68c2fe63-cec6-4d61-a8d8-5ea38d98d513",
"@type": [

  "http://api.webshield.io/type#Assertion",

  "http://api.webshield.io/type#Query",

  "http://api.webshield.io/type#Dataset",

  "http://api.webshield.io/type#Syndicated"

],

"http://api.webshield.io/prop#dataset_data": [{

    "@id": "_:uploaded_results",

    "@type": "http://api.webshield.io/type#DatasetData",

    "http://api.webshield.io/prop#dataset_ref": {

      "http://api.webshield.io/prop#dataset_id":
"e2e_aetna_synd_query_job_22b99638-5174-4827-a8ad-ff7385fb8eb7",
      "http://api.webshield.io/prop#sim_id":
"e2e_aetna_synd_query_job_simid"
    },

    "http://api.webshield.io/prop#domain_data": [{

        "@id": "http://id.webshield.io/com/aetna#012-345-67801",

        "@type": "http://aetna.schema.webshield.io/type#Subject",

        "http://aetna.schema.webshield.io/prop#Address": {

          "@id": "_:b1531d34-fd23-4143-b28e-4c401e90b15c",

          "@type": "http://aetna.schema.webshield.io/type#Address",

          "http://aetna.schema.webshield.io/prop#Address1stLine": "123
A Blvd",
          "http://aetna.schema.webshield.io/prop#Address2ndLine": "",

          "http://aetna.schema.webshield.io/prop#City": "Ex City",

          "http://aetna.schema.webshield.io/prop#State": "CA",

          "http://aetna.schema.webshield.io/prop#Zip5": "12345"

        },

        "http://aetna.schema.webshield.io/prop#DOB": "01/01/1999",

        "http://aetna.schema.webshield.io/prop#FirstName": "John",

        "http://aetna.schema.webshield.io/prop#Gender": "Male",

        "http://aetna.schema.webshield.io/prop#LastName": "Smith",

        "http://aetna.schema.webshield.io/prop#SSN": "987654321",

        "http://aetna.schema.webshield.io/prop#SSNValid": "Good",

        "http://aetna.schema.webshield.io/prop#SourceID":
"012-345-67801"
      }]

  }],

"http://api.webshield.io/prop#dataset_ref": {

  "http://api.webshield.io/prop#dataset_id":
"e2e_aetna_synd_query_job_22b99638-5174-4827-a8ad-ff7385fb8eb7",
  "http://api.webshield.io/prop#sim_id":
"e2e_aetna_synd_query_job_simid"
},

"http://api.webshield.io/prop#issue_time": "2015-10-03T03:29:04.756Z",

"http://api.webshield.io/prop#issuer": "https://pb.webshield.io",

"http://api.webshield.io/prop#responding_to":
"svcRequest_aetna_syndicated_query_61b267c5-f642-47a3-863b-013cbcefec30",
"http://api.webshield.io/prop#responding_to_policy":
"af35270e-03fc-4b3b-832b-7d34e416ee9e"

},

"http://api.webshield.io/prop#issue_time": "2015-10-03T03:29:04.757Z",

"http://api.webshield.io/prop#issuer": "add",

"http://api.webshield.io/prop#responding_to": "svcRequest_aetna_syndicated_query_61b267c5-f642-47a3-863b-013cbcefec30"

}]
  }],

"http://api.webshield.io/prop#dataset_ref": {

  "http://api.webshield.io/prop#dataset_id":
"e2e_aetna_synd_query_job_22b99638-5174-4827-a8ad-ff7385fb8eb7",
  "http://api.webshield.io/prop#sim_id":
"e2e_aetna_synd_query_job_simid"
},

"http://api.webshield.io/prop#issue_time": "2015-10-03T03:29:04.756Z",

"http://api.webshield.io/prop#issuer": "https://pb.webshield.io",

"http://api.webshield.io/prop#responding_to":
"svcRequest_aetna_syndicated_query_61b267c5-f642-47a3-863b-013cbcefec30",
"http://api.webshield.io/prop#responding_to_policy":
"af35270e-03fc-4b3b-832b-7d34e416ee9e"

},

"http://api.webshield.io/prop#issue_time": "2015-10-03T03:29:04.757Z",

"http://api.webshield.io/prop#issuer": "add",

"http://api.webshield.io/prop#responding_to": "svcRequest_aetna_syndicated_query_61b267c5-f642-47a3-863b-013cbcefec30"

}`
    unmarshalled *interface*{}

    err          error

)

err = json.Unmarshal([]byte(qjResult), &unmarshalled)

*if* err != nil {

    fmt.Printf("Unmarshal Error: %v", err)

}

_, err = Canonicalize(unmarshalled, *map*[string]*interface*{}{"@type"
: "http://api.webshield.io/type#SvcResponse"})
*if* err != nil {

    fmt.Printf("Canonicalize Error: %v\n", err)

    *return*

}
}

On Fri, Oct 9, 2015 at 3:08 AM, kazarena notifications@github.com wrote:

Hi @hapnermw https://github.com/hapnermw, I'll take a look.

— Reply to this email directly or view it on GitHub https://github.com/kazarena/json-gold/issues/4#issuecomment-146822395.

kazarena commented 9 years ago

@hapnermw, I've pushed an update to provide default JsonLdOptions if not provided (for all functions in JsonLdProcessor).

More comments:

1) your issue with framing with list of types is due to having []string as Go type instead of []interface{}:

framed, err = jsonLdProcessor.Frame(
    expanded,
    map[string]interface{}{
        "@type": []string{"http://api.webshield.io/type#SvcResponse"}},
    nil)

should be:

framed, err = jsonLdProcessor.Frame(
    expanded,
    map[string]interface{}{
        "@type": []interface{}{"http://api.webshield.io/type#SvcResponse"}},
    nil)

This is an awkward one: Golang doesn't treat []interface{} (expected) as a generalised form of []string. If you pass hard-coded JSON structures, you need to be careful to use []interface{} for lists and map[string]interface{} for maps. Any ideas how to make it easier are welcome!

2) duplicate @id's will cause all sorts of issues for most operations. Validation of this issue can be an expensive operation so I definitely wouldn't recommend adding it into the Processor itself. Feedback welcome!

hapnermw commented 9 years ago

Thanks for your efforts.

There is no way to validate id uniqueness since it is not a JLD requirement. What is needed is a clear explanation about how the processing algorithms compose nodes with the same id. I have a feeling that in the case I sent, the composition is broken but I haven't verified this.

Thanks, I should have seen that since I'm familiar with the interface{} encode/decode issue. The encode/json package doesn't clearly document this issue. It wouldn't hurt to add a few brief notes about details like this, dup ids, etc. to the docs to help beginners.

On Mon, Oct 12, 2015 at 6:27 AM, kazarena notifications@github.com wrote:

@hapnermw https://github.com/hapnermw, I've pushed an update to provide default JsonLdOptions if not provided (for all functions in JsonLdProcessor).

More comments:

1) your issue with framing with list of types is due to having []string as Go type instead of []interface{}:

framed, err = jsonLdProcessor.Frame( expanded, map[string]interface{}{ "@type": []string{"http://api.webshield.io/type#SvcResponse"}}, nil)

should be:

framed, err = jsonLdProcessor.Frame( expanded, map[string]interface{}{ "@type": []interface{}{"http://api.webshield.io/type#SvcResponse"}}, nil)

This is an awkward one: Golang doesn't treat []interface{} (expected) as a generalised form of []string. If you pass hard-coded JSON structures, you need to be careful to use []interface{} for lists and map[string]interface{} for maps. Any ideas how to make it easier are welcome!

2) duplicate @id https://github.com/id's will cause all sorts of issues for most operations. Validation of this issue can be an expensive operation so I definitely wouldn't recommend adding it into the Processor itself. Feedback welcome!

— Reply to this email directly or view it on GitHub https://github.com/kazarena/json-gold/issues/4#issuecomment-147247008.

hapnermw commented 9 years ago

Hi Again,

I've come across another bug in frame. It is converting

[ { "@type": " http://privacy.pn.schema.webshield.io/octx#aetna_poc_1", "@value": "MIICAwYJKoZIhvcNAQcDoIIB9DCCAfACAQAxggGrMIIBpwIBADCBjjB3MQswCQYD\nVQQGEwJVUzETMBEGA1UEChMKQWV0bmEgSW5jLjEoMCYGA1UECxMfR2VvUm9vdCBD\nZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTEpMCcGA1UEAxMgQWV0bmEgSW5jLiBDZXJ0\naWZpY2F0ZSBBdXRob3JpdHkCEz4AACq958ozFBQZpjsAAAAAKr0wDQYJKoZIhvcN\nAQEBBQAEggEADhOnJeJQEjL5zzhV6gptV14jGRWzgSQdTn+rWhKkJLhexNpZv9oh\nVfQeNnAhw+xPNtyDwWYi1Exy+OmDiGU1hqFAZ+GFWXEANF8ZSTrvkFlWwinJr8U0\nUG65VU8ogMiVT/xv9Un03SL7OrqvthwU36orTl1p5pyT4o+zwhA7V+R5iOic7WfT\nIwyVGns7PlD/iSinCJJJh+sr1mXvcisMRPjcPpBREti6W28YxvMwYOWee2c2/L+9\nhR/pSy2KwnPUBE8+dgRnspvyrOJH3XLIT9KQDjEfNZi4WBuznWCRrxSTuc378kva\nzv6qcZtu9iXI+4rqIOvb31g2lzpPXLnedTA8BgkqhkiG9w0BBwEwHQYJYIZIAWUD\nBAEqBBDQ67YPeZZZTfKInoMl0P1KgBBjxTuxyYt5XcGfGg8c2UXl\n" } ]

So, in some cases it is adding an empty string as a second element of a singleton value object array.

I've attached a program that reproduces this.

On Mon, Oct 12, 2015 at 4:20 PM, Mark Hapner hapnermw@gmail.com wrote:

Thanks for your efforts.

There is no way to validate id uniqueness since it is not a JLD requirement. What is needed is a clear explanation about how the processing algorithms compose nodes with the same id. I have a feeling that in the case I sent, the composition is broken but I haven't verified this.

Thanks, I should have seen that since I'm familiar with the interface{} encode/decode issue. The encode/json package doesn't clearly document this issue. It wouldn't hurt to add a few brief notes about details like this, dup ids, etc. to the docs to help beginners.

On Mon, Oct 12, 2015 at 6:27 AM, kazarena notifications@github.com wrote:

@hapnermw https://github.com/hapnermw, I've pushed an update to provide default JsonLdOptions if not provided (for all functions in JsonLdProcessor).

More comments:

1) your issue with framing with list of types is due to having []string as Go type instead of []interface{}:

framed, err = jsonLdProcessor.Frame( expanded, map[string]interface{}{ "@type": []string{"http://api.webshield.io/type#SvcResponse"}}, nil)

should be:

framed, err = jsonLdProcessor.Frame( expanded, map[string]interface{}{ "@type": []interface{}{"http://api.webshield.io/type#SvcResponse"}}, nil)

This is an awkward one: Golang doesn't treat []interface{} (expected) as a generalised form of []string. If you pass hard-coded JSON structures, you need to be careful to use []interface{} for lists and map[string]interface{} for maps. Any ideas how to make it easier are welcome!

2) duplicate @id https://github.com/id's will cause all sorts of issues for most operations. Validation of this issue can be an expensive operation so I definitely wouldn't recommend adding it into the Processor itself. Feedback welcome!

— Reply to this email directly or view it on GitHub https://github.com/kazarena/json-gold/issues/4#issuecomment-147247008.

kazarena commented 9 years ago

Hi @hapnermw,

I can't see your test program. Where can I find it?

hapnermw commented 9 years ago

Odd, it was attached. Possibly it got filtered out. I'll put it on GitHub.

On Wed, Dec 2, 2015 at 2:34 PM, kazarena notifications@github.com wrote:

Hi @hapnermw https://github.com/hapnermw,

I can't see your test program. Where can I find it?

— Reply to this email directly or view it on GitHub https://github.com/kazarena/json-gold/issues/4#issuecomment-161455474.