amazon-ion / ion-go

A Go implementation of Amazon Ion.
https://amazon-ion.github.io/ion-docs/
Apache License 2.0
174 stars 31 forks source link

Read then write symbols as symbols, rather than special structs #192

Open johnpyp opened 9 months ago

johnpyp commented 9 months ago

My goal is to write a simple cli that takes in a binary ion file that uses a shared symbol table, provide that symbol table, then output that "hydrated" ion to a pretty printed text file.

Unexpectedly, what happens is that all the symbols in the input ion get output as "symbol structs". I see some mention of them in the documentation, but I'm not very clear on how to handle them or turn them off when I'm doing this relatively transparent reading/writing.

Here's an example of the output:

...
            my_list: [
                {
                    Text: "MySymbolValue",
                    LocalSID: 396,
                    Source: null
                },
                {
                    Text: "MySymbolValueTwo",
                    LocalSID: 1414,
                    Source: null
                },
...

and what'd I'd expect:

my_list: [ 'MySymbolValue', 'MySymvolValueTwo' ]

symbol annotations also get erased from structs during the conversion.

Is there any way to make this work? Am I using the wrong decoding or unmarshalling / marshalling methods or something?

I also tried just writing to the writer directly without using a "root container", and that didn't seem to have an effect.


Here's the code for my poc, minus the getSharedSymbolTable func in another file that isn't really relevant:

package main

import (
    "bytes"
    "flag"
    "fmt"
    "os"
    "path"

    "github.com/amazon-ion/ion-go/ion"
)

type container struct {
    Root []interface{} `ion:"root"`
}

func main() {
    opts := getOpts()

    if opts.OutputFile == "" {
        fmt.Fprintln(os.Stderr, "missing required flag --output")
        os.Exit(1)
        return
    }

    sst := getSharedSymbolTable()
    cat := ion.NewCatalog(sst)

    reader := ion.NewReaderCat(os.Stdin, cat)
    dec := ion.NewDecoder(reader)
    buf := bytes.Buffer{}
    writer := ion.NewTextWriterOpts(&buf, ion.TextWriterPretty)
    enc := ion.NewEncoder(writer)

    container := container{
        Root: make([]interface{}, 0),
    }

    for {
        val, err := dec.Decode()
        if err == ion.ErrNoInput {
            break
        } else if err != nil {
            panic(err)
        }
        container.Root = append(container.Root, val)
    }

    if err := enc.Encode(container); err != nil {
        panic(err)
    }

    if err := enc.Finish(); err != nil {
        panic(err)
    }

    p := path.Clean(opts.OutputFile)

    os.WriteFile(p, buf.Bytes(), 0644)

    fmt.Println("Wrote converted ion to", p)

}

type Opts struct {
    OutputFile string
}

func getOpts() Opts {
    var outputFile string

    flag.StringVar(&outputFile, "output", "", "Relative or absolute path to an output file")

    flag.Parse()

    return Opts{
        OutputFile: outputFile,
    }
}