[Discussion] Do we need emit_json ?

[g41797]

lcmgen consists of two parts:

• parser - parses *lcm files and creates internal "db"
• emitter - reads "db" and creats cources for language in question

This makes development for other languages somewhat inconvenient (at least for me):

• using C for emitter
• forking repo
• or adding separated projects for the new language

Protocol buffers compiler solves this problem via compiler plugins:

A plugin is just a program which reads a CodeGeneratorRequest protocol buffer from standard input and then writes a CodeGeneratorResponse protocol buffer to standard output. These message types are defined in plugin.proto. We recommend that all third-party code generators be written as plugins, as this allows all generators to provide a consistent interface and share a single parser implementation.

I am not calling for changing the implementation

Just to think about saving results of the parsing in JSON file.

Generator for new language reads JSON and generates code

Credit to jbendes

[nosracd]

I agree that the current implementation is not very friendly to people who want to contribute and appreciate your ideas for improving it!

To make sure I'm understanding your ideas correctly, are you proposing that we add an emit_json so that lcm-gen can output JSON files in order for a follow-on generator to consume the JSON and output new languages? Also, sorry if I'm missing some of the context; that discord link doesn't appear to work for me even after logging in.

One other path forward we could go to make adding support for new languages a bit more developer-friendly: if we added some bindings for portions of lcm-gen we could allow for new emitters to be written in Python.

[g41797]

Or on Zig - it's my case

[jbendes]

Is the goal here to not contribute to the repo? I'm lost as to why a 2 step process is better than a 1 step process for generating types. The only benefit I'm seeing is that you wouldn't have to contribute new emitter code back to the repo. Is that the goal? I feel like there's another goal that I'm not understanding

[g41797]

Any color the customer wants, as long as it’s black.

Henry Ford (with regard to the color of the Model T's)

Not everything should be implemented in C.

My intent was to port *cm to Zig:

• gen - Zig
• communication - Zig only (not wrapper and stdclib)

Later the same approach can be applied to go and c#

[jbendes]

Okay so your goal was to be able to write emitters in a different language. Now I understand a lot more. That was never stated prior to this. So you're saying that you would like there to be an intermediate format that is "easier" to parse than the {z,l}cmtype itself? I'll move back over to discord for zcm conversation so I don't take over an lcm issue haha

[jbendes]

Just to close the loop, I believe this is already implemented in zcm by using zcm-gen --summary <path/to/type>

[g41797]

/* ZCM */
    if (gopt.getBool("summary")) {
        did_something = 1;
        zcm.dump(); JSON!!!
    }

/* LCM */
    if (getopt_get_bool(gopt, "debug")) {
        did_something = 1;
        lcmgen_dump(lcm); TEXT!!!
    }

now it's lcm turn

btw emit_json looks more symmetric/preferable solution

[g41797]

example_1.zsm:

struct example_t
{
    int64_t  timestamp;
    double   position[3];
    double   orientation[4];
    int32_t  num_ranges;
    int16_t  ranges[num_ranges];
    string   name;
    boolean  enabled;

    int32_t nExamples1;
    int32_t nExamples2;
    example_t subExamples[nExamples1][nExamples2];
    string subStrings[nExamples1][nExamples2];

    float complexNestedArray[5][nExamples1][3][nExamples2][4];

    int8_t:2 bitfield1;
    int8_t:3 bitfield2[4];

    const int8_t  test_const_8_max_hex  = 0xff;
    const int16_t test_const_16_max_hex = 0xffff;
    const int32_t test_const_32_max_hex = 0xffffffff;
    const int64_t test_const_64_max_hex = 0xffffffffffffffff;

    const float  test_const_float  = 1e-20;
    const double test_const_double = 12.1e200;

    const string test_const_string = "example";
}

JSON dump:

{
    "example_t" : 
    {
        "comment" : "",
        "constants" : 
        [
            {
                "comment" : "",
                "membername" : "test_const_8_max_hex",
                "type" : "int8_t",
                "val" : 
                {
                    "as_json" : -1,
                    "as_string" : "0xff"
                }
            },
            {
                "comment" : "",
                "membername" : "test_const_16_max_hex",
                "type" : "int16_t",
                "val" : 
                {
                    "as_json" : -1,
                    "as_string" : "0xffff"
                }
            },
            {
                "comment" : "",
                "membername" : "test_const_32_max_hex",
                "type" : "int32_t",
                "val" : 
                {
                    "as_json" : -1,
                    "as_string" : "0xffffffff"
                }
            },
            {
                "comment" : "",
                "membername" : "test_const_64_max_hex",
                "type" : "int64_t",
                "val" : 
                {
                    "as_json" : -1,
                    "as_string" : "0xffffffffffffffff"
                }
            },
            {
                "comment" : "",
                "membername" : "test_const_float",
                "type" : "float",
                "val" : 
                {
                    "as_json" : 9.9999996826552254e-21,
                    "as_string" : "1e-20"
                }
            },
            {
                "comment" : "",
                "membername" : "test_const_double",
                "type" : "double",
                "val" : 
                {
                    "as_json" : 1.21e+201,
                    "as_string" : "12.1e200"
                }
            },
            {
                "comment" : "",
                "membername" : "test_const_string",
                "type" : "string",
                "val" : 
                {
                    "as_json" : "\"example\"",
                    "as_string" : "\"example\""
                }
            }
        ],
        "hash" : 18359083813438051081,
        "members" : 
        [
            {
                "comment" : "",
                "membername" : "timestamp",
                "typename" : 
                {
                    "fullname" : "int64_t"
                }
            },
            {
                "comment" : "",
                "dims" : 
                [
                    {
                        "size" : "3",
                        "type" : "const"
                    }
                ],
                "membername" : "position",
                "typename" : 
                {
                    "fullname" : "double"
                }
            },
            {
                "comment" : "",
                "dims" : 
                [
                    {
                        "size" : "4",
                        "type" : "const"
                    }
                ],
                "membername" : "orientation",
                "typename" : 
                {
                    "fullname" : "double"
                }
            },
            {
                "comment" : "",
                "membername" : "num_ranges",
                "typename" : 
                {
                    "fullname" : "int32_t"
                }
            },
            {
                "comment" : "",
                "dims" : 
                [
                    {
                        "size" : "num_ranges",
                        "type" : "var"
                    }
                ],
                "membername" : "ranges",
                "typename" : 
                {
                    "fullname" : "int16_t"
                }
            },
            {
                "comment" : "",
                "membername" : "name",
                "typename" : 
                {
                    "fullname" : "string"
                }
            },
            {
                "comment" : "",
                "membername" : "enabled",
                "typename" : 
                {
                    "fullname" : "boolean"
                }
            },
            {
                "comment" : "",
                "membername" : "nExamples1",
                "typename" : 
                {
                    "fullname" : "int32_t"
                }
            },
            {
                "comment" : "",
                "membername" : "nExamples2",
                "typename" : 
                {
                    "fullname" : "int32_t"
                }
            },
            {
                "comment" : "",
                "dims" : 
                [
                    {
                        "size" : "nExamples1",
                        "type" : "var"
                    },
                    {
                        "size" : "nExamples2",
                        "type" : "var"
                    }
                ],
                "membername" : "subExamples",
                "typename" : 
                {
                    "fullname" : "example_t"
                }
            },
            {
                "comment" : "",
                "dims" : 
                [
                    {
                        "size" : "nExamples1",
                        "type" : "var"
                    },
                    {
                        "size" : "nExamples2",
                        "type" : "var"
                    }
                ],
                "membername" : "subStrings",
                "typename" : 
                {
                    "fullname" : "string"
                }
            },
            {
                "comment" : "",
                "dims" : 
                [
                    {
                        "size" : "5",
                        "type" : "const"
                    },
                    {
                        "size" : "nExamples1",
                        "type" : "var"
                    },
                    {
                        "size" : "3",
                        "type" : "const"
                    },
                    {
                        "size" : "nExamples2",
                        "type" : "var"
                    },
                    {
                        "size" : "4",
                        "type" : "const"
                    }
                ],
                "membername" : "complexNestedArray",
                "typename" : 
                {
                    "fullname" : "float"
                }
            },
            {
                "comment" : "",
                "membername" : "bitfield1",
                "typename" : 
                {
                    "fullname" : "int8_t",
                    "numbits" : 2,
                    "sign_extend" : false
                }
            },
            {
                "comment" : "",
                "dims" : 
                [
                    {
                        "size" : "4",
                        "type" : "const"
                    }
                ],
                "membername" : "bitfield2",
                "typename" : 
                {
                    "fullname" : "int8_t",
                    "numbits" : 3,
                    "sign_extend" : false
                }
            }
        ],
        "structname" : 
        {
            "fullname" : "example_t"
        }
    }
}