Should we consider more compact data format like Protobuf ?

[rjiejie]

I have compiled sail-riscv branch，and there are about 40k lines in output json file. Is there any performance issue when we parse information in other project ?

[ThinkOpenly]

Is there any performance issue when we parse information in other project ?

There is basically one downstream project at the moment: https://github.com/ThinkOpenly/RISC-V_ISA/. The JSON is copied into that project and becomes the database behind a website like https://thinkopenly.github.io/RISC-V_ISA/. I haven't looked too deeply into impacts of using JSON versus something more space-efficient.

One big reason for choosing JSON was its accessibility: easy to read, easy to load, easy to use, lots of runtime support.

The highest priorities of the project currently are completeness and accuracy. Performance has not been a priority.

Given that the current implementation basically pushes all relevant data into hash tables, then emits it as JSON, it should be pretty straightforward to emit different formats, subject to needing to be implemented in OCaml (or a secondary transformation from JSON).

[rjiejie]

Yes, I agree with you, the JSON is pretty good in most cases. There was a similar feature request about JSON backend from original SAIL repo long time ago: https://github.com/rems-project/sail/issues/144 but maintainer have no plan to implement that, they prefer to reuse SAIL or OCaml AST to extract needed information, it seems to be difficult to do that from AST. BTW, i use sail option -ddump_tc_ast to get AST, maybe it's not best way to construct AST for parsing ?

Do you have a try to reuse AST to extract needed information before implementing JSON backend ?

[ThinkOpenly]

There was a similar feature request about JSON backend from original SAIL repo long time ago: rems-project#144

Interesting. The discussion there is apt. For lightweight uses, JSON is "good enough", but it does take some effort to get usable JSON (as can be seen in this project). Fortunately, there are a LOT of lightweight uses.

but maintainer have no plan to implement that, they prefer to reuse SAIL or OCaml AST to extract needed information, it seems to be difficult to do that from AST.

This project extracts information from AST. It is not trivial, but it is not impossible.

BTW, i use sail option -ddump_tc_ast to get AST, maybe it's not best way to construct AST for parsing ?

If I were to call that a bottom-up approach, I have tended to use a top-down approach, working in the OCaml code to find the AST tagged variant records and walking through src/lib/parse_ast.ml to understand the structure. Both approaches have their pros and cons.

Do you have a try to reuse AST to extract needed information before implementing JSON backend ?

I don't understand this question. This project implements a backend that accepts parsed Sail code in the form of an AST, extracts salient data (into hash tables), and emits (some of) that data as JSON.