search

google / emboss

Emboss is a tool for generating code that reads and writes binary data structures.
Apache License 2.0
71 stars 21 forks source link

Standardized file format for .emb files #14

Open bitwiser73 opened 3 years ago

bitwiser73 commented 3 years ago

Hello, This project seems very promising. It would be great to use a standard format to allow more features built on .emb file declarations. The Kaitai project, which could be seen as an heavier alternative, benefits of such approach. Multiple generic tool like the IDE are able to use the same definition files.

https://formats.kaitai.io/ https://ide.kaitai.io/

Thank you

reventlov commented 3 years ago

I am not entirely certain what you mean.

If you mean a way for file explorer and protocol analyzer tools to use .emb files, I do have a plan for that. The Emboss compiler is divided into a front end and (at the moment, only one) back end. The front end does a lot of processing to the .emb (about 90% of the compiler is in the front end), and generates an "intermediate representation" (IR) in JSON format. The back end reads the JSON IR and generates a C++ header file.

A protocol analyzer or similar tool would use the IR, not the uncooked .emb -- the .emb format is designed for humans to write somewhat easily, while the IR format is designed for computers to read.

I have not stabilized or documented the IR yet because I only have the C++ back end, and I am not confident that the IR is entirely suitable for other uses. I plan to create a couple more back ends -- a protocol buffer "equivalent representation" back end (for Google reasons), a documentation generator back end, and at least one more language back end (probably Python, Rust, or Go), then document and IR and make it stable.

Other than that:

We do not have as many formats "out of the box" because Emboss has not been public as long as Kaitai Struct, and for various reasons I have not been able to release any internal Google-created formats. I would certainly welcome any contributions!

I do have .emb syntax highlighting support for Vim, and it should be possible to set up syntax highlighting for other IDEs and tools. There is also an autoformatter that is built on the exact same syntactic parser that the compiler front end uses, but the parser changes every time I add a new feature to the .emb file, so I do not have a plan to allow external tools to easily use it.

bitwiser73 commented 3 years ago

Thank you very much for this complete response. I was hoping for something like JSON support to eventually interface with some forensic software for file/disk analysis (nothing precise yet). The IR seems perfect, I will look forward to future Emboss versions.

AaronWebster commented 2 years ago

@bitwiser73, if you want to give it a go it should be pretty straightforward to add JSON hooks in the IR.