kaitai-io / kaitai_struct

Kaitai Struct: declarative language to generate binary data parsers in C++ / C# / Go / Java / JavaScript / Lua / Nim / Perl / PHP / Python / Ruby
https://kaitai.io
4.04k stars 199 forks source link

Unnamed fields are not visible in visualizers (ksv, Web IDE) #1064

Open generalmimon opened 1 year ago

generalmimon commented 1 year ago

Currently, _unnamed* fields (which are created by omitting the id key in seq attributes) do not appear in object trees of either the Web IDE or the console visualizer. This is bad because the "unnamed fields" feature was added to KS mainly to make reverse-engineering work easier, because it saves you from having to assign bogus ids for fields with unknown purpose manually. But the fact that these unnamed fields are not visible in visualizers defeats their utility for reverse-engineering, because you need to see the values of these yet unknown fields to figure out their purpose over time. @GreyCat confirmed that this is how unnamed fields should be handled in https://github.com/kaitai-io/kaitai_struct/issues/52#issuecomment-262875543:

Saving them (and even interpreting them as some proposed type) actually serves a very important function: you can view them in the visualizer. May be at first they won't make any sense (beyond seeing that it's a 4-byte long integer, for example), but at some moment later during reversing you might understand what it is - and, voila, all you have is to slap a name on it.


Personally, while working on the coreldraw_cdr.ksy specification and using ksdump a lot to generate JSON dumps showing how real files are parsed, I realized I need to see the unnamed fields in the dumps in order to progress in reverse engineering, so I was patching ksdump locally to display them:

lib/kaitai/struct/visualizer/obj_to_h.rb:30

-        next if k =~ /^_/
+        next if k =~ /^_(?!unnamed)/

But there's no reason why ksdump (or Web IDE) shouldn't handle this correctly out of the box.