search

kaitai-io / kaitai_struct

Kaitai Struct: declarative language to generate binary data parsers in C++ / C# / Go / Java / JavaScript / Lua / Nim / Perl / PHP / Python / Ruby
https://kaitai.io
3.91k stars 192 forks source link

Nim support #619

Open sealmove opened 4 years ago

sealmove commented 4 years ago

Support for the Nim programming language is now under development.

runtime library repo

deem0n commented 4 years ago

Is it ready for general use?

sealmove commented 4 years ago

Is it ready for general use?

Not yet, but I am actively working on it. Glad for your interest!

deem0n commented 4 years ago

I planned to use C++ runtime and hand-made nim wrapper for it, but native nim runtime would be much better!

sealmove commented 4 years ago

Yeah, I am trying to use as many native Nim features as possible, and produce as much human readable code as possible, so it takes some time.

GreyCat commented 4 years ago

631 — a discussion on grand rework of Nim backend.

GreyCat commented 4 years ago

I see that nim appeared in CI dashboard — although currently it is at 0%, it's a great step forward! Please tell me if I can be of any help with infrastructure?

GreyCat commented 4 years ago

I've noticed that Nim test run is currently failing due to compilation errors. Any chance that partial builder framework would of of any help here — i.e. doing the routine "create project" => "try to compile" => "exclude unbuildable files" => "create project again" => ad inf?

sealmove commented 4 years ago

I've noticed that Nim test run is currently failing due to compilation errors. Any chance that partial builder framework would of of any help here — i.e. doing the routine "create project" => "try to compile" => "exclude unbuildable files" => "create project again" => ad inf?

It's fixed with a rework of the testing mechanism

sealmove commented 4 years ago

Development focus has been shifted towards nimitai. Unfortunately this means Nim support for standard Kaitai Struct project will be delayed.

GreyCat commented 4 years ago

@sealmove, thanks for your update! Can I ask you to add it to awesome-kaitai list?

sealmove commented 4 years ago

@sealmove, thanks for your update! Can I ask you to add it to awesome-kaitai list?

Sure, I will do that!

KOLANICH commented 4 years ago

What are the plans on supporting nim runtime in the KSC? I mean is it going to be kept or removed because of nimitai? Should I add explicit support of nim-related arguments to kaitaiStructCompile.py?

sealmove commented 4 years ago

I don't see a reason to remove it. It might take quite a while, but at some point in time -after nimitai is finished- and having accumulated knowledge from nimitai, it will be very easy for me to finish it for the KSC too.

How is kaitaiStructCompile.py related?

KOLANICH commented 4 years ago

How is kaitaiStructCompile.py related?

It is a python lib wrapping (when dealing to KSC using it a user usually doesn't deals with KSC cli args (but it is possible to pass them if there is no direct support of the feature), he passes objects to a lib and backends containing the code translating these objects to backend-specific stuff) calls to KSC (2 backends are implemented, CLI (objects are translated to CLI args) and JVM (objects are translated to KSC internal structs), but JVM one is not distributed (#466) though it is much better (more secure (no CLI args are involved so shell injections are impossible), more faster (jvm is initialized once), has access to progress (coarse-grained) and KSC exceptions, but there is a severe issue with modules) because KSC is GPLed and I don't want to take part in GPL proliferation since GPL proliferation is copyright proliferation) and fetching. It is usually used to compile specs into python files when building pip packages (it has a setuptools plugin for it), but it is possible to use it to compile specs into another languages, for example as a part of build system (i.e. to write a plugin for meson) or when building a python package requires to build some code other than python (i.e. cexts implemented in C++ or rust).

sealmove commented 4 years ago

After 4 months of stupid ambition, I realized nimitai has failed. The idea was to re-implement Kaitai Struct as a Nim library. As it turns out, there is no compile-time yaml parser for Nim (or any other language actually, afaik). Implementing one + re-implementing Kaitai Struct with Nim macros turned out to be way too much for its benefits. Therefore I am coming back to the original plan!

I implemented a tiny tool called testify which takes care of the testing infrastructure (it was easier for me than figuring out how to adjust partial_builder). The results are already visible on dashboard (yeah I know, 3% is bad), so keep an eye!

sealmove commented 4 years ago

Ok, more than 60% of the tests pass. There are still a lot of things to adjust, but most features have been implemented. If you don't mind making some adjustments to the generated code, then you can already start using the back-end! (@deem0n). Soon I will write the page with Nim-specific notes explaining the API in detail. The API is different from the other back-ends' since Nim does not encourage object-oriented programming and I didn't want to make it feel foreign to Nim users. I think you will find it intuitive, direct, and easy to use.