search

kaitai-io / kaitai_struct

Kaitai Struct: declarative language to generate binary data parsers in C++ / C# / Go / Java / JavaScript / Lua / Nim / Perl / PHP / Python / Ruby
https://kaitai.io
4.03k stars 198 forks source link

Rust support #22

Open LogicAndTrick opened 8 years ago

LogicAndTrick commented 8 years ago

Adding this so it can be tracked, but I plan on doing most of this myself. I've not yet had a need for Rust but I've been wanting to try it out anyway, figured this is a good excuse.

I've created a runtime for Rust: https://github.com/kaitai-io/kaitai_struct_rust_runtime

I'll be adding matching tests and a compiler for Rust in the future.

Note that I don't know Rust at all so I'm learning as I go. If anybody has coded with it before I would very much appreciate advice, suggestions, or even a code review of the runtime!

GreyCat commented 8 years ago

Wow, that's cool :) May be you should post it to some Rust-related mailing lists / communities, to attract more peer review? I don't know a thing about Rust myself ;)

LogicAndTrick commented 8 years ago

After working with the language a bit, I've come to realise how difficult of a task this would be. Rust is certainly an interesting language to work with, but it comes with a huge list of new challenges that don't exist in other languages. I don't see myself being able to overcome them, so I'm going to quit.

There's a whole stack of architecture questions that are easy to answer in other languages, but actually very complex in Rust. The design of code generated by KS follows the easy patterns because that's how pretty much every other language works. But Rust is unique in many areas.

Rust has what they call a foreign function interface which lets you interact with C libraries, perhaps that's a more reasonable approach for Rust. But for now, I'm going to close this because I'm not able to make any more progress.

(I enjoyed experimenting with Rust: it sort of feels like a modern approach to C. The memory safety syntax is very complex for beginners, but the rest of language is quite simple and expressive. The tools are a pleasure to use as well. Would definitely recommend anyone to spend a weekend with it if they've ever been curious about it.)

GreyCat commented 8 years ago

Maybe someone would return to Rust implementation in future, so I thought it might be interesting to document your findings or at least map out possible obstacles on the path. What does the language lack that impedes KS implementation? Something about the type system? Memory management? Strict compile-time checks? Lack of "everything is an expression"? Something else?

LogicAndTrick commented 8 years ago

The main pain point I discovered is around the references to io, root, and parent in each object. If they could be removed somehow, I think I could probably get everything else going. From what I've found online, root/parent/self references require some pretty complex type chains, and the generated code would probably be pretty crazy as well. I don't see myself ever becoming proficient enough in Rust to be able to get that going correctly, and even if you could do it, the result would probably not be something that anybody would want to actually use.

If instance methods required these three pointers to be passed as arguments, then it might be more possible. For example:

// hard to do
fn get_calculated_instance(self) { ... }

// probably easier, but consumer must pass root/parent manually
fn get_calculated_instance(self, io: Stream, parent: Struct, root: Struct) { ... }

From what it seems, Rust is best approached by creating an architecture that specifically fits Rust's "memory safety first" style. Trying to modify an existing architecture so that it works in Rust is very difficult.

GreyCat commented 8 years ago

That's kind of strange. I guess that Rust has something that resembles an object (i.e. bundle of data + methods) and I don't see any difference in storing some "useful" data, i.e. attributes / instances vs storing a reference (pointer?) to io object that should be used for structure parsing, or parent, or root.

But that said, I know almost nothing about Rust, so I can only make theories.

GreyCat commented 7 years ago

I'm reorganizing all "new language support" issues, adding relevant tag. I'll reopen this one, as a few people asked for Rust support again, and it's better to keep it open, so it would be easier to find and people won't create duplicates again and again.

berkus commented 7 years ago

Yes, please, very much need Rust support from Kaitai!

Happy to help with discussions and possible implementation if needed.

GreyCat commented 7 years ago

@berkus You probably know of http://doc.kaitai.io/new_language.html — let's just follow. I'd be happy to launch a Rust-targeting branch as soon as we'll get basic hello_world.rs and basic tests for it.

berkus commented 7 years ago

No, this is new to me - I just started actually using Kaitai today, loving it so far.

Will take a look!

LogicAndTrick commented 7 years ago

I'm going to start tentatively taking a second look at this one because the second edition of the Rust book has a dedicated chapter on reference cycles which might help me solve the roadblocks I was hitting last year. I still have no experience with actual proper Rust code so if anybody has more experience with the language, I'd love some assistance!

Question for @GreyCat: I've manually assembled the code for a hello_world.rs file which passes basic tests, where would I put it? I don't think it's ready for CI scripts at this point, I still have to figure out the best way to separate the test project from the code project.

For the time being, it'd be good to get some feedback so here's what I've got so far for hello_world:

extern crate kaitai_struct;

use kaitai_struct::{Stream, Struct};

use std::io::{Cursor, Read, Seek};
use std::fs::File;

struct HelloWorld {
    one: u8,
}

impl Struct for HelloWorld {

    // Rust requires all fields to be initialised so we auto-generate an empty ctor
    fn empty() -> HelloWorld {
        HelloWorld { one: 0 }
    }

    // Passes any io::Error through to the caller using the ? operator
    fn from_file(path: &str) -> std::result::Result<HelloWorld, std::io::Error> {
        let mut buf = File::open(path)?;
        Ok(HelloWorld::new(
            &mut buf,
            &None::<Box<Struct>>,
            &None::<Box<Struct>>,
        ))
    }

    // Constructor from a stream
    // Box<> References to parent and root are temporary and the types will change.
    fn new<S: Stream>(
        stream: &mut S,
        parent: &Option<Box<Struct>>,
        root: &Option<Box<Struct>>,
    ) -> HelloWorld {
        let mut this = HelloWorld::empty();
        this.read(stream, &parent, &root);
        this
    }

    // Read takes a reference to stream/parent/root, this may change later.
    fn read<S: Stream>(
        &mut self,
        stream: &mut S,
        parent: &Option<Box<Struct>>,
        root: &Option<Box<Struct>>,
    ) {
        self.one = stream.read_u1().unwrap();
    }
}

#[cfg(test)]
mod tests {
    use kaitai_struct::Struct;
    use HelloWorld;

    #[test]
    fn test_hello_world() {
        let hw = HelloWorld::from_file("./src/fixed_struct.bin")
            .expect("File not found!");
        assert_eq!(hw.one, 0x50);
    }
}

I'm going to start working out the parent and root references, if I can figure that out then I'm hoping that it should be pretty basic stuff after that.

GreyCat commented 7 years ago

Looks really cool :)

As for code location, I'd just leave it here at the moment. It's not like putting it somewhere into the tests repo would help anything — we need test specs, we need the parser code to be generated, etc, so the hand-written generation result doesn't really help much in terms of CI or stuff like that.

May be we could ask some Rust online communities for some input / suggestions on this one?

berkus commented 7 years ago

I've implemented some parts in Rust, inspired by kaitai cpp generator here.

Maybe this gives you new ideas (matching .ksy file is here)

LogicAndTrick commented 7 years ago

Neat, you wouldn't by any chance know a good way to represent the parent/root references in Rust? I have looked at it a few times and it just seems to create a painful chain of types like Option<Weak<RefCell<Rc<T>>>> and the interface it creates seems pretty unreasonable for somebody to want to use.

berkus commented 7 years ago

It's pretty painful because of borrow checker. Usually peeps do it via a Vec and integer indices pointing to parent or children from a T.

Probably some rust tree package like ego_tree or idtree may give you help?

Otherwise I believe RefCell<Rc<T>> is the only way (but i'm no expert - been doing rust for barely two weeks now)

CWood1 commented 6 years ago

Since I don't know the status of this, I decided to start an alternate implementation. Still very much a WIP, but hello world now successfully compiles to this:

// This is a generated file! Please edit source .ksy file and use kaitai-struct-compiler to rebuild

use std::{
    option::Option,
    boxed::Box,
    io::Result
};

use kaitai_struct::{
    KaitaiStream,
    KaitaiStruct
};

pub struct HelloWorld {
    pub one: u8,
}

impl KaitaiStruct for HelloWorld {
    fn new<S: KaitaiStream>(stream: &mut S,
                            _parent: &Option<Box<KaitaiStruct>>,
                            _root: &Option<Box<KaitaiStruct>>)
                            -> Result<Self>
        where Self: Sized {
        let mut s = Self {
            one: 0,
        };

        s.read(stream, _parent, _root)?;

        Ok(s)
    }

    fn read<S: KaitaiStream>(&mut self,
                             stream: &mut S,
                             _parent: &Option<Box<KaitaiStruct>>,
                             _root: &Option<Box<KaitaiStruct>>)
                             -> Result<()>
        where Self: Sized {
        self.one = stream.read_u1()?;

        Ok(())
    }
}

I've tested this and confirmed it all working. My next step is to port some of the test suite over, writing test harnesses for Rust. From there, I can get them all passing, one by one.

GreyCat commented 6 years ago

@CWood1 Thanks, looks really cool! Note that:

CWood1 commented 6 years ago

Oh wow, that all looks much easier than I'd anticipated! Is there any documentation on generating and running the tests for a given language automatically? Tried to figure it out from travis.yml, but that does strange things with packages which won't work on my Arch installation.

Thanks for the help, I'll take a look at getting the test translator ported next.

GreyCat commented 6 years ago

Unfortunately, not now, as this whole thing was literally implemented a month or so ago. Travis yaml won't help much here, as specs are to autogenerated on the fly during normal build process.

To run test translator, you'd generally want to:

  1. Go to compiler dir
  2. sbt publishLocal to build compiler jars and "publish" it into local repo where translator project could find it
  3. Go to translator dir
  4. sbt run (or load that project into IDE and run it there) to try autogenerating all possible specs:
    • Sources for KST specs are in spec/kst
    • Results would be in spec/kst/out/$LANG
  5. Check generated specs and move it manually to spec/$LANG after checking that it actually does the job
CWood1 commented 6 years ago

So, I've got the translator outputting tests now. Few compilation issues here and there due to incorrect syntax, nothing too extreme.

Next question: how exactly does one run the compiler against the test suite, then the tests against the compiled specs? Once I get these compiler errors resolved, that's my next goal is to have 50something failing tests I can then make pass.

GreyCat commented 6 years ago

@CWood1 Could you show me what exactly have you achieved so far?

CWood1 commented 6 years ago

Okay, where we're at (I'll be pushing all this shortly, soon as I've got a couple compiler errors ironed out), I've got a rust library project under spec/rust. Inside of there, tests/ houses all of the tests.

Therefore, open questions:

Doing the above manually is relatively easy, though tedious. Are there ways by which I can script them, under existing frameworks? Or would that need to be developed from scratch?

GreyCat commented 6 years ago

How do I get the compiler to run, and output the results into spec/rust/src?

There is a huge mess with terminology we have here ;)

If by "compiler", you mean ksc (i.e. app that translates ksy files into parsing source code in target language), then normally, it is run with build-formats script, it runs the compiler on all formats for all target languages, and outputs results into compiled/$LANG.

If by "compiler", you mean kst (i.e. app that translates kst files into test specs in target languages), then it is right now runnable by invoking sbt run in its dir (or, equivalently, launching its main class in IDE).

I suspect there's slight misunderstanding on what needs to be done. Actually we have 3 "places" to update:

That's why I asked you to show what you've got, so I could understand which of these 3 things you were working on :)

Can I script updating spec/rust/src/lib.rs, to publicly declare all of the compiled modules in a way the tests can access?

No equivalent of these exists so far in any other languages. C++ might benefit from similar automation, but so far we've just added all tests manually to CMake project. This is actually a somewhat sloppy moment, but for compiled languages you don't usually have an option to go with "build everything that would build, and run the tests with everything's that we've built". Normally (for C#, C++, Java, Go), a single broken file ruins the build, so we employ various tricks to reduce that risk. Having a manually maintained list of tests that "should run ok" is a first step to mitigate that problem.

Is there a framework by which I can automatically run the above, and then invoke cargo test?

ci-all script invokes all ci-* scripts in sequence. We should probably create ci-rust, which would run thte tests and deliver some kind of (ideally, JUnit-like XML) report to test_out/.

CWood1 commented 6 years ago

Okay, that makes a little more sense now. To be explicit, for the first point, I was referring to ksc - I need to run ksc in order to generate the rust to then test with.

My situation is as follows:

Using these, I've generated a suite of tests from the kst sources, and put them in a directory, spec/rust/tests. spec/rust is a crate (rust library) for use as a testing harness.

As I see it, the next things that need to happen are as follows:

As an extension, I may look at autogenning the test harness itself - as mentioned, lib.rs needs to link to every Rust file under test, and other languages could benefit from similar automation as you mentioned.

It may even be worth attempting to update the test translator itself, to allow for outputting of multiple files. Under this framework, it would be possible to create an individual crate for every file under test. This eliminates the problem of a syntax error in one file entirely halting the testnig procedure, and other compiled langauges would also benefit from this as well.

I believe that should carry me to the end of this particular project, though if I've missed anything, please let me know. I'll post more updates on here as each of these happen, so my progress is visible to the world. Thank you so much for your help, it's very very much appreciated.

CWood1 commented 6 years ago

As a progress update on the above, I used Cargo's build script facilities to automatically copy over everything from compiled/rust into spec/rust/src.

It first clears the src directory. Then, one by one it copies the files across, building lib.rs in the process. This removes the need to manually specify lib.rs, as it now updates itself every time the test suite is executed.

GreyCat commented 6 years ago

As of right now, I'm yet to work out how to represent nullAssert in Rust,

Null assert is a comparison of some nullable values with null. "Nullable" are mostly the values that have if condition which can be false, and thus there would be no value, and typically we have some sort of null equivalent returned there.

and am entirely unsure what trueArrayAssert is, however these will be worked out in time.

"True" arrays are arrays which result from repeat-style constructs, i.e. arrays of some type. Usually they are represented by some sort of ArrayList, std::vector or similar dynamically resizable generic array type.

Opposed to them, "non-true", byte arrays are a special type which usually results from just specifying size: X without any type. They are typically special, as it's possible to handle them in much more efficient manner: many languages allow some kind of byte[] non-dynamic array or String-like type with strings of bytes.

Typically, comparison and assertion of "byte arrays" is done in exactly the same manner as comparison of other primitive types, but comparison of arrays frequently requires some special call like "assertDeepEquals" or something like that, that will compare values inside the container, not just the identity of containers.

GreyCat commented 6 years ago

It may even be worth attempting to update the test translator itself, to allow for outputting of multiple files. Under this framework, it would be possible to create an individual crate for every file under test. This eliminates the problem of a syntax error in one file entirely halting the testnig procedure, and other compiled langauges would also benefit from this as well.

Test translator already outputs one file per test/spec. The idea to compile every file as a separate test suite and launch it separately is definitely a solution, but its usefulness would vary from one language to another. For example, running ~140 executables for C++ is possible, but for Java it would be very slow. Right now, a test run takes ~1.4 s, but launching separate JVM for every suite would be 140x times slower => i.e. ~3 minutes.

We actually employ some crazy techniques like the one you can see in run-java, lines 31-67, i.e. we try to compile everything in one go, but if it fails, we try compiling each file in separate compiler invocation, which is obviously very slow (i.e. ~140x times slower), but it allows us to isolate the culprit(s), mark it/them up as "failed to compile" and proceed with running the rest of the tests.

If you could think of better alternatives, especially cross-language, that would be most welcome. So far the only way I see is creation of some sort of "universal build" system, which could handle all languages, but this is clearly a huge overkill for a task like this.

CWood1 commented 6 years ago

Okay, now we have a major issue. Because of the way Kaitai works, it expects structs/types/classes/whatever you want to call them, to be nested. Becuase of the way Rust works, they can't be. I'm yet to work out a solution to this.

Initially my thoughts were to collect each struct into some class somewhere, and have each class output itself in turn, but as far as fields go, there's no indication of what belongs to what. So I'm not entirely convinced.

GreyCat commented 6 years ago

@CWood1 That's not a really major issue, quite a few languages work that way. There are 2 distinct problems here, I'm not sure both apply to Rust:

class Foo {
  class Foo__Bar {
    class Foo__Bar__Baz {
    }
  }
}

you can set up innerClasses to false, for example, as PerlCompiler does. This would result in an order of calls to be like that:

class Foo {
}
class Foo__Bar {
}
class Foo__Bar__Baz {
}

This innerClasses thing actually affects the order of calls in compileClass method. If you want something completely different, you can always subclass ClassCompiler (for example, we do that for Go), and implement it differently.

CWood1 commented 6 years ago

Is there a way I can get it to output fields before methods? Because of the way Rust works, it needs

struct Foo {
// fields
}

impl KaitaiStruct for Foo {
// methods
}
GreyCat commented 6 years ago

Yes, it's slightly more complicated. To achieve that, one needs to subclass ClassCompiler and override compileClass method with different order of calls.

Go actually does exactly that — i.e. "class declaration" comes first and a Read() method and instance methods come after that.

In order to register a ClassCompiler-derived class, though, one needs to add it to Main.compile as well.

CWood1 commented 6 years ago

For full names, such as Foo__Bar, how does one then use that as a type? In that example, the type used in the Foo struct definition is Bar, and I've not the foggiest how to attach Foo__ to that.

As a general update, however, RustClassCompiler has now been defined, and compilation order works. A slew of errors have been ironed out, and progress is generally getting made on all fronts. In theory, the test suite should compile soon.

GreyCat commented 6 years ago

In that example, the type used in the Foo struct definition is Bar,

Could you show me the exact code that generates Bar and not Foo__Bar?

A slew of errors have been ironed out, and progress is generally getting made on all fronts.

That's really cool! Would you mind sharing your branch in public — may be I can review it and throw it some ideas, and may be help with integration into the CI?

CWood1 commented 6 years ago

The relevant code is in tests and compiler. I'll post links to the name stuff tomorrow, when I'm not about to sleep :P

Thank you once again, so very much, for your help in this project. I couldn't have gotten nearly this far without you. You've been truly invaluable. Thank you.

CWood1 commented 6 years ago

Update: a bunch more issues have now been resolved. I managed to figure out the issue with class naming and type naming, so that all works. My next issue is to figure out the fact that sometimes parent and root are needed in attribute readers, and sometimes multi-level parents are needed. That means somehow storing both in the struct itself, which the lifetime manager has already cried at me for. I have a few suggestions from colleagues though, which I'll research into a little more.

I'm also yet to work out how switch_bytearray should work. It isn't possible to directly compare vectors as one can do in other languages. Clearly it must be possible to have a special comparison for things such as vectors, for languages like Java and C++, though I'm yet to find it. If someone can point me to that before I find it manually, I'd be grateful, but until then I'll keep looking.

GreyCat commented 6 years ago

@CWood1 What would you think about merging your current effort into master, and making it appear in CI and tests?

Clearly it must be possible to have a special comparison for things such as vectors, for languages like Java and C++, though I'm yet to find it. If someone can point me to that before I find it manually, I'd be grateful, but until then I'll keep looking.

There's no good magic there, this is how JavaCompiler does it:

CWood1 commented 6 years ago

@CWood1 What would you think about merging your current effort into master, and making it appear in CI and tests?

So, at the minute, the tests don't acutally compile and run. I'd like to get the tests compiling first, if possible, because otherwise I fear I may actually break CI :stuck_out_tongue:. Feel free to disagree with me on that one, though.

I'll take a look into switchIfs shortly, and see how far I get with that. That should be enough to clear the next couple of hurdles at least.

Thanks so much for all of your help. Sorry things have slowed down considerably from the start of this project, I've started a new job which has my attention during the week, and I don't think they'd be too happy if I just carried on with my own stuff on the clock.

CWood1 commented 6 years ago

I'm alive, just to let everyone know. Sorry about the long hiatus, personal things got in the way that took priority.

I've fixed the switch stuff, and that's now working. Among the errors I'm now getting are problems with process_custom.ksy - it tries to import nested::deeply and my_custom_fx::MyCustomFx. Naturally, these don't exist, which results in a compilation error. I checked the other tests, doesn't exist there either, yet somehow they pass, so clearly something is being done at some level to make that work.

Where should these exist? Are these just "magic" files that need hand writing and checking into the test repo? Or should they be generated somehow, somewhere?

GreyCat commented 6 years ago

@CWood1 I'm super glad to hear that you're ok and still motivated to continue ;)

process_custom is described in the docs. This is indeed a way to invoke language-specific processing routine where it's needed, so these files you're missing are indeed hand-written. For example,

We're actually developing something called kaitai_compress, which is a collection of wrapper routines related to data compression/decompression, implemented for different languages, compatible with this "custom processing" idea.

CWood1 commented 6 years ago

I've taken a quick look at that, it looks like I'm going to have to add CustomDecoder to the KaitaiStruct Rust crate before I can make any more headway on this specific problem. That doesn't look too hard.

Roadmap so far:

Right now that's as far as I can tell, the undervars and warnings are responsible, between the two, for a few thousand errors, so it gets a bit hard to see the forest through the trees with those around.

The obvious things that need to happen once we compile is to fix any failing tests once everything compiles, then to formalise both crates and get them submitted to crates.io.

On the subject of crates.io, how would one go about that? KaitaiStruct is not yet ready, and I've obviously not started the other one at all, but once they're formalised and v1.0, how would I get them on the repo under the Kaitai project? Does someone have to submit it under the project, and I don't worry about it? Or do I do it under my own email?

GreyCat commented 6 years ago

Sorry for delayed response.

Roadmap so far:

I would actually think of getting your work merged into the master repo as top priority. Could you create a PR?

Look into creating a kaitai-compress Rust crate

This is very experimental thing, so probably a very distant goal.

On the subject of crates.io, how would one go about that? KaitaiStruct is not yet ready, and I've obviously not started the other one at all, but once they're formalised and v1.0, how would I get them on the repo under the Kaitai project?

https://github.com/kaitai-io/kaitai_struct_rust_runtime/ already exists under kaitai-io, I guess. I can add it into main umbrella project as a submodule whenever you'll say so.

Does someone have to submit it under the project, and I don't worry about it? Or do I do it under my own email?

If you mean crates.io and similar submissions, we can create special account for that, I guess, something like crates@kaitai.io? That domain's mail is currently served by zohomail, so there would be a web interface for that and all that stuff...

CWood1 commented 6 years ago

PRs has been created, and since my last message that's all the work I've had the time to do, I'm afraid.

That makes sense about the crate, on both points, I assume I can just ping someone on here or something and that'll get done? Or, failing that, I'd be happy to look after the Rust crate if needed, and run crates@kaitai.io accordingly. This is all a long way off either which way.

berkus commented 6 years ago

To publish just use the agreed email and run cargo login / cargo package / cargo publish - see here.

jonstewart commented 5 years ago

Coming at this with no context, is there anything I could work on? I'd be very glad to have Rust support in Kaitai.

GreyCat commented 5 years ago

@jonstewart Rust support was contributed by @CWood1, and the next major milestone would be to ensure that its tests run properly in CI. Unfortunately, I myself know very little about Rust, and my priorities for v0.9 lie in other tasks. If you'd be able to pick it up, that would be awesome ;)

CWood1 commented 5 years ago

Progress is being made, I'm sorry for the radio silence. I've been trying to fix the issue with the junit crate which has had me blocked. I'll be redoubling my effort in the new year.

CWood1 commented 5 years ago

If I do a little bit of reorganising, I'm hoping to be able to get the initial version merged in sometime in January.

jonstewart commented 5 years ago

@CWood1 Thanks for the update! I'd be happy to help, especially if you can point me at starting places. I have a modicum of Scala experience, a smidge of Rust (with a desire to learn more), and a heavy background in C++ and dealing with binary data. I'm trying to get my team to adopt both Kaitai and Rust, so am eager to learn more and participate.

CWood1 commented 5 years ago

So, to give full context to what's going on right now, cargo-test-junit appears to be dead. I've been trying other ways to get the xml file Travis needs, to no avail, so just now I've forked it. Reason being, one of its dependencies appears to be out of date. I'll get that updated and in, either today or tomorrow. At that point, updating the Kaitai repo to use the new dep will be nice and straight forward.

From there, process_custom hasn't been implemented at all, there's currently no way yet to walk back up the tree or process from root, and I'd like to look at getting the output nostd if I can, as a stretch goal. More work may end up surfacing as these are completed, but that's where we're at right now.

You probably won't be able to help very much until the tests are in and successfully working, though if my new way works, it won't be January, it'll be like tomorrow. Following on from that though, given you're not so familiar with Rust yet, process_custom will probably be easier, and I'll play with some weak pointers to try and get root/etc working.

Hang fire for now, I'm hoping I can speed this along and actually get some code merged (I'm getting impatient on that myself). Once that's done, things will pick up a lot more steam again, and hopefully not take too much longer to get this project finished.

GreyCat commented 5 years ago

Guys, just a little ahead warning: we're slowly transitioning from old CI to new CI (URL is probably subject to change).

The main difference between old CI and new CI is that:

The workflow for new one is similar and tests still live in kaitai_struct_tests repo, what is changing is:

If you'd be interested in doing Rust CI support the same way, it would probably require:

and that's it. Format compilation results (i.e. compiled/rust/*.rs) are already available, waiting to be launched.

CWood1 commented 5 years ago

Ack, I'll factor that in to my work. You've actually reminded me, there also needs to be a crate that invokes kaitai from build.rs, so we've got proper cargo integration.

I do wonder if that can be used for the test framework later on as well.