Add support for parsing multiple XTCE files

[jdiez17]

Closes #1. Seems to work for a very simple case. Not yet tested with the ACubeSAT XTCE files.

Note: there may be a more rust-idiomatic way of reading the XTCE files and creating roxmltree::Documents. The problem is that Document borrows the XML content string, so the content needs to be stored in a Vec first, and then we can create Document instances. Suggestions to make this better welcome.

[jdiez17]

Also managed to parse ACubeSAT XTCE definitions

use std::path::Path;
use xtce_rs::{mdb::MissionDatabase, parser, proc::containers::process};
use env_logger;

fn main() {
    env_logger::init();
    let mut mdb = MissionDatabase::new();

    let paths = [
        "yamcs-instance/src/main/yamcs/mdb/common/dt/base-dt.xml",
        "yamcs-instance/src/main/yamcs/mdb/common/dt/dt.xml",
        "yamcs-instance/src/main/yamcs/mdb/common/pus/pus.xml",
        "yamcs-instance/src/main/yamcs/mdb/comms/comms-dt.xml",
        "yamcs-instance/src/main/yamcs/mdb/comms/comms-xtce.xml",
        "yamcs-instance/src/main/yamcs/mdb/obc/obc-dt.xml",
        "yamcs-instance/src/main/yamcs/mdb/obc/obc-xtce.xml",
        "yamcs-instance/src/main/yamcs/mdb/adcs/adcs-xtce.xml",
        "yamcs-instance/src/main/yamcs/mdb/services/ST[03].xml"
    ].map(Path::new);

    let mdb = parser::parse_files(&paths).unwrap();
}

[xpromache]

Thanks!

If you would add instead of just a list of files, also a mdb_path:String next to each file, then you can attach the files not at the root but at random positions in the MDB tree - I thought it will be more complicated but looking now at the code I think all is required is in the parse_files instead of starting from an QualifiedName::empty(), to make a name based on the mdb_path associated to each doc.

Then it would be more compatible with Yamcs java.

BTW: you can change the original parse function to get rid of the mutable mdb argument and return the mdb instead (like your new parse_files).

BTW2: if you want to continue from where I left off with the implementation of the TM packet processing, let me know, I can hopefully provide some hints. My dream was to make it fully compatible with the Yamcs java (although I expect difficulties in referencing user defined algorithms).

[jdiez17]

I see, now I understand what you mean by a tree of files instead of a flat list. I guess being able to "insert" a XTCE definition at a nesting level greater than the root would allow you to split up your whole MDB definition into multiple files arbitrarily. Not exactly sure how that would work in practice, if you could link an example that would be very helpful.

I think the approach in the ACubeSAT XTCE files (which seems to be based in some Yamcs examples) of having multiple SpaceSystems, one per file, which reference each other is OK... but a bit limited; in that you have to define the whole set of "related" TM/TCs in one file.

if you want to continue from where I left off with the implementation of the TM packet processing, let me know, I can hopefully provide some hints

Yeah, I would like to continue with the implementation. I would actually start by adding parsing of command metadata, that seems to be mostly missing. And then I would also like to implement encoding of containers rather than only decoding.

BTW my goal with this is to write onboard software in Rust that uses the same MDB definition as the ground segment.

Your thoughts on any of this would be highly welcome!

[jdiez17]

BTW: you can change the original parse function to get rid of the mutable mdb argument and return the mdb instead (like your new parse_files).

should we modify the existing parse function? I would rather delete it and have a single API. The parse_files arguments may get a bit trickier than just a slice of Paths when we add a mdb_path though (for the single-xtce-file case)

[xpromache]

I see, now I understand what you mean by a tree of files instead of a flat list. I guess being able to "insert" a XTCE definition at a nesting level greater than the root would allow you to split up your whole MDB definition into multiple files arbitrarily. Not exactly sure how that would work in practice, if you could link an example that would be very helpful.

I cannot share a file but one system where this was useful was composed of multiple on-board computers, all running cFS (some NASA on-board software framework). The computers (nodes) had a set of common TM/TC for common functions and then each had in addition its own particular TM/TC list. For the common part, they reused the XTCE files by defining them for each node /node1 /node2. So you would end up with /node1/table_managemnt_tc, /node2/table_management_tc, etc having only one file defining the table_management_tc.

I think the approach in the ACubeSAT XTCE files (which seems to be based in some Yamcs examples) of having multiple SpaceSystems, one per file, which reference each other is OK... but a bit limited; in that you have to define the whole set of "related" TM/TCs in one file.

I think you can actually split it if you eliminate the condition of uniquely defined SpaceSystem Practically this: https://github.com/xpromache/xtce-rs/blob/master/src/parser/nametree.rs#L195 will return a duplicate name error but you can just not add it if it's already there. I would put this feature behind an option because allowing duplicated definitions of SpaceSystems may be confusing.

if you want to continue from where I left off with the implementation of the TM packet processing, let me know, I can hopefully provide some hints

Yeah, I would like to continue with the implementation. I would actually start by adding parsing of command metadata, that seems to be mostly missing. And then I would also like to implement encoding of containers rather than only decoding.

BTW my goal with this is to write onboard software in Rust that uses the same MDB definition as the ground segment.

Your thoughts on any of this would be highly welcome!

One issue you will face is that parsing commands out of binary packets cannot be generally done unambiguously. You can easily have multiple command definitions that return in the same binary.

I din't do much on-board software but XTCE seems a bit too generic for that. NASA cFS seem to be pushing for something called EDS (Electronic Data Sheet) for achieving this. Maybe you can have a look at that also.