Add the ability to create simulated packets

[ehsteve]

It is often very helpful to be able to create simulated packets especially in the early phases of development or for testing. The packet types FixedLength and VariableLength could be expanded to include a function (to_file?) to output binary data if given arrays of data to fill in the packet fields.

to_file(data: dict)

where dict includes the packet field names as indices and numpy arrays for the values. The data dict should mirror the data dict that is provided by parsing the file.

[ehsteve]

This issue is being worked in the packet_encoding branch. Basic functionality is already implemented and ready to be tested by anyone interested!

[tloubrieu-jpl]

@ddasilva , @ehsteve , I've seen this ticket is still open (but developed) and I was wondering if we could as well think of generating a documentation for the CCSDS packets from the ccsdspy objects.

If I refer to the CCSDS packet documentation I am using, the only missing bit for simple packets would be the 'description' , of the fields. For example:

    ccsdspy.PacketField(
            name="Instrument SCLK Time second", bit_length=32, data_type="uint", description="instrument clock in second since 2010-01-01"
        ),

For the converters it might be more complicated to document them from the code but we could re-use the doc-strings.

Having the documentation and the test files generated from a single reference, as code, would avoid discrepancies between them. That would also allow to manage the reference documentation (as code) in configuration, which is easier to track changes.

Did you have thoughts on that already ?

[ddasilva]

I'm open to this-- it really has no impact to existing users and it would be pretty nice to generate pretty HTML. Having the definitions expressed in python might not be the best authoritative source for the packet definitions in a large project, but if they're already in Python its is a nice bonus feature.

What do you think @ehsteve ? Would the feature proposed in the last comment be useful for you @jmbhughes? Maybe PUNCH could release nicely formatted HTML documentation of the packet definitions.

[jmbhughes]

~At the moment, we are just reading the definitions from a CSV file and were going to distribute that, but it might be nice to have them presented nicely in the website. If it was pretty and didn't require too much additional work, we'd use it probably.~ We may not be releasing the code that reads from CCSDS packets, but I still support adding this feature.

We also may try out this simulated packet creation to test our pipeline soon.

[tloubrieu-jpl]

Hi @ddasilva , @ehsteve ,

I might have a few hours to try to work on the code provided by @ehsteve to generate test datasets for EuropaClipper SDS and replace an ad-hoc script that I am not really able to maintain anymore.

Is there a plan to merge this code in the main branch ? If I validate that it works for me, would that help ?

Thanks,

Thomas

[ddasilva]

Yes! If you validate this works and log any remaining issues that will help a lot. It probably has a few merge conflicts since it was forked a while ago.