Issues with the XTCE export of OpenSatKit database

[xpromache]

Hello,

I've installed the OpenSatKIT (https://github.com/OpenSatKit/OpenSatKit) as per their instructions and I was trying to export the COSMOS database in XTCE.

I get some errors from the xtce_convert command:

nm@nm:~/OpenSatKit-master/cosmos$ xtce_converter --export /tmp/cx/
Loading COSMOS Configuration...
WARN: Unresolved specs during Gem::Specification.reset:
      rdoc (< 5, >= 3)
      rake (< 13, >= 0)
      coderay (~> 1.1.0)
      yard (< 0.10, >= 0.8, ~> 0.8)
      qtbindings (>= 4.8.6.2, ~> 4.8.6)
WARN: Clearing out unresolved specs.
Please report a bug if this causes problems.
Converting configuration to .xtce files in /tmp/cx/
/home/nm/.rbenv/versions/2.2.6/lib/ruby/gems/2.2.0/gems/cosmos-3.9.2/lib/cosmos/packets/packet_config.rb:337:in `block in to_xtce': undefined method `each' for nil:NilClass (NoMethodError)
        from /home/nm/.rbenv/versions/2.2.6/lib/ruby/gems/2.2.0/gems/cosmos-3.9.2/lib/cosmos/packets/packet_config.rb:302:in `each'
        from /home/nm/.rbenv/versions/2.2.6/lib/ruby/gems/2.2.0/gems/cosmos-3.9.2/lib/cosmos/packets/packet_config.rb:302:in `to_xtce'
        from /home/nm/.rbenv/versions/2.2.6/lib/ruby/gems/2.2.0/gems/cosmos-3.9.2/bin/xtce_converter:80:in `'
        from /home/nm/.rbenv/versions/2.2.6/bin/xtce_converter:23:in `load'
        from /home/nm/.rbenv/versions/2.2.6/bin/xtce_converter:23:in `
'

[xpromache]

I see that even with the error above there are actually files generated.

In cfe_evs.xtce, the string parameter PACKETID_APPNAME_Type seems like encoded in little endian and that doesn't make much sense - I can see it in the raw packets that it is just normal ordering (big endian). I also see the length is supposed to be Fixed but when I look in the packet it looks like null terminated.

      <xtce:StringParameterType name="PACKETID_APPNAME_Type" characterWidth="8" shortDescription="Application name. (Parent: Event packet information.)">
        <xtce:ByteOrderList>
          <xtce:Byte byteSignificance="0"/>
          <xtce:Byte byteSignificance="1"/>
          ....
          <xtce:Byte byteSignificance="18"/>
          <xtce:Byte byteSignificance="19"/>
        </xtce:ByteOrderList>
        <xtce:UnitSet/>
        <xtce:StringDataEncoding encoding="UTF-8">
          <xtce:SizeInBits>
            <xtce:Fixed>
              <xtce:FixedValue>160</xtce:FixedValue>
            </xtce:Fixed>
          </xtce:SizeInBits>
        </xtce:StringDataEncoding>

[xpromache]

One more issue: in the export, you make one XTCE file for each target with no link between them.

In the OpenSatKit configuration there are a bunch of targets on the same UDP interface and all follow the basic packet structure with the first two bytes CCSDS_STREAMID identifying basically the target (if my understanding is correct).

This information is lost in the XTCE export, because the exported packets (containers) do not share a parent container, they all redefine the basic packet structure.

[ghost]

I just downloaded the OpenSatKit and successfully exported to XTCE:

C:\git\OpenSatKit\cosmos>xtce_converter --export xtce
Loading COSMOS Configuration...
WARN: Unresolved specs during Gem::Specification.reset:
      rake (< 13, >= 0)
      yard (< 0.10, >= 0.8, ~> 0.8)
WARN: Clearing out unresolved specs.
Please report a bug if this causes problems.
Converting configuration to .xtce files in xtce
Success

Please ensure you're using the latest COSMOS version: 4.1.1.

[ghost]

All the items are encoded as LITTLE_ENDIAN because that's how OpenSatKit defines the packets:

TELEMETRY CFE_EVS EVENT_MSG_PKT <%= CfsKitConfig.processor_endian %> "Event Message Telemetry Packet"

Processor endian is defined here: https://github.com/OpenSatKit/OpenSatKit/blob/master/cosmos/lib/cfs_kit_config.rb which uses the value defined here: https://github.com/OpenSatKit/OpenSatKit/blob/master/cosmos/lib/hw_target.rb

[ghost]

You are correct in your last point that we process all targets independently when exporting to XTCE. Thus there will be no linkage between targets.

[xpromache]

What does it mean to encode a string in little endian? You have to read it backwards? Because in the raw packet dump I just see the string encoded in normal order.

[ghost]

You're right, that doesn't make sense. I think we have a bug with little endian strings: #701.