search

OpenC3 / cosmos

OpenC3 COSMOS
https://openc3.com
Other
112 stars 31 forks source link

Receiving error when converting NASA's cFE XTCE XML's #1096

Open rogueWookie opened 9 months ago

rogueWookie commented 9 months ago

Hi ~

Environment

Issue

I'm using your XTCE converter on various NASA CFS XML files. It is my understanding that their XMLs (for example this one) conforms to the XTCE standard.

According to your docs XTCE support is built-in and I can either convert a XTCE XML to COSMOS format or use it directly in my cmd_tlm/ folder (so long as I rename it to a .xtce extension).

I am receiving the same errors regardless of which option I take. The first option here is invoking the converter. The second option is copying the cfe_sb.xml file into my cmd_tlm/ folder (renaming it with a .xtce extension) and then rebuilding my plugin.

OPTION 1 (Using the converter)
❯ openc3.sh cli xtce_converter --import cfe_sb.xml --output ./
Processing cfe_sb.xml...
  Ignoring Unknown: <PackageFile>
  Ignoring Unknown: <Package>
  Ignoring Unknown: <DataTypeSet>
  Ignoring Unknown: <IntegerDataType>
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:394:in `xtce_process_element': undefined method `xtce_encoding=' for nil:NilClass (NoMethodError)

        @current_type.xtce_encoding = element.name
                     ^^^^^^^^^^^^^^^^
    from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:82:in `block (2 levels) in parse'
    from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:740:in `xtce_recurse_element'
    from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:743:in `block in xtce_recurse_element'
    from /usr/lib/ruby/gems/3.2.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:235:in `block in each'
    from /usr/lib/ruby/gems/3.2.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in `upto'
    from /usr/lib/ruby/gems/3.2.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in `each'
    from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:742:in `xtce_recurse_element'
    from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:743:in `block in xtce_recurse_element'
    from /usr/lib/ruby/gems/3.2.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:235:in `block in each'
    from /usr/lib/ruby/gems/3.2.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in `upto'
    from /usr/lib/ruby/gems/3.2.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in `each'
    from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:742:in `xtce_recurse_element'
    from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:743:in `block in xtce_recurse_element'
    from /usr/lib/ruby/gems/3.2.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:235:in `block in each'
    from /usr/lib/ruby/gems/3.2.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in `upto'
    from /usr/lib/ruby/gems/3.2.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in `each'
    from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:742:in `xtce_recurse_element'
    from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:81:in `block in parse'
    from /usr/lib/ruby/gems/3.2.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:235:in `block in each'
    from /usr/lib/ruby/gems/3.2.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in `upto'
    from /usr/lib/ruby/gems/3.2.0/gems/nokogiri-1.16.0-x86_64-linux/lib/nokogiri/xml/node_set.rb:234:in `each'
    from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:80:in `parse'
    from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:69:in `initialize'
    from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:41:in `new'
    from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/parsers/xtce_parser.rb:41:in `process'
    from /usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/packets/packet_config.rb:124:in `process_file'
    from /openc3/bin/openc3cli:251:in `xtce_converter'
    from /openc3/bin/openc3cli:676:in `<main>'
OPTION 2 (Using the file directly)
❯ openc3.sh cli rake build VERSION=6.0.0
gem build openc3-cosmos-cfs-sandbox
openc3cli validate openc3-cosmos-cfs-sandbox-6.0.0.gem
  Successfully built RubyGem
  Name: openc3-cosmos-cfs-sandbox
  Version: 6.0.0
  File: openc3-cosmos-cfs-sandbox-6.0.0.gem
Installing openc3-cosmos-cfs-sandbox-6.0.0.gem
  Ignoring Unknown: <PackageFile>
  Ignoring Unknown: <Package>
  Ignoring Unknown: <DataTypeSet>
  Ignoring Unknown: <IntegerDataType>
Error: 
RuntimeError : Error processing /tmp/d20240213-11-22j7da/CFS_SANDBOX/cmd_tlm/cfe_sb.xtce:
undefined method `xtce_encoding=' for nil:NilClass
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/system/system.rb:157:in `add_target'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/system/system.rb:140:in `block in initialize'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/system/system.rb:140:in `each'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/system/system.rb:140:in `initialize'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/models/target_model.rb:608:in `new'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/models/target_model.rb:608:in `deploy'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/models/plugin_model.rb:238:in `block in install_phase2'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/config/config_parser.rb:217:in `parse_loop'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/config/config_parser.rb:217:in `parse_file'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/models/plugin_model.rb:231:in `install_phase2'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/../bin/openc3cli:313:in `validate_plugin'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/../bin/openc3cli:647:in `<main>'

plugin.txt20240213-11-g558np:9: MAP_TARGET CFS_SANDBOX
Error: Unknown keyword and parameters for Target: MAP_TARGET CFS_SANDBOX
Backtrace:
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/models/target_model.rb:558:in `handle_config'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/models/plugin_model.rb:245:in `block in install_phase2'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/config/config_parser.rb:217:in `parse_loop'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/config/config_parser.rb:217:in `parse_file'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/openc3/models/plugin_model.rb:231:in `install_phase2'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/../bin/openc3cli:313:in `validate_plugin'
/usr/lib/ruby/gems/3.2.0/gems/openc3-5.14.0/lib/../bin/openc3cli:647:in `<main>'

[!NOTE] No idea what the error message _"Unknown keyword and parameters for Target: MAP_TARGET CFSSANDBOX" is implying. I've been building COSMOS plugins just fine lately with ingesting cFS sample app telemetry/commands. Here's my plugin.txt if it's of any interest...

VARIABLE cfs_sandbox_target_name CFS_SANDBOX
VARIABLE cmd 1234
VARIABLE tlm 1235
TARGET CFS_SANDBOX <%= cfs_sandbox_target_name %>
INTERFACE <%= cfs_sandbox_target_name %>_INT udp_interface.rb host.docker.internal <%= cmd %> <%= tlm %> nil nil nil 10.0 nil
MAP_TARGET <%= cfs_sandbox_target_name %>

I also tried other cFS files (e.g. cFS/cfe/modules/time/eds/cfe_time.xml) by typing openc3.sh cli xtce_converter --import cfe_time.xml --output ./ but got the same exact error about unknown method xtce_encodings=. Really wanted to have someone reproduce the error first from the cFS community (see this discussion) but this feels more related to the xtce_parser than their XMLs. Afraid I don't have a support contract but wanted to kindly report this if it is indeed a bug.

jmthomas commented 9 months ago

You'll notice in the output right at the top:

  Ignoring Unknown: <PackageFile>
  Ignoring Unknown: <Package>
  Ignoring Unknown: <DataTypeSet>
  Ignoring Unknown: <IntegerDataType>

Which means COSMOS doesn't know about those elements. Just for fun I added IntegerDataType to the xtce_parser at the same level as IntegerParameterType and IntegerArgumentType. This allowed things to continue but then I got a bunch more Ignoring Unknown statements. I'm afraid we just don't support these keywords.

ryanmelt commented 9 months ago

If you can provide the XTCE file, or at least the keywords used in it we can work on adding support

jmthomas commented 9 months ago

If you can provide the XTCE file, or at least the keywords used in it we can work on adding support

I was able to download the cfe_sb.xml from the link he provided.

rogueWookie commented 9 months ago

There's quite a few of them @ryanmelt but here they are.

Not sure these are relevant or not.

jmthomas commented 9 months ago

@rogueWookie please email us at support@openc3.com if you have a support contract and would like to prioritize implementing some of these. Otherwise you're more than welcome to edit our xtce_parser and submit a PR!