Problems exporting and importing XTCE

stanislaw commented 3 years ago

Hello,

First of all, thank you for creating this tool. We are evaluating CCDD in combination with COSMOS and NASA cFS.

At this point, we are running CCDD from CCDD.jar and also we have set up running CCDD from Intellij IDEA (we can step through the code with a debugger). The commit is f1641de.

Problem 1: Importing and exporting XTCE does not work when running CCDD from jar

We take an XTCE example from the CCDD-2/Docs/CCDD_Users_Guide.pdf (page 200).
We save the XTCE content to an .xtce file.
We open the CCDD program and try to import the file via "Data / Import data / XTCE".
We select the file and click ok.
The dialog disappears and then nothing happens in the user interface, nothing is printed to the message log except:

Importing table(s) from .../guide.xtce

We see no tables in the Data / Edit table(s)

The exporting does not work - after we click ok on the Export dialog - the dialog box closes and nothing happens (file is not created, no log message).

Problem 2: Running CCDD from IDEA IDE

The steps are the same as in the description above but the manifestation is different:
We are getting an SQL error CCDD : Database Error : Cannot retrieve TABLE_DATA_VAR_NAMES list
The file contents seem to be imported even though we cannot say if the imported data is consistent.

We are ready to assist you in the investigation of this issue.

The stack trace from IDEA:

Command that failed: SELECT * from find_command_arguments('{Structure: Cmd Arg Ref,XTCE Structure,XTCE Command}');
CCDD : Database Error : Cannot retrieve TABLE_DATA_VAR_NAMES list
Command that failed: SELECT key FROM mycommand WHERE key = (SELECT MAX(key) FROM mycommand);
Command that failed: INSERT INTO mycommand (key, index, command_name, command_code, description, command_argument) VALUES (1, '1', 'NoOp', '', 'No operation', ''), (2, '2', 'EngineArmEnableDisable', '', 'Arm main engine', ''), (3, '3', 'EngineFireEnableDisable', '', 'Fire main engine', ''), (4, '4', 'EngineThrustLevel', '', 'Engine thrust level', ''); 
java.sql.SQLException: Invalid SQL command; ERROR: relation "mycommand" does not exist
  Position: 19
    at CCDD.CcddDbCommandHandler.executeDbStatement(CcddDbCommandHandler.java:263)
    at CCDD.CcddDbCommandHandler.executeDbQuery(CcddDbCommandHandler.java:117)
    at CCDD.CcddDbTableCommandHandler.buildAndExecuteAdditionCommand(CcddDbTableCommandHandler.java:2882)
    at CCDD.CcddDbTableCommandHandler.modifyTableData(CcddDbTableCommandHandler.java:2556)
    at CCDD.CcddFileIOHandler.createImportedTable(CcddFileIOHandler.java:2522)
    at CCDD.CcddFileIOHandler.createTablesFromDefinitions(CcddFileIOHandler.java:2255)
    at CCDD.CcddFileIOHandler.importFiles(CcddFileIOHandler.java:1680)
    at CCDD.CcddFileIOHandler$3.execute(CcddFileIOHandler.java:1440)
    at CCDD.CcddBackgroundCommand$1.doInBackground(CcddBackgroundCommand.java:148)
    at CCDD.CcddBackgroundCommand$1.doInBackground(CcddBackgroundCommand.java:141)
    at javax.swing.SwingWorker$1.call(SwingWorker.java:295)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at javax.swing.SwingWorker.run(SwingWorker.java:334)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

Screenshot from 2021-07-28 14-37-45

<?xml version="1.0" encoding="UTF-8" standalone="no"?> 
<SpaceSystem xmlns="http://www.omg.org/space/xtce" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
name="User_s_Guide" shortDescription="User's Guide" 
xsi:schemaLocation="https://www.omg.org/spec/XTCE/20061101 https://www.omg.org/spec/XTCE/20061101/06-11-
06.xsd">
  <LongDescription>Project used for the import and export example in Appendix C of the user's 
guide</LongDescription> 
   <AncillaryDataSet> 
      <AncillaryData name="XML: Application ID">applicationID</AncillaryData> 
      <AncillaryData name="XML: Function Code">functionCode</AncillaryData> 
   </AncillaryDataSet> 
   <Header classification="DOMAIN" date="Mon Apr 15 12:49:03 CDT 2019" validationStatus="Working" 
version="1.0"> 
      <AuthorSet> 
         <Author>rmcclune</Author> 
      </AuthorSet> 
      <NoteSet> 
         <Note>CCDD Version: 1.5.2 (4-15-2019)</Note> 
         <Note>Date: Mon Apr 15 12:49:03 CDT 2019</Note> 
         <Note>Project: User's Guide</Note> 
         <Note>Host: localhost:5432</Note> 
         <Note>Endianess: big</Note> 
      </NoteSet> 
   </Header> 
   <SpaceSystem name="MyCommand" shortDescription="MyCommand"> 
      <Header classification="INTERFACE" validationStatus="Working" version="1.0"/> 
      <CommandMetaData> 
         <ArgumentTypeSet> 
            <EnumeratedArgumentType name="ARM_Type"> 
               <UnitSet/> 
               <IntegerDataEncoding bitOrder="mostSignificantBitFirst" encoding="unsigned" sizeInBits="1"/> 
               <EnumerationList> 
                  <Enumeration label="Enable" value="0"/> 
                  <Enumeration label="Disable" value="1"/> 
               </EnumerationList> 
            </EnumeratedArgumentType> 
            <EnumeratedArgumentType name="FIRE_Type"> 
               <UnitSet/> 
               <IntegerDataEncoding bitOrder="mostSignificantBitFirst" encoding="unsigned" sizeInBits="1"/> 
               <EnumerationList> 
                  <Enumeration label="Enable" value="0"/> 
                  <Enumeration label="Disable" value="1"/> 
               </EnumerationList> 
            </EnumeratedArgumentType> 
            <FloatArgumentType name="THRUST_LEVEL_Type" sizeInBits="32"> 
               <LongDescription>Set thrust level</LongDescription>
               <UnitSet> 
                  <Unit>percent</Unit> 
               </UnitSet> 
               <FloatDataEncoding encoding="IEEE754_1985" sizeInBits="32"/> 
               <ValidRangeSet> 
                  <ValidRange maxExclusive="100.0" minExclusive="0.0"/> 
               </ValidRangeSet> 
            </FloatArgumentType> 
         </ArgumentTypeSet> 
         <MetaCommandSet> 
            <MetaCommand name="NoOp"> 
               <LongDescription>No operation</LongDescription> 
            </MetaCommand> 
            <MetaCommand name="EngineArmEnableDisable"> 
               <LongDescription>Arm main engine</LongDescription> 
               <ArgumentList> 
                  <Argument argumentTypeRef="ARM_Type" name="ARM"/> 
               </ArgumentList> 
               <CommandContainer name="EngineArmEnableDisable"> 
                  <EntryList> 
                     <ArgumentRefEntry argumentRef="ARM"/> 
                  </EntryList> 
               </CommandContainer> 
            </MetaCommand> 
            <MetaCommand name="EngineFireEnableDisable"> 
               <LongDescription>Fire main engine</LongDescription> 
               <ArgumentList> 
                  <Argument argumentTypeRef="FIRE_Type" name="FIRE"/> 
               </ArgumentList> 
               <CommandContainer name="EngineFireEnableDisable"> 
                  <EntryList> 
                     <ArgumentRefEntry argumentRef="FIRE"/> 
                  </EntryList> 
               </CommandContainer> 
            </MetaCommand> 
            <MetaCommand name="EngineThrustLevel"> 
               <LongDescription>Engine thrust level</LongDescription> 
               <ArgumentList> 
                  <Argument argumentTypeRef="THRUST_LEVEL_Type" name="THRUST_LEVEL"/> 
               </ArgumentList> 
               <CommandContainer name="EngineThrustLevel">
                  <EntryList> 
                     <ArgumentRefEntry argumentRef="THRUST_LEVEL"/> 
                  </EntryList> 
               </CommandContainer> 
            </MetaCommand> 
         </MetaCommandSet> 
      </CommandMetaData> 
   </SpaceSystem> 
   <SpaceSystem name="MyStructure" shortDescription="MyStructure"> 
      <Header classification="INTERFACE" validationStatus="Working" version="1.0"/> 
      <TelemetryMetaData> 
         <ParameterTypeSet> 
            <FloatParameterType name="latitude_Type" sizeInBits="32"> 
               <LongDescription>Location: north-south</LongDescription> 
               <UnitSet/> 
               <FloatDataEncoding encoding="IEEE754_1985" sizeInBits="32"/> 
            </FloatParameterType> 
            <FloatParameterType name="longitude_Type" sizeInBits="32"> 
               <LongDescription>Location: east-west</LongDescription> 
               <UnitSet/> 
               <FloatDataEncoding encoding="IEEE754_1985" sizeInBits="32"/> 
            </FloatParameterType> 
            <IntegerParameterType name="width_Type" signed="false" sizeInBits="16"> 
               <UnitSet> 
                  <Unit>meters</Unit> 
               </UnitSet> 
               <IntegerDataEncoding bitOrder="mostSignificantBitFirst" encoding="unsigned" 
sizeInBits="16"/> 
            </IntegerParameterType> 
            <IntegerParameterType name="depth_Type" signed="false" sizeInBits="16"> 
               <UnitSet> 
                  <Unit>meters</Unit> 
               </UnitSet> 
               <IntegerDataEncoding bitOrder="mostSignificantBitFirst" encoding="unsigned" 
sizeInBits="16"/> 
            </IntegerParameterType> 
            <IntegerParameterType name="height_Type" signed="false" sizeInBits="16"> 
               <UnitSet> 
                  <Unit>meters</Unit> 
               </UnitSet>
               <IntegerDataEncoding bitOrder="mostSignificantBitFirst" encoding="unsigned" 
sizeInBits="16"/> 
            </IntegerParameterType> 
            <ArrayParameterType arrayTypeRef="velocity_Type" name="velocity_Array" numberOfDimensions="1"/> 
            <FloatParameterType name="velocity_Type" sizeInBits="64"> 
               <UnitSet/> 
               <FloatDataEncoding encoding="IEEE754_1985" sizeInBits="64"/> 
            </FloatParameterType> 
            <EnumeratedParameterType name="engine_arm_Type"> 
               <LongDescription>Engine armed status</LongDescription> 
               <UnitSet/> 
               <IntegerDataEncoding bitOrder="mostSignificantBitFirst" encoding="unsigned" sizeInBits="1"/> 
               <EnumerationList> 
                  <Enumeration label="Off" value="0"/> 
                  <Enumeration label="Arm" value="1"/> 
               </EnumerationList> 
            </EnumeratedParameterType> 
            <EnumeratedParameterType name="engine_fire_Type"> 
               <LongDescription>Engine fire status</LongDescription> 
               <UnitSet/> 
               <IntegerDataEncoding bitOrder="mostSignificantBitFirst" encoding="unsigned" sizeInBits="1"/> 
               <EnumerationList> 
                  <Enumeration label="Off" value="0"/> 
                  <Enumeration label="Fire" value="1"/> 
               </EnumerationList> 
            </EnumeratedParameterType> 
            <IntegerParameterType name="thrust_level_Type" signed="false" sizeInBits="16"> 
               <LongDescription>Engine thrust level</LongDescription> 
               <UnitSet> 
                  <Unit>percent</Unit> 
               </UnitSet> 
               <IntegerDataEncoding bitOrder="mostSignificantBitFirst" encoding="unsigned" 
sizeInBits="16"/> 
               <ValidRange maxInclusive="100" minInclusive="0"/> 
            </IntegerParameterType> 
         </ParameterTypeSet> 
         <ParameterSet> 
            <Parameter name="latitude" parameterTypeRef="latitude_Type"/> 
            <Parameter name="longitude" parameterTypeRef="longitude_Type"/> 
            <Parameter name="width" parameterTypeRef="width_Type"/> 
            <Parameter name="depth" parameterTypeRef="depth_Type"/>
            <Parameter name="height" parameterTypeRef="height_Type"/> 
            <Parameter name="velocity" parameterTypeRef="velocity_Array"/> 
            <Parameter name="engine_arm" parameterTypeRef="engine_arm_Type"/> 
            <Parameter name="engine_fire" parameterTypeRef="engine_fire_Type"/> 
            <Parameter name="thrust_level" parameterTypeRef="thrust_level_Type"/> 
         </ParameterSet> 
         <ContainerSet> 
            <SequenceContainer name="MyStructure"> 
               <EntryList> 
                  <ParameterRefEntry parameterRef="latitude"/> 
                  <ParameterRefEntry parameterRef="longitude"/> 
                  <ParameterRefEntry parameterRef="width"/> 
                  <ParameterRefEntry parameterRef="depth"/> 
                  <ParameterRefEntry parameterRef="height"/> 
                  <ArrayParameterRefEntry parameterRef="velocity"> 
                     <DimensionList> 
                        <Dimension> 
                           <StartingIndex> 
                              <FixedValue>0</FixedValue> 
                           </StartingIndex> 
                           <EndingIndex> 
                              <FixedValue>2</FixedValue> 
                           </EndingIndex> 
                        </Dimension> 
                     </DimensionList> 
                  </ArrayParameterRefEntry> 
                  <ParameterRefEntry parameterRef="engine_arm"/> 
                  <ParameterRefEntry parameterRef="engine_fire"/> 
                  <ParameterRefEntry parameterRef="thrust_level"/> 
               </EntryList> 
            </SequenceContainer> 
         </ContainerSet> 
      </TelemetryMetaData> 
   </SpaceSystem> 
</SpaceSystem>

stanislaw commented 3 years ago

Just tried doing the same on a more fresh Ubuntu machine (importing the sample XTCE file to a clean new project):

.../CCDD (CCDD-2)*$ git log -1
commit f1641deb805a64a6b336989f11737eac82f23f8f (HEAD -> CCDD-2, origin/CCDD-2)
Author: bawillisJSC <77999883+bawillisJSC@users.noreply.github.com>
Date:   Thu Jun 17 13:21:40 2021 -0500

    Update README.md

$ java -jar CCDD.jar -logPath logs
Command that failed: DELETE FROM __dbu_info;
Command that failed: SELECT * from find_command_arguments('{Structure: Cmd Arg Ref,XTCE Structure,XTCE Command}');
CCDD : Database Error : Cannot retrieve TABLE_DATA_VAR_NAMES list
Command that failed: SELECT _key_ FROM mycommand WHERE _key_ = (SELECT MAX(_key_) FROM mycommand);
java.sql.SQLException: Invalid SQL command; ERROR: relation "mycommand" does not exist
  Position: 19
    at CCDD.CcddDbCommandHandler.executeDbStatement(CcddDbCommandHandler.java:263)
    at CCDD.CcddDbCommandHandler.executeDbQuery(CcddDbCommandHandler.java:117)
    at CCDD.CcddDbTableCommandHandler.buildAndExecuteAdditionCommand(CcddDbTableCommandHandler.java:2882)
    at CCDD.CcddDbTableCommandHandler.modifyTableData(CcddDbTableCommandHandler.java:2556)
    at CCDD.CcddFileIOHandler.createImportedTable(CcddFileIOHandler.java:2522)
    at CCDD.CcddFileIOHandler.createTablesFromDefinitions(CcddFileIOHandler.java:2255)
    at CCDD.CcddFileIOHandler.importFiles(CcddFileIOHandler.java:1680)
    at CCDD.CcddFileIOHandler$3.execute(CcddFileIOHandler.java:1440)
    at CCDD.CcddBackgroundCommand$1.doInBackground(CcddBackgroundCommand.java:148)
    at CCDD.CcddBackgroundCommand$1.doInBackground(CcddBackgroundCommand.java:1)
    at javax.swing.SwingWorker$1.call(SwingWorker.java:295)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at javax.swing.SwingWorker.run(SwingWorker.java:334)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Command that failed: INSERT INTO mycommand (_key_, _index_, command_name, command_code, description, command_argument) VALUES (1, '1', 'NoOp', '', 'No operation', ''), (2, '2', 'EngineArmEnableDisable', '', 'Arm main engine', ''), (3, '3', 'EngineFireEnableDisable', '', 'Fire main engine', ''), (4, '4', 'EngineThrustLevel', '', 'Engine thrust level', '');

Screenshot from 2021-07-28 19-02-50

$ java -version
openjdk version "1.8.0_292"
OpenJDK Runtime Environment (build 1.8.0_292-8u292-b10-0ubuntu1~18.04-b10)
OpenJDK 64-Bit Server VM (build 25.292-b10, mixed mode)

$ sudo update-alternatives --config java
[sudo] password for stanislaw: 
There are 2 choices for the alternative java (providing /usr/bin/java).

  Selection    Path                                            Priority   Status
------------------------------------------------------------
  0            /usr/lib/jvm/java-11-openjdk-amd64/bin/java      1111      auto mode
  1            /usr/lib/jvm/java-11-openjdk-amd64/bin/java      1111      manual mode
* 2            /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java   1081      manual mode

The current master branch is confirmed to work (exporting and importing XTCE without errors)

commit a26585e646cc869f80a3deeab869de68e25da20e (HEAD -> master, origin/master, origin/HEAD)
Author: Bryan Willis <bryan.willis@uky.edu>
Date:   Thu Nov 5 17:36:53 2020 -0500

    Quick fix for an issue that would not allow you to create and open a new database.

bawillisJSC commented 3 years ago

Looking into this issue. Thanks for providing so many details that will help isolate the problem.

bawillisJSC commented 3 years ago

CCDD version 2.0.27 addresses this issue.

nasa / CCDD

Problems exporting and importing XTCE #91