search

yamcs / quickstart

Get started with Yamcs
https://yamcs.org
20 stars 21 forks source link

Handling multi packet receiving #7

Closed bruno-luca closed 2 years ago

bruno-luca commented 2 years ago

Hi! I'm new to Yamcs but i find it extremely powerful. I'm having some trouble while implementing some features in my project. Let me explain what i want to do:

I have a python script that read from 2 CSV files some diagnostic information and some telemetry information. This data is packed into two differents CCSDS packets using SpacePyLibrary, and then those packets are sequentially sent to the same Yamcs DataLink via UDP.

I have specifiedn in src/main/yamcs/mdb/xtce.xml my own parameters, as follow:

...

<ParameterSet>
      <Parameter name="CCSDS_Packet_ID" parameterTypeRef="CCSDS_Packet_ID_Type" />
      <Parameter name="CCSDS_Packet_Sequence" parameterTypeRef="CCSDS_Packet_Sequence_Type" />
      <Parameter name="CCSDS_Packet_Length" parameterTypeRef="CCSDS_Packet_Length_Type" />

      <Parameter name="Enum_Para_1" parameterTypeRef="EnumerationTest" />
      <Parameter name="Enum_Para_2" parameterTypeRef="EnumerationTest" />
      <Parameter name="Enum_Para_3" parameterTypeRef="EnumerationTest" />

      <!--  TELEMETRY PARAMETERS    -->

      <Parameter name="gps_lat" parameterTypeRef="float_t" />
      <Parameter name="gps_lon" parameterTypeRef="float_t" />
      <Parameter name="gps_alt" parameterTypeRef="float_t" />

      <Parameter name="gyro_x" parameterTypeRef="float_t" />
      <Parameter name="gyro_y" parameterTypeRef="float_t" />
      <Parameter name="gyro_z" parameterTypeRef="float_t" />

      <Parameter name="acc_x" parameterTypeRef="float_t" />
      <Parameter name="acc_y" parameterTypeRef="float_t" />
      <Parameter name="acc_z" parameterTypeRef="float_t" />

      <Parameter name="mag_x" parameterTypeRef="float_t" />
      <Parameter name="mag_y" parameterTypeRef="float_t" />
      <Parameter name="mag_z" parameterTypeRef="float_t" />

      <Parameter name="roll" parameterTypeRef="float_t" />
      <Parameter name="pitch" parameterTypeRef="float_t" />
      <Parameter name="yaw" parameterTypeRef="float_t" />

      <Parameter name="temp" parameterTypeRef="float_t" />
      <Parameter name="pres" parameterTypeRef="float_t" />

      <!--  DIAGNOSTIC PARAMETERS   -->

      <Parameter name="cpu_temp" parameterTypeRef="float_t" />
      <Parameter name="cpu_user_time" parameterTypeRef="float_t" />
      <Parameter name="cpu_system_time" parameterTypeRef="float_t" />
      <Parameter name="cpu_idle_time" parameterTypeRef="float_t" />
      <Parameter name="cpu_percent" parameterTypeRef="float_t" />
      <Parameter name="cpu_freq" parameterTypeRef="float_t" />

      <Parameter name="mem_used_perc" parameterTypeRef="float_t" />
      <Parameter name="mem_used" parameterTypeRef="float_t" />
      <Parameter name="mem_available" parameterTypeRef="float_t" />
      <Parameter name="mem_active" parameterTypeRef="float_t" />
      <Parameter name="mem_inactive" parameterTypeRef="float_t" />
      <Parameter name="mem_swap_used_perc" parameterTypeRef="float_t" />
      <Parameter name="mem_swap_used" parameterTypeRef="float_t" />
      <Parameter name="mem_swap_available" parameterTypeRef="float_t" />

      <Parameter name="disk_used_perc" parameterTypeRef="float_t" />
      <Parameter name="disk_used" parameterTypeRef="float_t" />
      <Parameter name="disk_free" parameterTypeRef="float_t" />

      <Parameter name="conn_bytes_sent" parameterTypeRef="float_t" />
      <Parameter name="conn_bytes_recv" parameterTypeRef="float_t" />
      <Parameter name="conn_packets_sent" parameterTypeRef="float_t" />
      <Parameter name="conn_packets_recv" parameterTypeRef="float_t" />
      <Parameter name="conn_errin" parameterTypeRef="float_t" />
      <Parameter name="conn_errout" parameterTypeRef="float_t" />
      <Parameter name="conn_down_bitrate" parameterTypeRef="float_t" />
      <Parameter name="conn_up_bitrate" parameterTypeRef="float_t" />

</ParameterSet>

<ContainerSet>
    <SequenceContainer abstract="true" name="CCSDSPacket">
        <EntryList>
            <ParameterRefEntry parameterRef="CCSDS_Packet_ID" />
            <ParameterRefEntry parameterRef="CCSDS_Packet_Sequence" />
            <ParameterRefEntry parameterRef="CCSDS_Packet_Length" />
        </EntryList>
    </SequenceContainer>
    <SequenceContainer name="TelemetryPacket">
        <EntryList>
            <ParameterRefEntry parameterRef="gps_lat" />
            <ParameterRefEntry parameterRef="gps_lon" />
            <ParameterRefEntry parameterRef="gps_alt" />
            <ParameterRefEntry parameterRef="gyro_x"/>
            <ParameterRefEntry parameterRef="gyro_y"/>
            <ParameterRefEntry parameterRef="gyro_z"/>
            <ParameterRefEntry parameterRef="acc_x"/>
            <ParameterRefEntry parameterRef="acc_y"/>
            <ParameterRefEntry parameterRef="acc_z"/>
            <ParameterRefEntry parameterRef="mag_x"/>
            <ParameterRefEntry parameterRef="mag_y"/>
            <ParameterRefEntry parameterRef="mag_z"/>
            <ParameterRefEntry parameterRef="roll" />
            <ParameterRefEntry parameterRef="pitch" />
            <ParameterRefEntry parameterRef="yaw" />
            <ParameterRefEntry parameterRef="pres" />
            <ParameterRefEntry parameterRef="temp" />
        </EntryList>
        <BaseContainer containerRef="CCSDSPacket">
            <RestrictionCriteria>
                <ComparisonList>
                    <Comparison value="0" parameterRef="CCSDS_Packet_ID/Version" />
                    <Comparison value="TM" parameterRef="CCSDS_Packet_ID/Type" />
                </ComparisonList>
            </RestrictionCriteria>
        </BaseContainer>
    </SequenceContainer>

    <SequenceContainer name="DiagnosticPacket">
        <EntryList>
            <ParameterRefEntry parameterRef="cpu_temp" />
            <ParameterRefEntry parameterRef="cpu_user_time" />
            <ParameterRefEntry parameterRef="cpu_system_time" />
            <ParameterRefEntry parameterRef="cpu_idle_time" />
            <ParameterRefEntry parameterRef="cpu_percent" />
            <ParameterRefEntry parameterRef="cpu_freq" />
            <ParameterRefEntry parameterRef="mem_used_perc" />
            <ParameterRefEntry parameterRef="mem_used" />
            <ParameterRefEntry parameterRef="mem_available" />
            <ParameterRefEntry parameterRef="mem_active" />
            <ParameterRefEntry parameterRef="mem_inactive" />
            <ParameterRefEntry parameterRef="mem_swap_used_perc" />
            <ParameterRefEntry parameterRef="mem_swap_used" />
            <ParameterRefEntry parameterRef="mem_swap_available" />
            <ParameterRefEntry parameterRef="disk_used_perc" />
            <ParameterRefEntry parameterRef="disk_used" />
            <ParameterRefEntry parameterRef="disk_free" />
            <ParameterRefEntry parameterRef="conn_bytes_sent" />
            <ParameterRefEntry parameterRef="conn_bytes_recv" />
            <ParameterRefEntry parameterRef="conn_packets_sent" />
            <ParameterRefEntry parameterRef="conn_packets_recv" />
            <ParameterRefEntry parameterRef="conn_errin" />
            <ParameterRefEntry parameterRef="conn_errout" />
            <ParameterRefEntry parameterRef="conn_down_bitrate" />
            <ParameterRefEntry parameterRef="conn_up_bitrate" />
        </EntryList>
        <BaseContainer containerRef="CCSDSPacket">
            <RestrictionCriteria>
                <ComparisonList>
                    <Comparison value="0" parameterRef="CCSDS_Packet_ID/Version" />
                    <Comparison value="TM" parameterRef="CCSDS_Packet_ID/Type" />
                </ComparisonList>
            </RestrictionCriteria>
        </BaseContainer>
    </SequenceContainer>
    <SequenceContainer name="Spacecraft">
        <EntryList>
        </EntryList>
        <BaseContainer containerRef="TelemetryPacket">
            <RestrictionCriteria>
                <ComparisonList>
                    <Comparison value="NotPresent" parameterRef="CCSDS_Packet_ID/SecHdrFlag" />
                    <Comparison value="100" parameterRef="CCSDS_Packet_ID/APID" />
                </ComparisonList>
            </RestrictionCriteria>
        </BaseContainer>
    </SequenceContainer>
</ContainerSet>

...

This is the python script reading from file and sending to Yamcs:

...
for i in range(1,1000):
        try:
            telemetry_pack = CubeHAPSTelemetryPacket()
            diagnostic_pack = CubeHAPSDiagnosticPacket()

            tm_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

            telem = t_file.readline()
            diag = d_file.readline()

            telem = t_file.readline().strip().split(';')
            diag = d_file.readline().strip().split(';')

            #print(f"\n\ntelemetry: {telemetry}\ndiagnostic: {diagnostic}\n--------------------------------------------------------------------------------\n")
            #print(len(diagnostic))

            #SET TELEMETRY PARAMETERS
            telemetry_pack.packet.gps_lat = float(telem[1])
            telemetry_pack.packet.gps_lon = float(telem[2])
            telemetry_pack.packet.gps_alt = float(telem[3])
            telemetry_pack.packet.gyro_x = float(telem[4])
            telemetry_pack.packet.gyro_y = float(telem[5])
            telemetry_pack.packet.gyro_z = float(telem[6])
            telemetry_pack.packet.acc_x = float(telem[7])
            telemetry_pack.packet.acc_y = float(telem[8])
            telemetry_pack.packet.acc_z = float(telem[9])
            telemetry_pack.packet.mag_x = float(telem[10])
            telemetry_pack.packet.mag_y = float(telem[11])
            telemetry_pack.packet.mag_z = float(telem[12])
            telemetry_pack.packet.roll = float(telem[13])
            telemetry_pack.packet.pitch = float(telem[14])
            telemetry_pack.packet.yaw = float(telem[15])
            telemetry_pack.packet.pres = float(telem[16])
            telemetry_pack.packet.temp = float(telem[17])
            telemetry_pack.packet.setPacketLength()

            print(f"""
            SENDING:

            gps_lat = {telem[1]}
            gps_lon = {telem[2]}
            gps_alt = {telem[3]}
            gyro_x = {telem[4]}
            gyro_y = {telem[5]}
            gyro_z = {telem[6]}
            acc_x = {telem[7]}
            acc_y = {telem[8]}
            acc_z = {telem[9]}
            mag_x = {telem[10]}
            mag_y = {telem[11]}
            mag_z = {telem[12]}
            roll = {telem[13]}
            pitch = {telem[14]}
            yaw = {telem[15]}
            pres = {telem[16]}
            temp = {telem[17]}

            """)

            tm_socket.sendto(telemetry_pack.getBytes(), ('127.0.0.1', 10015))
            sleep(3)
            #SET DIAGNOSTIC PARAMETERS
            diagnostic_pack.packet.cpu_temp = float(diag[1])
            diagnostic_pack.packet.cpu_user_time = float(diag[2])
            diagnostic_pack.packet.cpu_system_time = float(diag[3])
            diagnostic_pack.packet.cpu_idle_time = float(diag[4])
            diagnostic_pack.packet.cpu_percent = float(diag[5])
            diagnostic_pack.packet.cpu_freq = float(diag[6])
            diagnostic_pack.packet.mem_used_perc = float(diag[7])
            diagnostic_pack.packet.mem_used = float(diag[8])
            diagnostic_pack.packet.mem_available = float(diag[9])
            diagnostic_pack.packet.mem_active = float(diag[10])
            diagnostic_pack.packet.mem_inactive = float(diag[11])
            diagnostic_pack.packet.mem_swap_used_perc = float(diag[12])
            diagnostic_pack.packet.mem_swap_used = float(diag[13])
            diagnostic_pack.packet.mem_swap_available = float(diag[14])
            diagnostic_pack.packet.disk_used_perc = float(diag[15])
            diagnostic_pack.packet.disk_used = float(diag[16])
            diagnostic_pack.packet.disk_free = float(diag[17])
            diagnostic_pack.packet.conn_bytes_sent = float(diag[18])
            diagnostic_pack.packet.conn_bytes_recv = float(diag[19])
            diagnostic_pack.packet.conn_packets_sent = float(diag[20])
            diagnostic_pack.packet.conn_packets_recv = float(diag[21])
            diagnostic_pack.packet.conn_errin = float(diag[22])
            diagnostic_pack.packet.conn_errout = float(diag[23])
            diagnostic_pack.packet.conn_down_bitrate = float(diag[24])
            diagnostic_pack.packet.conn_up_bitrate = float(diag[25])
            diagnostic_pack.packet.setPacketLength()

            print(f"""

            cpu_temp = {diag[1]}
            cpu_user_time = {diag[2]}
            cpu_system_time = {diag[3]}
            cpu_idle_time = {diag[4]}
            cpu_percent = {diag[5]}
            cpu_freq = {diag[6]}
            mem_used_perc = {diag[7]}
            mem_used = {diag[8]}
            mem_available = {diag[9]}
            mem_active = {diag[10]}
            mem_inactive = {diag[11]}
            mem_swap_used_perc = {diag[12]}
            mem_swap_used = {diag[13]}
            mem_swap_available = {diag[14]}
            disk_used_perc = {diag[15]}
            disk_used = {diag[16]}
            disk_free = {diag[17]}
            conn_bytes_sent = {diag[18]}
            conn_bytes_recv = {diag[19]}
            conn_packets_sent = {diag[20]}
            conn_packets_recv = {diag[21]}
            conn_errin = {diag[22]}
            conn_errout = {diag[23]}
            conn_down_bitrate = {diag[24]}
            conn_up_bitrate = {diag[25]}

            -------------------------------------------------------------------

            """)

            tm_socket.sendto(diagnostic_pack.getBytes(), ('127.0.0.1', 10015))

            #print(f"sended:\ntelemetry: {telem}\ndiagnostic: {diag[5]}\n\n--------------------------------------------------------------------------\n")
        except Exception as e:
            print('Closing...')

        sleep(3)
        os.system('clear')
...

when i send the first packet, it also changes some diagnostic parameters, that should not be modified by that packet. Following there is a screenshot of my parameters status after the first packet has been sent. As you can see telemetry field contained in the packet have been modified correctly, but also some diagnostic field, like cpu status, has been changed from the first packet while those should just be modified from the second one.

Screenshot from 2022-03-03 10-17-24

The same happen when the second packet is sent. Diagnostic parameters are modified correctly, but some telemetry fields (such as gyroscope states) get modified too:

Screenshot from 2022-03-03 10-20-48

How can i tell Yamcs only to update field that are present in the packet received?

fqqb commented 2 years ago

Hi, it seems that both TelemetryPacket and DiagnosticPacket have the same BaseContainer configuration.

For Yamcs to be able to distinguish how to decode incoming binary, it needs another criteria. For example you could use a different APID for each packet type. Your sender script will need to be able to set the appropriate APID field in the primary CCSDS header.

Something like this:

<SequenceContainer name="TelemetryPacket">
    <EntryList>
        <ParameterRefEntry parameterRef="gps_lat" />
        <!-- ... -->
    </EntryList>
    <BaseContainer containerRef="CCSDSPacket">
        <RestrictionCriteria>
            <ComparisonList>
                <Comparison value="0" parameterRef="CCSDS_Packet_ID/Version" />
                <Comparison value="TM" parameterRef="CCSDS_Packet_ID/Type" />
                <Comparison value="100" parameterRef="CCSDS_Packet_ID/APID" />
            </ComparisonList>
        </RestrictionCriteria>
    </BaseContainer>
</SequenceContainer>
<SequenceContainer name="DiagnosticPacket">
    <EntryList>
        <ParameterRefEntry parameterRef="cpu_temp" />
        <!-- ... -->
    </EntryList>
    <BaseContainer containerRef="CCSDSPacket">
        <RestrictionCriteria>
            <ComparisonList>
                <Comparison value="0" parameterRef="CCSDS_Packet_ID/Version" />
                <Comparison value="TM" parameterRef="CCSDS_Packet_ID/Type" />
                <Comparison value="101" parameterRef="CCSDS_Packet_ID/APID" />
            </ComparisonList>
        </RestrictionCriteria>
    </BaseContainer>
</SequenceContainer>

There's some other solutions like using a custom type field discriminator in an intermediate container. But APID seems decent if you're not using it for other purposes.

bruno-luca commented 2 years ago

Thanks! It worked fro me!