Fix file parsing for typetags which provide more that 1 datapoints per timestamp

[nitin710]

• Need a fix
  • EM - Gets 2 columns. powerStatus + RecordStratus
  • AK - Gets 2 columns. Packet number of Requesting typetag + requesting typetag
  • DO - Gets 2 columns. Typetag + overflow count
  • DC - Gets 2 columns.Typetag + cutoff count
  • RD- Gets 2 columns. TL + TU. But it can extend to n in the future

File name	Change
_AK	Needs a fix. Currently it is being parsed as any other data type, which is not correct LocalTimestamp EmotiBitTimestamp PacketNumber DataLength TypeTag ProtocolVersion DataReliability TransactionInitiatingPacketNumber TransactionTypetag
_EM	Needs a fix. Currently it is being parsed as any other data type, which is not correct LocalTimestamp EmotiBitTimestamp PacketNumber DataLength TypeTag ProtocolVersion DataReliability RecordingStatus PowerStatus

[connectedfuturelabs]

Details about DC file

• A raw DC packet looks something like EMOTIBIT_TIME, PACKET_NUMBER, DATA_POINTS, DC, 2, 100, EA,1,EL,1 // here, 1 indicates num of clipping events per stream
• Note that the DATA_POINTS=2x(#typetags that clipped)
• Since the parser does not currently handle "composite payloads", each element of the DC packet is added to a new line in the parsed format
• so the DC column in the parsed file looks something like (the number indicates how many clipping events were created for that typetag)

EA
1
EL
1

• instead of creating a single DC row indicating all typetags that got clipped (+num of clipping events) in a single row

EA	1	EL	1

[nitin710]

A raw DC packet may look something like (a mock reconstruction from the data from your example) 8649190,57393,4,DC,2,100,EA,1,EL,1

The "payload"( = data in the packet) has been highlighted in bold. Since this "payload" has 4 elements, the parsed file currently contains 4 rows. In this example,

1726017865.346473,3671153.500,57393,4,DC,2,100,EA
1726019524.644097,5330499.000,57393,4,DC,2,100,1
1726021183.941721,6989844.500,57393,4,DC,2,100,EL
1726022843.239344,8649190.000,57393,4,DC,2,100,1  // This is the correct timestamp

The correct DC timestamp corresponds to the last row that shares the same packet number (4th row, in this case). The other 3 values are erroneously interpolated and need the proposed fix to resolve this issue. If you look for the same timestamp in the _EA.csv file, you should find a value that lies outside the defined range. For EDA, the lower limit is ~0.03uS. As shown below (snippet from the EA file),there is a value which was marked by the EmotiBit as out-of-bound.

1726022842.016380,8647967.000,57196,2,EA,1,100,0.762820
1726022842.082378,8648033.000,57214,1,EA,1,100,0.807232
1726022842.149376,8648100.000,57228,2,EA,1,100,0.883354
1726022842.216374,8648167.000,57228,2,EA,1,100,1.129873
1726022842.282372,8648233.000,57245,1,EA,1,100,1.463219
1726022842.349370,8648300.000,57262,2,EA,1,100,1.714521
1726022842.416368,8648367.000,57262,2,EA,1,100,1.562693
1726022842.482366,8648433.000,57283,1,EA,1,100,0.103756
1726022842.549364,8648500.000,57297,2,EA,1,100,0.035919
1726022842.616362,8648567.000,57297,2,EA,1,100,0.070382
1726022842.682360,8648633.000,57314,1,EA,1,100,0.298969
1726022842.749359,8648700.000,57331,2,EA,1,100,0.361031
1726022842.816357,8648767.000,57331,2,EA,1,100,0.300021
1726022842.882355,8648833.000,57348,1,EA,1,100,0.292010
1726022842.949353,8648900.000,57362,2,EA,1,100,0.291669
1726022843.016351,8648967.000,57362,2,EA,1,100,0.288872
1726022843.082349,8649033.000,57380,1,EA,1,100,0.148077
1726022843.149347,8649100.000,57397,2,EA,1,100,0.037356
1726022843.216345,8649167.000,57397,2,EA,1,100,0.030255  // OUT_OF_BOUNDS
1726022843.282343,8649233.000,57414,1,EA,1,100,0.040042
1726022843.348841,8649299.500,57429,2,EA,1,100,0.157869
1726022843.415339,8649366.000,57429,2,EA,1,100,0.370860
1726022843.482337,8649433.000,57446,1,EA,1,100,0.297417
1726022843.548835,8649499.500,57464,2,EA,1,100,0.284288
1726022843.615334,8649566.000,57464,2,EA,1,100,0.282290
1726022843.682332,8649633.000,57481,1,EA,1,100,0.269177
1726022843.748830,8649699.500,57497,2,EA,1,100,0.266162
1726022843.815328,8649766.000,57497,2,EA,1,100,0.266528
1726022843.882326,8649833.000,57513,1,EA,1,100,0.213631
1726022843.948824,8649899.500,57529,2,EA,1,100,0.240438