PRELIMINARY DECODING COMPARISON

sirbastiano commented 3 days ago

Dear Antonio, as previously discussed, on the left the s1isp decoder and on the right RichHall decoder.

Seems there are some fields that are different like the ones for the Nominal Tx Replica.

They should refer to the same record but I will double check..

avalentino commented 3 days ago

Thanks a lot @sirbastiano

Two points:

the data class fields of the s1isp decoder always refer to the values stored in the data. Very often such values shall be interpreted according to what described in the S1-IF-ASD-PL-0007, "Sentinel-1 SAR Space Packet Protocol Data Unit", Issue 13. In such cases properties or functions are provided in the s1isp source code implementing such conversion rules (see, e.g., the tx_pulse_length in your example). The key point is to compare the proper values with the results of the RichHall's sentinel1decoder
apparently you are comparing different ISPs: packet_sequence_count, space_packet_count and pri_count all look different in the left vs right column of the comparison. This looks strange to me.

sirbastiano commented 3 days ago

@avalentino, thanks, gotcha now! Brilliant as always! 🚀 .. double checked and I was comparing different ISPs.

These indeed are the important differences on the same ISPs:

('Packet Sequence Count', 1.0, 129.0) ('Packet Data Length', 7654.0, 7653.0) ('Fine Time', 0.35900115966796875, 23527.0) ('Rx Gain', -6.0, 12.0) ('Tx Ramp Rate', 1344932774550.9956, 34770.0) ('Tx Pulse Start Frequency', -29704503.224123616, 12970.0) ('Tx Pulse Length', 4.41724329115483e-05, 1658.0) ('PRI', 0.0005194923216780943, 19499.0) ('SWST', 9.593783529221075e-05, 3601.0) ('SWL', 4.683663272527256e-05, 1758.0)

Aside for the conversion of codes to physical properties, I found just these two differences:

Packet sequence count for richall wraps around 64, so when yours turns 65 the richall turns 1 after that. Same happens at 129 and so on....
Packet Data Length for richa is always +1 the one of s1isp

I put the headers extracted by richall and by s1isp on the product (S1A_S3_RAW__0SDH_20240524T213606_20240524T213631_054018_069139_241A.SAFE) in HH pol here:

https://drive.google.com/drive/folders/1MizXHu869IZCdhLo3LB_KDJpV2Tg8XwW?usp=sharing

For reference, the very stupid code used for comparison:

import pandas as pd

meta_rich.read_pickle('meta_rich.pkl')
meta_s1isp.read_pickle('meta_s1isp.pkl')

richa_to_s1isp = {
    "Packet Version Number": "packet_version_number",
    "Packet Type": "packet_type",
    "Secondary Header Flag": "secondary_header_flag",
    "PID": "pid",
    "PCAT": "pcat",
    "Sequence Flags": "sequence_flags",
    "Packet Sequence Count": "packet_sequence_count",
    "Packet Data Length": "packet_data_length",
    "Coarse Time": "coarse_time",
    "Fine Time": "fine_time",
    "Sync": "sync_marker",
    "Data Take ID": "data_take_id",
    "ECC Number": "ecc_num",
    "Test Mode": "test_mode",
    "Rx Channel ID": "rx_channel_id",
    "Instrument Configuration ID": "instrument_configuration_id",
    "Sub-commutated Ancilliary Data Word Index": "data_word_index",
    "Space Packet Count": "space_packet_count",
    "PRI Count": "pri_count",
    "Error Flag": "error_flag",
    "BAQ Mode": "baq_mode",
    "BAQ Block Length": "baq_block_length",
    "Range Decimation": "range_decimation",
    "Rx Gain": "rx_gain",
    "Tx Ramp Rate": "tx_ramp_rate",
    "Tx Pulse Start Frequency": "tx_pulse_start_freq",
    "Tx Pulse Length": "tx_pulse_length",
    "Rank": "rank",
    "PRI": "pri",
    "SWST": "swst",
    "SWL": "swl",
    "SAS SSB Flag": "ssb_flag",
    "Polarisation": "polarization",
    "Temperature Compensation": "temperature_compensation",
    "Calibration Mode": "cal_mode",
    "Tx Pulse Number": "tx_pulse_number",
    "Signal Type": "signal_type",
    "Swap Flag": "swap",
    "Swath Number": "swath_number",
    "Number of Quads": "number_of_quads"
}

VERBOSE = False

idx = 65

row_rich = meta_rich.iloc[idx]
row_s1isp = meta_s1isp.iloc[idx]

keys = list(row_rich.keys())

non_equal_elements = []

for key in keys:
    try: 
        elem_rich = row_rich[key]
        elem_s1isp = row_s1isp[richa_to_s1isp[key]]

        if VERBOSE:
            print(key, elem_rich, float(elem_s1isp), elem_rich == float(elem_s1isp))

        if not elem_rich == float(elem_s1isp):
            non_equal_elements.append((key, float(elem_rich), float(elem_s1isp)))

    except:
        print('Missing key:', key)
        continue

for elem in non_equal_elements:
    print(elem)

sirbastiano commented 3 days ago

I think that a flag like enum_value for parsing the stream and turning the codes into physical params would benefit the community.

avalentino commented 3 days ago

Aside for the conversion of codes to physical properties, I found just these two differences:

Packet sequence count for richall wraps around 64, so when yours turns 65 the richall turns 1 after that. Same happens at 129 and so on....

Packet Data Length for richa is always +1 the one of s1isp

For the first one I would need to check the specification document. For the second it is still a case of codes vs computed parameters values. The specification doc clearly says that the actual "Packet Data Length" is the decoded value plus one.

avalentino / s1isp

PRELIMINARY DECODING COMPARISON #8