Closed gth828r closed 11 months ago
FYI, I changed 18-e2sm.rs
to contain this in order to test:
#![allow(dead_code, unreachable_patterns, non_camel_case_types)]
mod e2sm {
include!(concat!(env!("OUT_DIR"), "/e2sm.rs"));
}
fn main() {
use asn1_codecs::{aper::AperCodec, PerCodecData};
eprintln!("E2SM");
env_logger::init();
// This will break
let decode_str = "080000000100000000002043514901200000";
// This one works if it is uncommented
//let decode_str = "08000000010000000000204351490120000000";
let decode_hex = hex::decode(decode_str).unwrap();
let mut codec_data = PerCodecData::from_slice_aper(&decode_hex);
let e2sm_kpmv3_pdu = e2sm::E2SM_KPM_IndicationMessage::aper_decode(&mut codec_data);
eprintln!("e2sm_kpmv3_pdu: {:#?}", e2sm_kpmv3_pdu.unwrap());
}
It is possible that things won't work on master because the untested REAL support that I added isn't working. But in a separate project, I have commented out all REAL values from the KPM spec and regenerated the code, and I see the same errors that I see with my REAL support added. The REAL support does not seem to affect this particular message in terms of changing the length of enum values or anything like that.
Lastly, these were my raw notes from debugging this. It includes the hex value of the message, the log messages, and what has been deserialized so far. If you cannot understand what is in here, feel free to ask me about it or ignore it -- I just wanted to share it in case it helps save time:
Full message: 080000000100000000002043514901200000
Bytes: 08
[2023-10-25T00:50:05Z TRACE trybuild007::e2sm] decode: E2SM_KPM_IndicationMessage
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::aper::decode] decode_sequence_header: extensible: true
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] PerCodecData: offset: 1
[2023-10-25T00:50:05Z TRACE trybuild007::e2sm] decode: E2SM_KPM_IndicationMessageIndicationMessage_formats
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::aper::decode] decode_choice_idx: lb: 0, ub: 1, extensible: true
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::common::decode::decode_internal] decode_constrained_whole_number: lb: 0, ub: 1
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] Decoding Bits as Integer. offset: 2, bits: 1
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] Decoded Value: 0
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] PerCodecData: offset: 3
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] PerCodecData: offset: 3
[2023-10-25T00:50:05Z TRACE trybuild007::e2sm] decode: E2SM_KPM_IndicationMessage_Format1
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::aper::decode] decode_sequence_header: extensible: true
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::common::decode] 2 optionals found
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] PerCodecData: offset: 6
[2023-10-25T00:50:05Z TRACE trybuild007::e2sm] decode: MeasurementData
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::aper::decode] decode_length_determinent:
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::common::decode::decode_internal] decode_constrained_length_determinent, lb: 1, ub: 65535
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::common::decode::decode_internal] decode_constrained_whole_number: lb: 1, ub: 65535
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] Aligning Codec Buffer with 2 bits
E2SM_KPM_IndicationMessage
E2SM_KPM_IndicationMessageIndicationMessage_formats
E2SM_KPM_IndicationMessage_Format1
----------------------------------------------
Bytes: 0000
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] Decoding Bits as Integer. offset: 8, bits: 16
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] Decoded Value: 0
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::common::decode::decode_internal] decoded length : 1
E2SM_KPM_IndicationMessage
E2SM_KPM_IndicationMessageIndicationMessage_formats
E2SM_KPM_IndicationMessage_Format1
MeasurementData (measurement data list length 1)
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] PerCodecData: offset: 24
[2023-10-25T00:50:05Z TRACE trybuild007::e2sm] decode: MeasurementDataItem
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::aper::decode] decode_sequence_header: extensible: true
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::common::decode] 1 optionals found
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] PerCodecData: offset: 26
[2023-10-25T00:50:05Z TRACE trybuild007::e2sm] decode: MeasurementRecord
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::aper::decode] decode_length_determinent:
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] Aligning Codec Buffer with 6 bits
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] Decoding Bits as Integer. offset: 33, bits: 7
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] Decoded Value: 1
----------------------------------------------
Bytes: 0001
E2SM_KPM_IndicationMessage
E2SM_KPM_IndicationMessageIndicationMessage_formats
E2SM_KPM_IndicationMessage_Format1
MeasurementData (measurement data list length 1)
MeasurementDataItem
MeasurementRecord (measurement record item list length 1)
Bytes: 00
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] PerCodecData: offset: 40
[2023-10-25T00:50:05Z TRACE trybuild007::e2sm] decode: MeasurementRecordItem
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::aper::decode] decode_choice_idx: lb: 0, ub: 2, extensible: true
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::common::decode::decode_internal] decode_constrained_whole_number: lb: 0, ub: 2
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] Decoding Bits as Integer. offset: 41, bits: 2
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] Decoded Value: 0
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] PerCodecData: offset: 43
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] PerCodecData: offset: 43
[2023-10-25T00:50:05Z TRACE trybuild007::e2sm] decode: MeasurementRecordItem_integer
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::aper::decode] decode_integer: Lower: Some(0) Upper:Some(4294967295) Extensible: false
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::common::decode::decode_internal] decode_constrained_whole_number: lb: 0, ub: 4294967295
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::common::decode::decode_internal] bytes_needed : 4
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::common::decode::decode_internal] decode_constrained_length_determinent, lb: 1, ub: 4
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::common::decode::decode_internal] decode_constrained_whole_number: lb: 1, ub: 4
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] Decoding Bits as Integer. offset: 43, bits: 2
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] Decoded Value: 0
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::common::decode::decode_internal] decoded length : 1
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] PerCodecData: offset: 45
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] Aligning Codec Buffer with 3 bits
E2SM_KPM_IndicationMessage
E2SM_KPM_IndicationMessageIndicationMessage_formats
E2SM_KPM_IndicationMessage_Format1
MeasurementData (measurement data list length 1)
MeasurementDataItem
MeasurementRecord (measurement record item list length 1)
MeasurementRecordItem
MeasurementRecordItem_integer (val = 0)
----------------------------------------------
Bytes: 00
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] Decoding Bits as Integer. offset: 48, bits: 8
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] Decoded Value: 0
E2SM_KPM_IndicationMessage
E2SM_KPM_IndicationMessageIndicationMessage_formats
E2SM_KPM_IndicationMessage_Format1
MeasurementData (measurement data list length 1)
MeasurementDataItem
MeasurementRecord (measurement record item list length 1)
MeasurementRecordItem
MeasurementRecordItem_integer (val = 0)
???? What is this part
Bytes: 0000
E2SM_KPM_IndicationMessage
E2SM_KPM_IndicationMessageIndicationMessage_formats
E2SM_KPM_IndicationMessage_Format1
MeasurementData (measurement data list length 1)
MeasurementDataItem
MeasurementRecord (measurement record item list length 1)
MeasurementRecordItem
MeasurementRecordItem_integer (val = 0)
???? What is this part
MeasurementInfoList (measurement info list length 1)
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] PerCodecData: offset: 56
[2023-10-25T00:50:05Z TRACE trybuild007::e2sm] decode: MeasurementInfoList
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::aper::decode] decode_length_determinent:
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::common::decode::decode_internal] decode_constrained_length_determinent, lb: 1, ub: 65535
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::common::decode::decode_internal] decode_constrained_whole_number: lb: 1, ub: 65535
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] Decoding Bits as Integer. offset: 56, bits: 16
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] Decoded Value: 0
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::common::decode::decode_internal] decoded length : 1
----------------------------------------------
Bytes: 0020
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] PerCodecData: offset: 72
[2023-10-25T00:50:05Z TRACE trybuild007::e2sm] decode: MeasurementInfoItem
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::aper::decode] decode_sequence_header: extensible: true
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] PerCodecData: offset: 73
[2023-10-25T00:50:05Z TRACE trybuild007::e2sm] decode: MeasurementType
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::aper::decode] decode_choice_idx: lb: 0, ub: 1, extensible: true
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::common::decode::decode_internal] decode_constrained_whole_number: lb: 0, ub: 1
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] Decoding Bits as Integer. offset: 74, bits: 1
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] Decoded Value: 0
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] PerCodecData: offset: 75
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] PerCodecData: offset: 75
[2023-10-25T00:50:05Z TRACE trybuild007::e2sm] decode: MeasurementTypeName
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::aper::decode::decode_charstrings] decode_printable_string: lb: Some(1), ub: Some(150), is_extensible: true
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::common::decode::decode_internal] decode_constrained_length_determinent, lb: 1, ub: 150
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::common::decode::decode_internal] decode_constrained_whole_number: lb: 1, ub: 150
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] Decoding Bits as Integer. offset: 76, bits: 8
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] Decoded Value: 2
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::common::decode::decode_internal] decoded length : 3
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] PerCodecData: offset: 84
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] Aligning Codec Buffer with 4 bits
E2SM_KPM_IndicationMessage
E2SM_KPM_IndicationMessageIndicationMessage_formats
E2SM_KPM_IndicationMessage_Format1
MeasurementData (measurement data list length 1)
MeasurementDataItem
MeasurementRecord (measurement record item list length 1)
MeasurementRecordItem
MeasurementRecordItem_integer (val = 0)
???? What is this part
MeasurementInfoList (measurement info list length 1)
MeasurementInfoItem
MeasurementType
MeasurementTypeName (length 3)
----------------------------------------------
Bytes: 435149
E2SM_KPM_IndicationMessage
E2SM_KPM_IndicationMessageIndicationMessage_formats
E2SM_KPM_IndicationMessage_Format1
MeasurementData (measurement data list length 1)
MeasurementDataItem
MeasurementRecord (measurement record item list length 1)
MeasurementRecordItem
MeasurementRecordItem_integer (val = 0)
???? What is this part
MeasurementInfoList (measurement info list length 1)
MeasurementInfoItem
MeasurementType
MeasurementTypeName (length 3, val=CQI)
----------------------------------------------
Bytes: 01
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] PerCodecData: offset: 112
[2023-10-25T00:50:05Z TRACE trybuild007::e2sm] decode: LabelInfoList
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::aper::decode] decode_length_determinent:
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] Decoding Bits as Integer. offset: 113, bits: 7
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] Decoded Value: 1
E2SM_KPM_IndicationMessage
E2SM_KPM_IndicationMessageIndicationMessage_formats
E2SM_KPM_IndicationMessage_Format1
MeasurementData (measurement data list length 1)
MeasurementDataItem
MeasurementRecord (measurement record item list length 1)
MeasurementRecordItem
MeasurementRecordItem_integer (val = 0)
???? What is this part
MeasurementInfoList (measurement info list length 1)
MeasurementInfoItem
MeasurementType
MeasurementTypeName (length 3, val=CQI)
LabelInfoList (length 1)
----------------------------------------------
Bytes: 200000
Bits: 0010 0000 0000 0000 0000 0000
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] PerCodecData: offset: 120
[2023-10-25T00:50:05Z TRACE trybuild007::e2sm] decode: LabelInfoItem
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::aper::decode] decode_sequence_header: extensible: true
[2023-10-25T00:50:05Z TRACE asn1_codecs::per] PerCodecData: offset: 121
[2023-10-25T00:50:05Z TRACE trybuild007::e2sm] decode: MeasurementLabel
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::aper::decode] decode_sequence_header: extensible: true
[2023-10-25T00:50:05Z TRACE asn1_codecs::per::common::decode] 24 optionals found
E2SM_KPM_IndicationMessage
E2SM_KPM_IndicationMessageIndicationMessage_formats
E2SM_KPM_IndicationMessage_Format1
MeasurementData (measurement data list length 1)
MeasurementDataItem
MeasurementRecord (measurement record item list length 1)
MeasurementRecordItem
MeasurementRecordItem_integer (val = 0)
???? What is this part
MeasurementInfoList (measurement info list length 1)
MeasurementInfoItem
MeasurementType
MeasurementTypeName (length 3, val=CQI)
LabelInfoList (length 1)
LabelInfoItem
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { cause: BufferTooShort, msg: "PerCodec:GetBitError:Requested Bit 24, Remaining bits 22", context: [] }', /resources/hampi-sr
c/examples/tests/18-e2sm.rs:18:55
I am fairly confident that the deserialization is working other than the reported issue, because the expected value for "MeasurementTypeName" is "CQI", and that is what is deserialized fairly late in the message.
By the way, https://github.com/srsran/srsRAN_Project/blob/374200deefd8e1b96fab7328525fd593a808a641/lib/asn1/e2ap/e2sm_kpm.cpp#L1696 is where I saw that the serialized code does not include extension fields.
@gth828r: Can you also add the generated Rust structure here to the issue? I looked at the decoder code and looks like there might be an issue with optional_count
used in decode_sequence_header_common
inside aper/decode/mod.rs
, which might in turn be caused by an incorrect optional_fields
in the generated code, so would like to look at the generated struct
.
I am fairly confident that the deserialization is working other than the reported issue, because the expected value for "MeasurementTypeName" is "CQI", and that is what is deserialized fairly late in the message.
I also don't think the problem is due to the REAL
deserialization, but this is likely to do with the generated struct
which incorrectly counts the optional extension fields.
@gth828r : I am reasonably sure this is the code generation issue, I am also counting components beyond the extension markers as part of sequence components. They are encoded differently. Assigning this issue to myself. I will most likely visit this next week.
@gth828r: Can you also add the generated Rust structure here to the issue? I looked at the decoder code and looks like there might be an issue with
optional_count
used indecode_sequence_header_common
insideaper/decode/mod.rs
, which might in turn be caused by an incorrectoptional_fields
in the generated code, so would like to look at the generatedstruct
.
Here is a version of the generated struct:
#[derive(
asn1_codecs_derive :: AperCodec, Debug, PartialEq, serde :: Serialize, serde :: Deserialize,
)]
#[asn(type = "SEQUENCE", extensible = true, optional_fields = 24)]
pub struct MeasurementLabel {
#[asn(optional_idx = 0)]
pub no_label: Option<MeasurementLabelNoLabel>,
#[asn(optional_idx = 1)]
pub plmn_id: Option<PLMNIdentity>,
#[asn(optional_idx = 2)]
pub slice_id: Option<S_NSSAI>,
#[asn(optional_idx = 3)]
pub five_qi: Option<FiveQI>,
#[asn(optional_idx = 4)]
pub qfi: Option<QosFlowIdentifier>,
#[asn(optional_idx = 5)]
pub qci: Option<QCI>,
#[asn(optional_idx = 6)]
pub qc_imax: Option<QCI>,
#[asn(optional_idx = 7)]
pub qc_imin: Option<QCI>,
#[asn(optional_idx = 8)]
pub ar_pmax: Option<MeasurementLabelARPmax>,
#[asn(optional_idx = 9)]
pub ar_pmin: Option<MeasurementLabelARPmin>,
#[asn(optional_idx = 10)]
pub bitrate_range: Option<MeasurementLabelBitrateRange>,
#[asn(optional_idx = 11)]
pub layer_mu_mimo: Option<MeasurementLabelLayerMU_MIMO>,
#[asn(optional_idx = 12)]
pub sum: Option<MeasurementLabelSUM>,
#[asn(optional_idx = 13)]
pub dist_bin_x: Option<MeasurementLabelDistBinX>,
#[asn(optional_idx = 14)]
pub dist_bin_y: Option<MeasurementLabelDistBinY>,
#[asn(optional_idx = 15)]
pub dist_bin_z: Option<MeasurementLabelDistBinZ>,
#[asn(optional_idx = 16)]
pub pre_label_override: Option<MeasurementLabelPreLabelOverride>,
#[asn(optional_idx = 17)]
pub start_end_ind: Option<MeasurementLabelStartEndInd>,
#[asn(optional_idx = 18)]
pub min: Option<MeasurementLabelMin>,
#[asn(optional_idx = 19)]
pub max: Option<MeasurementLabelMax>,
#[asn(optional_idx = 20)]
pub avg: Option<MeasurementLabelAvg>,
#[asn(optional_idx = 21)]
pub ssb_index: Option<MeasurementLabelSsbIndex>,
#[asn(optional_idx = 22)]
pub non_go_b_b_fmode_index: Option<MeasurementLabelNonGoB_BFmode_Index>,
#[asn(optional_idx = 23)]
pub mimo_mode_index: Option<MeasurementLabelMIMO_mode_Index>,
}
I appreciate you taking the time to look into it!
@gth828r : Thanks for this update
#[derive( asn1_codecs_derive :: AperCodec, Debug, PartialEq, serde :: Serialize, serde :: Deserialize, )] #[asn(type = "SEQUENCE", extensible = true, optional_fields = 24)]
This optional_fields
above is the issue, this should have been 21. I will be looking at fixing this in the next week.
@gth828r : I am looking into this and looks like this is a bit more involved than I additionally thought. Needs some changes in the compiler to get it right. This might take a bit more time.
@gth828r : Following is hex message and decoded message not sure if that's fine. Can you please check it? (Most values seem to be None
!)
message: 080000000100000000002043514901200000
E2SM
e2sm_kpmv3_pdu: E2SM_KPM_IndicationMessage {
indication_message_formats: IndicationMessage_Format1(
E2SM_KPM_IndicationMessage_Format1 {
meas_data: MeasurementData(
[
MeasurementDataItem {
meas_record: MeasurementRecord(
[
Integer(
MeasurementRecordItem_integer(
0,
),
),
],
),
incomplete_flag: None,
},
],
),
meas_info_list: Some(
MeasurementInfoList(
[
MeasurementInfoItem {
meas_type: MeasName(
MeasurementTypeName(
"CQI",
),
),
label_info_list: LabelInfoList(
[
LabelInfoItem {
meas_label: MeasurementLabel {
no_label: Some(
MeasurementLabelNoLabel(
0,
),
),
plmn_id: None,
slice_id: None,
five_qi: None,
qfi: None,
qci: None,
qc_imax: None,
qc_imin: None,
ar_pmax: None,
ar_pmin: None,
bitrate_range: None,
layer_mu_mimo: None,
sum: None,
dist_bin_x: None,
dist_bin_y: None,
dist_bin_z: None,
pre_label_override: None,
start_end_ind: None,
min: None,
max: None,
avg: None,
},
},
],
),
},
],
),
),
granul_period: None,
},
),
}
@gth828r : Following is hex message and decoded message not sure if that's fine. Can you please check it? (Most values seem to be
None
!)
At a glance, this looks right to me! Based on source diving, I believe that for the MeasurementLabel
struct, only the no_label
field should be set and everything else in there should be None
.
@gth828r : Yes. I have pushed a PR (#106) for this. This is a good enough for now solution. Properly fixing this would require a lot more work (and note the codec support for fields beyond extension markers is still not there).
I'll give it a try and make sure it works on my end. I added an unimportant comment to the test code in the PR. I don't know enough about the compiler code and other bits to add a timely meaningful review to those.
I mentioned this on the PR too, but I was able to successfully run the test for this on my system.
The PR is merged. I will be closing this and other issues after I make 0.6.0
release.
Thanks for your fast help!
This is available now in crate version 0.6.0
.
When deserializing at a sequence prelude, the code currently seems to assume that extension fields will be present when looking at optional flags. If the serialization code doesn't include the extension fields, then this will result in an error.
As an example, consider the following sequence: https://github.com/gabhijit/hampi/blob/master/examples/specs/e2sm/E2SM-KPM.asn#L55
There are 24 optional fields, but 3 of them are extension fields. The deserializer will fail if not all of the fields are present, with the following message:
Here is an example serialized
E2SM-KPM-IndicationMessage
that does not include the extension fields:You can add fake serialized extension fields by appending another
00
to the end of the message, and then deserialization will work fine.