Inconsistent Pairs of Segments

[evangelos-bitsikas]

In Section 6.3.2, Figures 6 to 11 highlight potential inconsistent Pairs of Segments (PoS).

What is the possible way to evaluate if cellularlint can identify the alleged inconsistent segments?

Input:

T1: From this time onward the UE shall cipher and integrity protect all NAS signalling messages with the selected NAS ciphering and NAS integrity algorithms.
T2: From this time onward, all NAS messages exchanged between the UE and the MME are sent integrity protected and except for the messages specified in clause 4.4.5, all NAS messages exchanged between the UE and the MME are sent ciphered.

Output: Yes/No

[Masfiqur-Mim]

Hi @evangelos-bitsikas,

Thank you for raising the discussion. To fully understand how CellularLint can detect such PoS, the user need to make sure the models are properly trained under the informed fine-tuning approach (Method detailed in Step 2 under Running the Experiments section in README). Very soon we will also provide the complete raw PoS data (created and filtered from the data/cp_corpus_4G.txt and data/cp_corpus_5G.txt) with this repository to enable users the complete informed fine-tuning on full dataset we used. This is the last piece of the puzzle we couldn't open-source to make sure the conference artifact evaluation process remains fully automated.

Another quick evaluation could be to check if the text pairs are present in either of the cp_corpus_xG.txt as individual segments.

Thanks.

[evangelos-bitsikas]

@Masfiqur-Mim Hello, just to clarify: if I have two segments or paragraphs from the specification, is there any way to check if cellularlint can determine whether the articles are contradictory or not?

I understand that Step 2 under the "Running the Experiments" section in the README requires you to perform informed fine-tuning. But suppose I am a developer and I want to ensure there are no inconsistencies. There should be some predefined model after the fine-tuning so that I can verify contradictions directly.

[Masfiqur-Mim]

Hi @evangelos-bitsikas, thanks for your suggestion. There is a plan to release a complete fine-tuned version of the models for such purpose. We will let you know.