Closed evangelos-bitsikas closed 2 months ago
Hi @evangelos-bitsikas,
Thank you for raising the discussion. To fully understand how CellularLint can detect such PoS, the user need to make sure the models are properly trained under the informed fine-tuning approach (Method detailed in Step 2 under Running the Experiments
section in README). Very soon we will also provide the complete raw PoS data (created and filtered from the data/cp_corpus_4G.txt
and data/cp_corpus_5G.txt
) with this repository to enable users the complete informed fine-tuning on full dataset we used. This is the last piece of the puzzle we couldn't open-source to make sure the conference artifact evaluation process remains fully automated.
Another quick evaluation could be to check if the text pairs are present in either of the cp_corpus_xG.txt
as individual segments.
Thanks.
@Masfiqur-Mim Hello, just to clarify: if I have two segments or paragraphs from the specification, is there any way to check if cellularlint can determine whether the articles are contradictory or not?
I understand that Step 2 under the "Running the Experiments" section in the README requires you to perform informed fine-tuning. But suppose I am a developer and I want to ensure there are no inconsistencies. There should be some predefined model after the fine-tuning so that I can verify contradictions directly.
Hi @evangelos-bitsikas, thanks for your suggestion. There is a plan to release a complete fine-tuned version of the models for such purpose. We will let you know.
In Section 6.3.2, Figures 6 to 11 highlight potential inconsistent Pairs of Segments (PoS).
What is the possible way to evaluate if cellularlint can identify the alleged inconsistent segments?
Input:
Output: Yes/No