v1 dataset: for code pairs extracted, compile and run all, collect statistics

[chunhualiao]

2.5k pairs : compile and run all

for conversational dialogue datasets: only last code pairs matters.

• select 10% sample or easy to extract data pairs : save results and compile/run

[bin123apple]

remove .mod and .o files,

[chunhualiao]

many Cpp files cannot be compiled, even though the pipeline is designed to make sure they can be compiled and run.

• [x] looking to Cpp files that failed compilation : leo https://github.com/bin123apple/Fortran2Cpp/issues/64
• [ ] Bin: debugging the pipeline using a few examples, why it let through some wrong codes? using log file by Leo too added into the dataset generation engine script.

[bin123apple]

It is highly possible that the bug is caused by this line: https://github.com/bin123apple/Fortran2Cpp/blob/main/dataset_generation/extract_final_data_pairs.py#L14. As the correct cpp/fortran may not always appear in the end of the dialogue.

Cause: Sometimes, in the pipeline, while trying to compare the incompilable cpp code with the compilable fortran code, the cpp code will be incomplete again.

[bin123apple]

1. Record the output code that cause the problem.

2. Traceback: recalled evidence in each step.

   • help us understand why you have wrong ends in the end of dialogue?
   • And why you need to extract code for dialogue ending with wrong codes?

In your pipeline, which step do you have correct cpp and fortran codes both can pass compilation and execution ?

• please locate this step, the save the cpp and fortran code in to somewhere .
• avoid using the end of diaglouge

[bin123apple]

See https://github.com/bin123apple/Fortran2Cpp/blob/main/data/example.json