search

bin123apple / Fortran2Cpp

Fortran2Cpp: A new model designed for the Code translation between the Fortran and C++
Apache License 2.0
4 stars 1 forks source link

Questions about steps in Section 3 in the paper #61

Open LChenGit opened 2 weeks ago

LChenGit commented 2 weeks ago

Hi @bin123apple @chunhualiao @peihunglin

here are some of my questions about the steps in Section 3 of the paper.

  1. How does the questioner and answer system stand out from the traditional inference system? Are there any papers showing using LLMs as questioner are more effective? If not, how should we describe this system? Are we going to list it as one of the contributions?
  2. Prompt polish. 2.1 Questioner prompt 2.1. This prompt asks to add unit tests for the original Fortran and translated C++ codes. However, it does not explicitly ask for the same unit tests. Even the examples in the prompts are different unit test cases. How are we handling the functional consistence in the translated code if we use different test cases? For example, the translated code may functionally differ from the Fortran code. But it also passes all the tests as it is compilable and executable, and the unit tests differ from those in the Fortran code.
  3. Step 3 also raises my concern about functional consistency between the original and translated codes. The step also modifies the code after adding the unit tests in step 2. Not sure if it is the best option.
  4. In step 3, if the compilation error is about the header file, why LLM cannot handle it with the compilation error message? Why do we need to list it aside from the general compilation errors?
  5. Step 4 checks the executions of the original Fortran code and the translated C++ code. As the codes are running unit tests at this step, why do we need an LLM to check it?
  6. Step 6 is designed to keep format consistency, but it simply says, " Give the correct and complete Fortran-c++ code pair based on your last modification." Is this enough for LLMs to do the job?
chunhualiao commented 2 weeks ago

log files: https://github.com/bin123apple/Fortran2Cpp/tree/main/dataset_generation/example_results

chunhualiao commented 2 weeks ago

The current workflow does has many issues. We should plan on a better version 2. Can you redesign and draft a more intuitive one based on info. you have and my code review comments? The new dataset v2 should be straightforward and paired Fortran vs. C++ versions. I think Bin need our help for the new design.

On Wed, Aug 28, 2024 at 4:01 PM Le Chen @.***> wrote:

  1. How does the questioner and answer system stand out from the traditional inference system? Are there any papers showing using LLMs as questioner are more effective? If not, how should we describe this system? Are we going to list it as one of the contributions?
  2. Prompt polish. 2.1 Questioner prompt 2.1. This prompt asks to add unit tests for the original Fortran and translated C++ codes. However, it does not explicitly ask for the same unit tests. Even the examples in the prompts are different unit test cases. How are we handling the functional consistence in the translated code if we use different test cases? For example, the translated code may functionally differ from the Fortran code. But it also passes all the tests as it is compilable and executable, and the unit tests differ from those in the Fortran code.
  3. Step 3 also raises my concern about functional consistency between the original and translated codes. The step also modifies the code after adding the unit tests in step 2. Not sure if it is the best option.
  4. In step 3, if the compilation error is about the header file, why LLM cannot handle it with the compilation error message? Why do we need to list it aside from the general compilation errors?
  5. Step 4 checks the executions of the original Fortran code and the translated C++ code. As the codes are running unit tests at this step, why do we need an LLM to check it?
  6. Step 6 is designed to keep format consistency, but it simply says, " Give the correct and complete Fortran-c++ code pair based on your last modification." Is this enough for LLMs to do the job?

— Reply to this email directly, view it on GitHub https://github.com/bin123apple/Fortran2Cpp/issues/61, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMNIRSCEWO5T6U742DCOE3ZTZJFBAVCNFSM6AAAAABNJHJGDCVHI2DSMVQWIX3LMV43ASLTON2WKOZSGQ4TGMJRHEZDAMA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

LChenGit commented 2 weeks ago

I thought Step 4 was checking execution errors, but from the log, it checks the execution results.