wimvanderbauwhede / RefactorF4Acc

An Automated Fortran Code Refactoring Tool to Facilitate Acceleration of Numerical Simulations
Other
58 stars 9 forks source link

Results of a "Hello, world" attempt #8

Closed rouson closed 6 years ago

rouson commented 6 years ago

I am very interested in using RefactorF4Acc on the zgoubi project, a ~300,000-line beam physics code developed at Brookhaven National Laboratory in the U.S. I haven't had success in running refactorF4Acc.pl so I wrote a 26-line "Hello, world!" program (hello.tar.gz) that exhibits a few of the important features of zgoubi, including implicit typing, a common block, fixed-form source, and subroutine dummy arguments with unspecified intent. I'm attaching the "Hello, world!" program and providing the transcript below that shows how to build and run the code as well as a depiction of the directory structure and the refactorF4Acc.pl error message.

It would be great to include a working version of this "Hello, world!" or something similar in the RefactorF4Acc tests subdirectory as a way for users to get started.

$ tar xf hello.tar.gz 
$ cd hello
$ make
gfortran -o hello src/main.f src/greet_world.f -I src/include
$ ./hello 
 Hello, world!
 All is right with the world.
$ tree
.
├── Makefile
├── rf4a.cfg
└── src
    ├── greet_world.f
    ├── include
    │   └── error_codes.h
    └── main.f

2 directories, 5 files
$ export PERL5LIB="$PERL5LIB:${HOME}/Builds/RefactorF4Acc"
$ export PATH="$PATH:${HOME}/Builds/RefactorF4Acc/bin"
$ cat rf4a.cfg 
TOP = hello
PREFIX = .
SRCDIRS = src
NEWSRCPATH = refactored-source
$ refactorF4acc.pl -c rf4a.cfg 
Expected ClosingParen, got EOF at /home/rouson/Builds/RefactorF4Acc/Fortran/Expression/Evaluator/Parser.pm line 201.
    Fortran::Expression::Evaluator::Parser::_match(Fortran::Expression::Evaluator::Parser=HASH(0x558dede18408), "ClosingParen") called at /home/rouson/Builds/RefactorF4Acc/Fortran/Expression/Evaluator/Parser.pm line 244
    Fortran::Expression::Evaluator::Parser::_function_call(Fortran::Expression::Evaluator::Parser=HASH(0x558dede18408)) called at /home/rouson/Builds/RefactorF4Acc/Fortran/Expression/Evaluator/Parser.pm line 217
    Fortran::Expression::Evaluator::Parser::_value(Fortran::Expression::Evaluator::Parser=HASH(0x558dede18408)) called at /home/rouson/Builds/RefactorF4Acc/Fortran/Expression/Evaluator/Parser.pm line 360
    Fortran::Expression::Evaluator::Parser::_factor(Fortran::Expression::Evaluator::Parser=HASH(0x558dede18408)) called at /home/rouson/Builds/RefactorF4Acc/Fortran/Expression/Evaluator/Parser.pm line 369
    Fortran::Expression::Evaluator::Parser::_exponential(Fortran::Expression::Evaluator::Parser=HASH(0x558dede18408)) called at /home/rouson/Builds/RefactorF4Acc/Fortran/Expression/Evaluator/Parser.pm line 302
    Fortran::Expression::Evaluator::Parser::_term(Fortran::Expression::Evaluator::Parser=HASH(0x558dede18408)) called at /home/rouson/Builds/RefactorF4Acc/Fortran/Expression/Evaluator/Parser.pm line 336
    Fortran::Expression::Evaluator::Parser::_expression(Fortran::Expression::Evaluator::Parser=HASH(0x558dede18408)) called at /home/rouson/Builds/RefactorF4Acc/Fortran/Expression/Evaluator/Parser.pm line 260
    Fortran::Expression::Evaluator::Parser::_statement(Fortran::Expression::Evaluator::Parser=HASH(0x558dede18408)) called at /home/rouson/Builds/RefactorF4Acc/Fortran/Expression/Evaluator/Parser.pm line 175
    Fortran::Expression::Evaluator::Parser::_program(Fortran::Expression::Evaluator::Parser=HASH(0x558dede18408)) called at /home/rouson/Builds/RefactorF4Acc/Fortran/Expression/Evaluator/Parser.pm line 128
    Fortran::Expression::Evaluator::Parser::parse("character(len", HASH(0x558dede18750)) called at /home/rouson/Builds/RefactorF4Acc/RefactorF4Acc/Parser/Expressions.pm line 171
    RefactorF4Acc::Parser::Expressions::parse_expression("character(len", HASH(0x558decbb0690), HASH(0x558decbade30), "hello") called at /home/rouson/Builds/RefactorF4Acc/RefactorF4Acc/Parser.pm line 4245
    RefactorF4Acc::Parser::_parse_assignment("character(len=5)greeting", HASH(0x558decbb0690), HASH(0x558decbade30), "hello") called at /home/rouson/Builds/RefactorF4Acc/RefactorF4Acc/Parser.pm line 1257
    RefactorF4Acc::Parser::_analyse_lines("hello", HASH(0x558decbade30)) called at /home/rouson/Builds/RefactorF4Acc/RefactorF4Acc/Parser.pm line 99
    RefactorF4Acc::Parser::parse_fortran_src("hello", HASH(0x558decbade30)) called at /home/rouson/Builds/RefactorF4Acc/bin/refactorF4acc.pl line 168
    main::main() called at /home/rouson/Builds/RefactorF4Acc/bin/refactorF4acc.pl line 67
rouson commented 6 years ago

Also, I just tried simplifying this example down to two lines of Fortran source code and found that refactorF4acc.pl fails without the -g flag. Is this flag required for all codes?

$ ls
main.f  rf4a.cfg
$ cat rf4a.cfg 
TOP = hello
PREFIX = .
SRCDIRS = .
NEWSRCPATH = refactored-source
$ cat main.f 
      PROGRAM hello
      END PROGRAM
$ refactorF4acc.pl -c rf4a.cfg 
SUB hello DOES NOT HAVE RefactoredArgs
BOOM! 271 hello : $VAR1 = {
  'FreeForm' => 0,
  'Entries' => {
    'Set' => {},
    'List' => []
  },
  'UndeclaredCommonVars' => {
    'Set' => {},
    'List' => []
  },
  'DoneInitTables' => 1,
  'UsedGlobalVars' => {
    'List' => [],
    'Set' => {}
  },
  'ExGlobArgs' => {
    'Set' => {},
    'List' => []
  },
  'DeclaredOrigArgs' => {
    'Set' => {},
    'List' => []
  },
  'Entry' => 0,
  'HasRefactoredArgs' => 0,
  'RefactorGlobals' => 0,
  'DeclaredOrigLocalVars' => {
    'Set' => {},
    'List' => []
  },
  'LocalParameters' => {
    'List' => [],
    'Set' => {}
  },
  'Callers' => {},
  'UndeclaredOrigArgs' => {
    'Set' => {},
    'List' => []
  },
  'UndeclaredOrigLocalVars' => {
    'Set' => {},
    'List' => []
  },
  'Recursive' => 0,
  'RefactoredCode' => [
    [
      '      program hello',
      {
        'Indent' => '      ',
        'Ref' => 0,
        'Signature' => {
          'Name' => 'hello',
          'Program' => 1,
          'Args' => {
            'List' => []
          }
        },
        'Block' => {
          'Nest' => 1,
          'InBlock' => {},
          'Type' => 'program',
          'LineID' => 0,
          'Name' => 'hello'
        },
        'LineID' => 0,
        'AccPragma' => {},
        'SubroutineSig' => [
          '      ',
          'hello',
          []
        ],
        'ExGlobVarDeclHook' => 'Signature'
      }
    ],
    [
      '      end program',
      {
        'ExtraIncludesHook' => 1,
        'ExtraModulesHook' => 1,
        'LineID' => 1,
        'Block' => $VAR1->{'RefactoredCode'}[0][1]{'Block'},
        'Indent' => '      ',
        'Ref' => 0,
        'AccPragma' => {}
      }
    ]
  ],
  'UsedParameters' => {
    'Set' => {},
    'List' => []
  },
  'TabFormat' => 0,
  'Vars' => {
    'Subsets' => {
      'CommonVars' => {
        'Subsets' => {
          'UndeclaredCommonVars' => $VAR1->{'UndeclaredCommonVars'},
          'DeclaredCommonVars' => {
            'List' => [],
            'Set' => {}
          }
        }
      },
      'Args' => {
        'Subsets' => {
          'ExGlobArgs' => $VAR1->{'ExGlobArgs'},
          'ExInclArgs' => {
            'Set' => {},
            'List' => []
          },
          'OrigArgs' => {
            'List' => [],
            'Subsets' => {
              'UndeclaredOrigArgs' => $VAR1->{'UndeclaredOrigArgs'},
              'DeclaredOrigArgs' => $VAR1->{'DeclaredOrigArgs'}
            }
          }
        }
      },
      'Parameters' => {
        'Subsets' => {
          'UsedParameters' => $VAR1->{'UsedParameters'},
          'LocalParameters' => $VAR1->{'LocalParameters'},
          'IncludedParameters' => {
            'Set' => {},
            'List' => []
          },
          'ParametersFromContainer' => {
            'List' => [],
            'Set' => {}
          }
        }
      },
      'LocalVars' => {
        'Subsets' => {
          'OrigLocalVars' => {
            'Subsets' => {
              'DeclaredOrigLocalVars' => $VAR1->{'DeclaredOrigLocalVars'},
              'UndeclaredOrigLocalVars' => $VAR1->{'UndeclaredOrigLocalVars'}
            }
          },
          'UsedLocalVars' => {
            'Set' => {},
            'List' => []
          },
          'ExInclLocalVars' => {
            'List' => [],
            'Set' => {}
          }
        }
      }
    }
  },
  'OrigLocalVars' => $VAR1->{'Vars'}{'Subsets'}{'LocalVars'}{'Subsets'}{'OrigLocalVars'},
  'RefactoredArgs' => {
    'List' => [],
    'Set' => {}
  },
  'ExGlobVarDeclHook' => 1,
  'MaskedIntrinsics' => {},
  'ConflictingParams' => {},
  'Source' => './main.f',
  'FStyle' => 'F77',
  'Program' => 1,
  'Name' => 'hello',
  'HasType' => '',
  'UsedLocalVars' => $VAR1->{'Vars'}{'Subsets'}{'LocalVars'}{'Subsets'}{'UsedLocalVars'},
  'AnnLines' => $VAR1->{'RefactoredCode'},
  'LocalVars' => $VAR1->{'Vars'}{'Subsets'}{'LocalVars'},
  'ExInclLocalVars' => $VAR1->{'Vars'}{'Subsets'}{'LocalVars'}{'Subsets'}{'ExInclLocalVars'},
  'ParametersFromContainer' => $VAR1->{'Vars'}{'Subsets'}{'Parameters'}{'Subsets'}{'ParametersFromContainer'},
  'IncludedParameters' => $VAR1->{'Vars'}{'Subsets'}{'Parameters'}{'Subsets'}{'IncludedParameters'},
  'OrigArgs' => $VAR1->{'Vars'}{'Subsets'}{'Args'}{'Subsets'}{'OrigArgs'},
  'ExInclArgs' => $VAR1->{'Vars'}{'Subsets'}{'Args'}{'Subsets'}{'ExInclArgs'},
  'ReferencedLabels' => {},
  'Parameters' => $VAR1->{'Vars'}{'Subsets'}{'Parameters'},
  'Elemental' => 0,
  'Args' => $VAR1->{'Vars'}{'Subsets'}{'Args'},
  'RenamedInheritedExGLobs' => {
    'Set' => {}
  },
  'HasBlocks' => 0,
  'Status' => 3,
  'InheritedParameters' => {
    'List' => [],
    'Set' => {}
  },
  'Commons' => {},
  'Includes' => {},
  'Pure' => 0,
  'CalledEntries' => {
    'Set' => {},
    'List' => []
  },
  'CalledSubs' => {
    'Set' => {},
    'List' => []
  },
  'HasCommons' => 0,
  'CommonVars' => $VAR1->{'Vars'}{'Subsets'}{'CommonVars'},
  'RenamedInheritedExGlobs' => {
    'List' => [],
    'Set' => {}
  },
  'DeclaredCommonVars' => $VAR1->{'Vars'}{'Subsets'}{'CommonVars'}{'Subsets'}{'DeclaredCommonVars'}
};
 at /home/rouson/Desktop/Builds/RefactorF4Acc/RefactorF4Acc/Analysis/ArgumentIODirs.pm line 271.
    RefactorF4Acc::Analysis::ArgumentIODirs::_analyse_src_for_iodirs(HASH(0x555b98b3eef0), "hello") called at /home/rouson/Desktop/Builds/RefactorF4Acc/RefactorF4Acc/Analysis/ArgumentIODirs.pm line 77
    RefactorF4Acc::Analysis::ArgumentIODirs::_determine_argument_io_direction_core(HASH(0x555b98b3eef0), "hello") called at /home/rouson/Desktop/Builds/RefactorF4Acc/RefactorF4Acc/Analysis/ArgumentIODirs.pm line 60
    RefactorF4Acc::Analysis::ArgumentIODirs::determine_argument_io_direction_rec(HASH(0x555b98b3eef0), "hello") called at /home/rouson/Desktop/Builds/RefactorF4Acc/RefactorF4Acc/Refactoring.pm line 83
    RefactorF4Acc::Refactoring::refactor_all(HASH(0x555b98b3eef0), "hello", "") called at /home/rouson/Desktop/Builds/RefactorF4Acc/bin/refactorF4acc.pl line 192
    main::main() called at /home/rouson/Desktop/Builds/RefactorF4Acc/bin/refactorF4acc.pl line 67
rouson commented 6 years ago

In case it's helpful, I also reduced the first example above to one that exhibits what appears to be the same failure even with the -g flag and after cutting out ~20 lines of code:

$ cat rf4a.cfg 
TOP = hello
PREFIX = .
SRCDIRS = .
NEWSRCPATH = refactored-source
$ cat main.f
      subroutine greet_world(string)
        character(len=*) string
        write(*,*) string
      end subroutine

      program hello
        implicit none
        call greet_world("Hello, world!")
      end program
$ refactorF4acc.pl -c rf4a.cfg -g
Expected ClosingParen, got EOF at /home/rouson/Desktop/Builds/RefactorF4Acc/Fortran/Expression/Evaluator/Parser.pm line 201.
    Fortran::Expression::Evaluator::Parser::_match(Fortran::Expression::Evaluator::Parser=HASH(0x556d8fb4e6f0), "ClosingParen") called at /home/rouson/Desktop/Builds/RefactorF4Acc/Fortran/Expression/Evaluator/Parser.pm line 244
    Fortran::Expression::Evaluator::Parser::_function_call(Fortran::Expression::Evaluator::Parser=HASH(0x556d8fb4e6f0)) called at /home/rouson/Desktop/Builds/RefactorF4Acc/Fortran/Expression/Evaluator/Parser.pm line 217
    Fortran::Expression::Evaluator::Parser::_value(Fortran::Expression::Evaluator::Parser=HASH(0x556d8fb4e6f0)) called at /home/rouson/Desktop/Builds/RefactorF4Acc/Fortran/Expression/Evaluator/Parser.pm line 360
    Fortran::Expression::Evaluator::Parser::_factor(Fortran::Expression::Evaluator::Parser=HASH(0x556d8fb4e6f0)) called at /home/rouson/Desktop/Builds/RefactorF4Acc/Fortran/Expression/Evaluator/Parser.pm line 369
    Fortran::Expression::Evaluator::Parser::_exponential(Fortran::Expression::Evaluator::Parser=HASH(0x556d8fb4e6f0)) called at /home/rouson/Desktop/Builds/RefactorF4Acc/Fortran/Expression/Evaluator/Parser.pm line 302
    Fortran::Expression::Evaluator::Parser::_term(Fortran::Expression::Evaluator::Parser=HASH(0x556d8fb4e6f0)) called at /home/rouson/Desktop/Builds/RefactorF4Acc/Fortran/Expression/Evaluator/Parser.pm line 336
    Fortran::Expression::Evaluator::Parser::_expression(Fortran::Expression::Evaluator::Parser=HASH(0x556d8fb4e6f0)) called at /home/rouson/Desktop/Builds/RefactorF4Acc/Fortran/Expression/Evaluator/Parser.pm line 260
    Fortran::Expression::Evaluator::Parser::_statement(Fortran::Expression::Evaluator::Parser=HASH(0x556d8fb4e6f0)) called at /home/rouson/Desktop/Builds/RefactorF4Acc/Fortran/Expression/Evaluator/Parser.pm line 175
    Fortran::Expression::Evaluator::Parser::_program(Fortran::Expression::Evaluator::Parser=HASH(0x556d8fb4e6f0)) called at /home/rouson/Desktop/Builds/RefactorF4Acc/Fortran/Expression/Evaluator/Parser.pm line 128
    Fortran::Expression::Evaluator::Parser::parse("character(len", HASH(0x556d8fb4e6c0)) called at /home/rouson/Desktop/Builds/RefactorF4Acc/RefactorF4Acc/Parser/Expressions.pm line 171
    RefactorF4Acc::Parser::Expressions::parse_expression("character(len", HASH(0x556d8fb3c920), HASH(0x556d8e8cbe30), "greet_world") called at /home/rouson/Desktop/Builds/RefactorF4Acc/RefactorF4Acc/Parser.pm line 4245
    RefactorF4Acc::Parser::_parse_assignment("character(len=*)string", HASH(0x556d8fb3c920), HASH(0x556d8e8cbe30), "greet_world") called at /home/rouson/Desktop/Builds/RefactorF4Acc/RefactorF4Acc/Parser.pm line 1257
    RefactorF4Acc::Parser::_analyse_lines("greet_world", HASH(0x556d8e8cbe30)) called at /home/rouson/Desktop/Builds/RefactorF4Acc/RefactorF4Acc/Parser.pm line 99
    RefactorF4Acc::Parser::parse_fortran_src("greet_world", HASH(0x556d8e8cbe30)) called at /home/rouson/Desktop/Builds/RefactorF4Acc/RefactorF4Acc/Parser.pm line 1902
    RefactorF4Acc::Parser::_parse_subroutine_and_function_calls("hello", HASH(0x556d8e8cbe30)) called at /home/rouson/Desktop/Builds/RefactorF4Acc/RefactorF4Acc/Parser.pm line 118
    RefactorF4Acc::Parser::parse_fortran_src("hello", HASH(0x556d8e8cbe30)) called at /home/rouson/Desktop/Builds/RefactorF4Acc/bin/refactorF4acc.pl line 168
    main::main() called at /home/rouson/Desktop/Builds/RefactorF4Acc/bin/refactorF4acc.pl line 67
wimvanderbauwhede commented 6 years ago

Hi Damian,

Thank you for your efforts! I had fixed the parse error you got in my devel branch but I have not yet merged that one into master. As soon as I run the regression tests and merged, I will add the Hello, World examples you've provided.

The -g flag is currently necessary so I changed it into a no-op.

It would be nice if my compiler could handle your 300K codebase. It depends more on the code features than on the size of the code. At the moment I don't really have time to test it, and that's a pity as I am quite keen to do this.

rouson commented 6 years ago

ping

If there's any chance you can push the parser fix shortly, I'd like to present the "Hello, world!" example in a teleconference scheduled for this Thursday morning, U.S. Pacific time.

wimvanderbauwhede commented 6 years ago

I have just merged the devel branch so the "Hello, World!" example is now working. I've also added a generate shell script and a Makefile for the generated code.

wimvanderbauwhede commented 6 years ago

I also verified the NIST tests and the Shallow Water, they all work so I'm closing this issue.

rouson commented 6 years ago

Awesome! RefactorF4Acc rocks! I love what it does to the "Hello, world!" example. I'll begin investigating whether I can use RefactorF4Acc on zgoubi. I will submit a pull request with a GETTING_STARTED.md that highlights describes the "Hello, world!" example.