Fortran runtime error

[jjunum]

I installed atchem2 and there was no problem for a while. Suddenly, I believe after some updates in Ubuntu 18.04 LST, I got Fortran runtime errors as follows:

jch@Aerosol:~/Research/AtChem2$ ./atchem2 AtChem2 v1.2-dev

---

Directories

Model directory is: model Output directory is: model/output Reaction Rates directory is: model/output/reactionRates Configuration directory is: model/configuration Constraints directory is: model/constraints Environment Constraints directory is: model/constraints/environment Photolysis Constraints directory is: model/constraints/photolysis Species Constraints directory is: model/constraints/species MCM directory is: mcm Shared library is: model/configuration/mechanism.so

---

Species and reactions

Number of Species = 63 Number of Reactions = 146

Size of lhs = 233 Size of rhs = 204

Reading reactants (lhs) from mechanism.reac... Reading products (rhs) from mechanism.prod... Finished reading lhs and rhs data.

Reading species names from mechanism.species... Finished reading species names.

Reading initial concentrations... 1 SA 0.000E+00 ... 15 O2M 0.000E+00 Finished reading initial concentrations.

---

Species requiring detailed rate output

Reading which species require detailed rate output... 1 SO2AQ ... 9 SULF_NO2 Finished reading which species require detailed rate output. Species requiring detailed rate output (number of species found): 9

Reading ro2 numbers from mechanism.ro2... Finished reading ro2 numbers. Reading solver parameters from file...

Solver parameters:

        atol:   1.000E-03
        rtol:   1.000E-04
   deltaMain:   1.000E-04
    lookBack:         100
     maxStep:   1.000E+03

preconBandUpper: 750 preconBandLower: 750 solverType: SPGMR + Banded Preconditioner

Finished reading solver parameters from file.

Reading model parameters from file...

Model parameters:

                               number of steps:         929
                           step size (seconds):   0.360E+04
                  species interpolation method: piecewise linear
               conditions interpolation method: piecewise linear
                           ratesOutputStepSize:       43200
               reaction rates output step size:       43200
                                modelStartTime:        3600
                        jacobianOutputStepSize:           0
                                      latitude:   0.126E+03
                                     longitude:   0.373E+02
                                day/month/year:  10/ 5/2016

---

Finished reading model parameters from file.

---

Species of Interest

Reading concentration output from file... Finished reading concentration output from file. Output required for concentration of 24 species: 1 OH ... 24 SULF_NO2

---

Photolysis

Reading photolysis numbers from file... 1 ... 61 Finished reading photolysis numbers. Number of photolysis numbers: 35 Looking for photolysis constants file... Checking that photolysis constants exist in file... Photolysis constants file is empty. No photolysis constants applied, so trying constrained photolysis rates file... Looking for photolysis constraints file... Checking that photolysis constraints exist in file... Photolysis constraint file is empty, so all photolysis rates will be calculated. Reading all photolysis rates from file... 1 6.073E-05 1.743E+00 4.740E-01 J1 1.000E+00 ... 61 7.537E-04 4.990E-01 2.660E-01 J61 1.000E+00 Finished reading all photolysis rates. Number of all photolysis rates: 35

---

Environment variables

Reading environment variables... Number of environment variables: 10 1 TEMP 298.15 2 PRESS 1013.25 3 RH CONSTRAINED 4 H2O CALC 5 DEC CALC 6 BLHEIGHT CONSTRAINED 7 DILUTE NOTUSED 8 JFAC NOTUSED 9 ROOF OPEN Finished reading environment variables.

Checking for constrained environment variables... maximum number of environment variable constraint data points: 937 Reading constraint data for RH Finished reading constraint data. Reading constraint data for BLHEIGHT Finished reading constraint data. Finished checking for constrained environment variables.

---

Constraints

Counting the variable-concentration species to be constrained (in file speciesConstrained.config)... Finished counting the names of variable-concentration constrained species. Number of names of variable-concentration constrained species: 7 Counting the fixed-concentration species to be constrained (in file speciesConstant.config)... Finished counting the names of fixed-concentration constrained species. Number of names of fixed-concentration constrained species: 4 Setting size of constraint arrays, n = 11 Reading in the names of variable-concentration constrained species... Finished reading the names of variable-concentration constrained species 1 SO2 ... 7 O3 maximum number of species constraint data points: 928 Reading concentration data for variable-concentration constrained species... SO2 ... NO2 ... O3 ... Reading in the names and concentration of the fixed constrained species (in file speciesConstant.config)... 1 CH4 4.900E+14 ... 4 O2 5.000E+10 Finished reading in the names and concentration of fixed-concentration species. Finished reading constrained species. Initialising concentrations of constrained species... Finished initialising concentrations of constrained species.

---

Problem stats

                    neq = 52

numberOfConstrainedSpecies = 11 t0 = 3.600E+03

setting maxnumsteps ier = 0 setting maxstep ier = 0

---

Model run

At line 16 of file model/configuration//mechanism.f90 Fortran runtime error: Index '1' of dimension 1 of array 'q' above upper bound of -94734093647877 jch@Aerosol:~/Research/AtChem2$

I couldn't figure it out, so tried to reinstall, but unfortunately got a similar Fortran runtime error during the installation.

jch@Aerosol:~/Research/AtChem2-master$ ./atchem2 AtChem2 v1.2-dev

---

Directories

Model directory is: model Output directory is: model/output Reaction Rates directory is: model/output/reactionRates Configuration directory is: model/configuration Constraints directory is: model/constraints Environment Constraints directory is: model/constraints/environment Photolysis Constraints directory is: model/constraints/photolysis Species Constraints directory is: model/constraints/species MCM directory is: mcm Shared library is: model/configuration/mechanism.so

---

Species and reactions

Number of Species = 29 Number of Reactions = 71

Size of lhs = 114 Size of rhs = 100

Reading reactants (lhs) from mechanism.reac... Reading products (rhs) from mechanism.prod... Finished reading lhs and rhs data.

Reading species names from mechanism.species... Finished reading species names.

Reading initial concentrations... 1 CH4 4.900E+13 ... 4 NO2 2.400E+11 Finished reading initial concentrations.

---

Species requiring detailed rate output

Reading which species require detailed rate output... 1 OH 2 HO2 Finished reading which species require detailed rate output. Species requiring detailed rate output (number of species found): 2

Reading ro2 numbers from mechanism.ro2... Finished reading ro2 numbers. Reading solver parameters from file...

Solver parameters:

        atol:   1.000E-03
        rtol:   1.000E-04
   deltaMain:   1.000E-04
    lookBack:         100
     maxStep:   1.000E+02

preconBandUpper: 750 preconBandLower: 750 solverType: SPGMR + Banded Preconditioner

Finished reading solver parameters from file.

Reading model parameters from file...

Model parameters:

                               number of steps:          48
                           step size (seconds):   0.900E+03
                  species interpolation method: piecewise linear
               conditions interpolation method: piecewise linear
                           ratesOutputStepSize:        3600
               reaction rates output step size:        1800
                                modelStartTime:           0
                        jacobianOutputStepSize:           0
                                      latitude:   0.515E+02
                                     longitude:   0.130E+00
                                day/month/year:  21/ 6/2010

---

Finished reading model parameters from file.

---

Species of Interest

Reading concentration output from file... Finished reading concentration output from file. Output required for concentration of 8 species: 1 CH4 ... 8 CH3O2

---

Photolysis

Reading photolysis numbers from file... 1 ... 61 Finished reading photolysis numbers. Number of photolysis numbers: 35 Looking for photolysis constants file... Checking that photolysis constants exist in file... Photolysis constants file is empty. No photolysis constants applied, so trying constrained photolysis rates file... Looking for photolysis constraints file... Checking that photolysis constraints exist in file... Photolysis constraint file is empty, so all photolysis rates will be calculated. Reading all photolysis rates from file... 1 6.073E-05 1.743E+00 4.740E-01 J1 1.000E+00 ... 61 7.537E-04 4.990E-01 2.660E-01 J61 1.000E+00 Finished reading all photolysis rates. Number of all photolysis rates: 35

---

Environment variables

Reading environment variables... Number of environment variables: 10 1 TEMP 298.15 2 PRESS 1013.25 3 RH NOTUSED 4 H2O 3.91e+17 5 DEC 0.41 6 BLHEIGHT NOTUSED 7 DILUTE NOTUSED 8 JFAC NOTUSED 9 ROOF OPEN Finished reading environment variables.

Checking for constrained environment variables... Finished checking for constrained environment variables.

---

Constraints

Counting the variable-concentration species to be constrained (in file speciesConstrained.config)... Finished counting the names of variable-concentration constrained species. Number of names of variable-concentration constrained species: 0 Counting the fixed-concentration species to be constrained (in file speciesConstant.config)... Finished counting the names of fixed-concentration constrained species. Number of names of fixed-concentration constrained species: 0 Setting size of constraint arrays, n = 0 Skipped reading the names of variable-concentration constrained species Reading concentration data for variable-concentration constrained species... Reading in the names and concentration of the fixed constrained species (in file speciesConstant.config)... Finished reading in the names and concentration of fixed-concentration species. Finished reading constrained species. Initialising concentrations of constrained species... Finished initialising concentrations of constrained species.

---

Problem stats

                    neq = 29

numberOfConstrainedSpecies = 0 t0 = 0.000E+00

setting maxnumsteps ier = 0 setting maxstep ier = 0

---

Model run

**At line 245 of file src/solverFunctions.f90 Fortran runtime error: Index '61' of dimension 1 of array 'j' above upper bound of 60

Error termination. Backtrace:

0 0x7f9e941bfd3a

1 0x7f9e941c0849

2 0x7f9e941c0ec6

3 0x5618952bb62d

4 0x5618952c2694

5 0x5618952cf92b

6 0x5618952d1002

7 0x7f9e944a0c41

8 0x7f9e944a089f

9 0x7f9e9449e8fa

10 0x5618952d0999

11 0x5618952cccfb

12 0x5618952cee03

13 0x7f9e93fd40b2

14 0x56189527698d

15 0xffffffffffffffff**

jch@Aerosol:~/Research/AtChem2-master$

I have no idea about this. Could you give me help? Thank you very much.

Sincerely, Jun

[rs028]

This looks very similar to issue #431. Maybe it is a matter of compiler version or something else related to Ubuntu's updates?

[AlfredMayhew]

Apologies if this is the wrong place to post this, but I am also getting very similar errors to those described here and in issue #431. I am using Ubuntu, and the error began this morning after updating to Ubuntu 20.04.1 LTS. My install was working fine yesterday afternoon, so I'm quite sure that the update is the cause.

I have tried cloning a fresh AtChem2, as well as removing my atchem-lib directory and going through the install process for cvode and openlibm again, but I still end up with the same error of Fortran runtime error: Index '61' of dimension 1 of array 'j' above upper bound of 60. Hopefully this clarifies that there is some issue with regards to the latest Ubuntu update.

All the best, Alfie

[spco]

Thanks @AlfredMayhew - that's useful to have confirmation of.

Are you able to do any debugging to understand what is going wrong? To my eyes there's no issue with our code, but clearly there's something going on, and it could be our fault! I don't have access to a Ubuntu 20.04 machine, but if you're able to find out what's going on then it might be possible to either correct our code (if the update is exposing a genuine flaw in our code that previously was being glossed over) or introduce a workaround if the error sits with Ubuntu.

Thanks, Sam

[spco]

To start with, could you confirm the versions of Ubuntu and of gfortran that you were using (a) before you updated, and (b) now?

[rs028]

More in general, can you make a list of which packages were updated between yesterday and today?

[AlfredMayhew]

I'd definitely be happy to help understand what's going on here, though you may find yourself running into the limitations of my technical knowledge! My current gfortran version is "GNU Fortran (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0", and as mentioned before I am using "Ubuntu 20.04.1 LTS".

I'm unsure of how to find the previous versions of ubuntu and gfortran, but will check if you provide instructions. I believe my Ubuntu version was 18.something. Probably Ubuntu 18.04.5 LTS as that was the last LTS version I believe.

I can provide my pdkg.log file to give information about all of the package changes over the update. Though of course there are thousands of changes so I'm not sure how helpful that would be, but you could search through for any packages you were interested in. Also happy to follow any more instructions that may provide helpful information.

[rs028]

Ah, a version jump would be an awful lot of updates, so probably not very useful. Can you find out the version of the compiler before the update?

[AlfredMayhew]

I'm getting a bit lost here. Am I right in thinking that the compiler is gcc? or gfortran? If so, then the following are the lines from the dpkg.log file that contain the words "upgrade" or "install" and "gcc" or "gfortran":

2020-12-08 09:20:19 install libgfortran5:amd64 <none> 10.2.0-5ubuntu1~20.04
2020-12-08 09:20:19 install libgfortran-9-dev:amd64 <none> 9.3.0-17ubuntu1~20.04
2020-12-08 09:20:19 install gfortran-9:amd64 <none> 9.3.0-17ubuntu1~20.04
2020-12-08 09:56:29 install libgfortran4:amd64 <none> 7.5.0-6ubuntu2

2020-12-08 09:19:34 upgrade gfortran-7:amd64 7.5.0-3ubuntu1~18.04 7.5.0-6ubuntu2
2020-12-08 09:19:35 upgrade libgfortran-7-dev:amd64 7.5.0-3ubuntu1~18.04 7.5.0-6ubuntu2
2020-12-08 09:19:35 upgrade libgfortran4:amd64 7.5.0-3ubuntu1~18.04 7.5.0-6ubuntu2
2020-12-08 09:20:19 upgrade gfortran:amd64 4:7.4.0-1ubuntu2.3 4:9.3.0-1ubuntu2

2020-12-08 09:15:49 install gcc-10-base:amd64 <none> 10.2.0-5ubuntu1~20.04
2020-12-08 09:15:49 install libgcc-s1:amd64 <none> 10.2.0-5ubuntu1~20.04
2020-12-08 09:18:32 install gcc-9-base:amd64 <none> 9.3.0-17ubuntu1~20.04
2020-12-08 09:19:09 install libuno-salhelpergcc3-3:amd64 <none> 1:6.4.6-0ubuntu0.20.04.1
2020-12-08 09:19:09 install libuno-cppuhelpergcc3-3:amd64 <none> 1:6.4.6-0ubuntu0.20.04.1
2020-12-08 09:19:11 install libuno-purpenvhelpergcc3-3:amd64 <none> 1:6.4.6-0ubuntu0.20.04.1
2020-12-08 09:19:32 install libgcc-9-dev:amd64 <none> 9.3.0-17ubuntu1~20.04
2020-12-08 09:19:32 install gcc-9:amd64 <none> 9.3.0-17ubuntu1~20.04

2020-12-08 09:18:40 upgrade libgcc1:amd64 1:8.4.0-1ubuntu1~18.04 1:10.2.0-5ubuntu1~20.04
2020-12-08 09:19:31 upgrade gcc-8-base:amd64 8.4.0-1ubuntu1~18.04 8.4.0-3ubuntu2
2020-12-08 09:19:33 upgrade gcc:amd64 4:7.4.0-1ubuntu2.3 4:9.3.0-1ubuntu2
2020-12-08 09:19:34 upgrade gcc-7:amd64 7.5.0-3ubuntu1~18.04 7.5.0-6ubuntu2
2020-12-08 09:19:35 upgrade libgcc-7-dev:amd64 7.5.0-3ubuntu1~18.04 7.5.0-6ubuntu2
2020-12-08 09:19:36 upgrade gcc-7-base:amd64 7.5.0-3ubuntu1~18.04 7.5.0-6ubuntu2

From the release notes for 20.04 it says that GCC has been upgraded to 9.3. Again, I'm a bit out of my depth here, so sorry if this isn't helpful and/or I'm getting jargon confused!

[Ekimmai]

I am getting the same error.

I installed the latest version of AtChem on my Mac yesterday. All dependencies were downloaded new yesterday using homebrew.

[rs028]

Ok, seems there is a problem here :) I have no problems with the following versions:

GNU Fortran (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39)

and

GNU Fortran (GCC) 8.3.0

Can everybody here (and possibly @xiaoky97 and @vinthony from #431) post their output of gfortran --version, please?

[AlfredMayhew]

My output is: GNU Fortran (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0

[Ekimmai]

GNU Fortran (Homebrew GCC 10.2.0) 10.2.0

[rs028]

I had some free time on my hands and played around with some virtual machines. I have tested four versions of Ubuntu and it seems the problems start with version 20. Here are the details (I can add the versions of other libraries if needed):

Ubuntu	16.04.7	18.04.3	20.04.1	20.10	Manjaro
gfortran	5.4	7.5	9.3	10.2	10.2
cmake	3.5.1	3.10.2	3.16.3	3.16.3	3.19.1
python	2.7.12	2.7.15	3.8.2	3.8.6	3.8.6
libblas.so	3.6.0	3.7.1	3.9.0	3.9.0	3.9.0
liblapack.so	3.6.0	3.7.1	3.9.0	3.9.0	3.9.0
	OK	OK	crash	crash	crash

I am not sure what the source of the problem is, but one thing I noticed is that, starting with Ubuntu 20, the command python calls python3 (there is a strange package called python-is-python3 that needs to be installed). The mechanism.* files seem okay though, so I don't know if it is relevant.

EDIT: I have tested Manjaro 20 as well, and get the same error. It is basically the same configuration as Ubuntu 20.10, so at least we know it is not something Ubuntu specific.

[spco]

thanks @rs028 - I'm also taking a look at it today (not likely to solve it today though). I can see it with Homebrew gfortran 10.2 on MacOS. I can see some of what's happening, but am still in the process of figuring out the why. And why it is different to previously! I'll continue.

[spco]

The move to python3 as default is a good spot, but hopefully unrelated.

[rs028]

With a little ln -s magic, I forced the system to use Python 2.7.18, but it does not seem to make a difference.

[spco]

I've tracked it down to the update_p function from mechanism. f90 that is autogenerated, and then loaded in at runtime. I need to further understand what's going on, but something in the calling of that function is leading to the indices of j being renamed from 1 to 61 to 0 to 60, causing us our issue. I think I may need to build a standalone example to understand further.

[spco]

Hard-coding the size of j (rather than using an allocatable) allows the simulation to complete. I have yet to work out whether this is our fault or a compiler issue, and how to fix it either way.

[spco]

(Sorry everyone, I'm using this page partly for my own debugging documentation, so apologies for the spam. Hopefully it will also mean that the information is there for posterity as well).

I've explored removing the shared library functionality - so the autogenerated mechanism.f90 becomes a first-class source file like the rest. Oddly, this still exhibits the error. So it doesn't seem to be related to the use of a shared library. Changing from allocatable to statically allocated again fixes the issue.

I'm now thinking this must be an obvious bug in the mechanism.f90 source file, but will keep digging!

[spco]

Further to the above, it's more subtle - I had removed the shared library, but it was still using iso_c_binding, bind(c,name='update_p'), and real(c_double).

When I remove the shared library and remove the bind(c,name='update_p') (and thus change to use real(kind=DP)) then the issue does not appear. So it is related to the C interface.

[rs028]

Hard-coding j is not the same solution proposed by @vinthony for #431, is it?

[spco]

Not quite. Our issue is that the lower and upper bound of j get changed from 1/61 to 0/60 incorrectly. @vinthony's solution just adds an extra element to the array so that we don't call an index past the end. But it will likely not solve the problem that the lbound and ubound have changed - so now j(1) would refer to the second element (when it should refer to the first). You can see the resulting problem that would cause!

[rs028]

I see. Do you think it was doing the same error before or is it new behaviour?

[spco]

I think it's all part of the same behaviour. It seems we're doing something not standards-compliant (and interoperability with C is likely to be somewhere you really do need to be compliant). I'm hopeful of digging down to the root cause, but have no idea which way it will go in terms of fixing it!

[spco]

Fixed it 🤞! (Pull Request to be submitted tomorrow when I'm not meant to be heading to bed!)

[spco]

This should have now been fixed by #438 - please let me know whether it does so. I will close this issue for now, but please reopen if it still doesn't work. Thanks!

[rs028]

Awesome, thanks!

I think it's all part of the same behaviour. It seems we're doing something not standards-compliant (and interoperability with C is likely to be somewhere you really do need to be compliant). I'm hopeful of digging down to the root cause, but have no idea which way it will go in terms of fixing it!

What I meant is whether a model running on previous versions of Ubuntu (or whatever package of it broke with the version update) would still show the same error in counting the j index.

EDIT: never mind, I see the answer is in the comment of PR #438 :)