End of File Runtime Error

[datlee]

I'm running into a strange runtime issue when attempting to apply the software to my data, and unfortunately the error message isn't providing enough information to allow me to diagnose the problem. The code iterates through the atmosphere fitting stage without difficulty for a while before throwing the following runtime message.

At line 214 of file ../src/util_gfortran.f90 (unit = 11, file = 'TAPE11') Fortran runtime error: Sequential READ or WRITE not allowed after EOF marker, possibly use REWIND or BACKSPACE Error: Command 'cd /Users/advisor/software/TelFit-1.2/rundir1/;sh runlblrtm_v3.sh' failed in directory /Users/advisor/software/TelFit-1.2/rundir1/

I've verified that read/write permissions exist for rundir1/TAPE11, and indeed the file is updated as LBLRTM explores the parameter space. I've also successfully run fit_example/Fit.py on the same system without difficulty.

I've attempted to look into the source code for the offending file, but I am unable to locate any file called util_gfortran.f90 in the TelFit distribution that has the listed code snippet at line 214.

The software was compiled from TelFit v1.2 on OS X 10.10.3 using gfortran 4.8.5, and I reconfigured the base distribution to allow processing of low-resolution spectra as described on the FAQ page.

[kgullikson88]

Could you send the contents of the file /Users/advisor/software/TelFit-1.2/rundir1/TAPE5? Those types of cryptic errors are coming from the base LBLRTM code, and usually happen because the input gets wonky. If it runs for a while without problem, it may be trying to give an unphysical parameter value (co2 mixing ratio < 0 or something) or a value that is too big for the fortran-formatted file. If that is the case, setting tighter bounds should fix the issue.

What is the last set of parameters that it tries before crashing? It should get printed to the terminal at each iteration.

[datlee]

The last block of summary output from LBLRTM is below. I'll send a separate email with the TAPE5 file, since I can't figure out how to attach a file using the issue tracker.

Running exec: lblrtm

STOP LBLRTM EXIT

real 0m4.828s user 0m3.354s sys 0m1.368s Adjusting wavelength calibration by at most 0 nm Fitting Resolution Optimal resolution found at R = 883.204173621 Fraction of points used in X^2 evaluation: 1.0 X^2 = 377.22080777 Parameter Value Fitting? Bounds

---

temperature 2.85310E+02 True 280.45 - 290.45 h2o 6.07815E+01 True 1 - 99 o2 2.12000E+05 True 50000 - 1e+06

[kgullikson88]

Hmm, so this is going to be one of those issues. The parameter values look fine. The TAPE5 file runs just fine on my linux computer where I compiled with the intel fortran compiler, but not my Mac that used gfortran ( it gives the same error as you get). So it is an operating system/compiler-dependent issue.

I think the issue could be that the format on the TAPE5 file is slightly different from what LBLRTM wants, and that some compilers are more lenient than others. Specifically, I am concerned about the atmosphere specification (the blocks that look like)

0.0000E+000.9425E+030.3054E+03     AA L AAAAAAAAAAAA
 0.61910440E+04 0.36850000E+03 0.22157275E-01 0.32000000E+00 0.19814457E+00 0.18459507E+01 0.21200600E+06 0.39965398E-19
 0.10000000E-03 0.32074577E-02 0.10000000E-03 0.17638640E-03

Should maybe be more like

 0.000E+00 9.425E+03 3.054E+03     AA L AAAAAAAAAAAA
 0.61910440E+04 0.36850000E+03 0.22157275E-01 0.32000000E+00 0.19814457E+00 0.18459507E+01 0.21200600E+06 0.39965398E-19
 0.10000000E-03 0.32074577E-02 0.10000000E-03 0.17638640E-03

Try this: Near line 196 of MakeTape5.py, change from

outfile.write(writer("(3E10.4, 5x, 2A1, 1x, A1, 1x, A%i)" % min(39, nmol)).write(

to

outfile.write(writer("(3E10.3, 5x, 2A1, 1x, A1, 1x, A%i)" % min(39, nmol)).write(

Let me know if that fixes the issue.

[kgullikson88]

The easiest way for me to troubleshoot this would probably be for you to use the MakeModel.Modeler class to make a self-contained script that has the issue. You can probably get that by printing out out all of the arguments that your current script is passing to modeler.MakeModel(). I will also need the atmosphere profile if you are using something other than the default.

[datlee]

Update: I applied the change you suggested, and we then ran into a very similar issue but instead of pointing toward TAPE11, the error is now referencing the TAPE12 output file. I attempted to make a similar fix to the one you suggested, but I haven't been able to find the offending code lines.

I will attempt to run MakeModel.Modeler as you suggested and provide the results once I have them. I have not been running an alternative atmosphere model.

[kgullikson88]

David,

The script you sent me runs fine on both of my computers, as does the TAPE5 you included. I might not have explained it well enough - I need a minimal example of a script that fails so I can try to figure out where it is going wrong. Does buildTestModel.py fail on your computer?

For any future people reading this post, here is the buildTestModel file:

from MakeModel import Modeler
import numpy

temp = 285.45
hum = 17.0
o2 = 2.12e05
pres = 711.645

pix = int(round( 2.0 * (2.3e3-1.45e3)/1.25 ))
waves = [(1.45e3+float(i)*12.5) for i in range(pix)]
grid = numpy.array(waves, dtype="f8")
lowfreq = 1e7/waves[-1]
highfreq = 1e7/waves[0]

mBuild = Modeler(debug=True)
#mBuild.MakeModel(pressure=695.9,temperature=285.45,humidity=17.0,angle=30.2, resolution=1600.0, save=True, libfile="testlib.dat", lat=32.42, alt=3.18)
mBuild.MakeModel(wavegrid=grid, lowfreq=lowfreq, highfreq=highfreq, pressure=695.9,temperature=temp,humidity=hum,angle=30.2, o2=o2,resolution=1600.0, lat=32.42, alt=3.18, save=True)

[datlee]

Ah, okay. Somehow I didn't manage to understand that you needed a script that fails. I'm not sure exactly where the problem lies now, since TelFit seems to be successfully reading the TAPE5 file produced by the new output format, or at least that's what I'm inferring now that the code fails to read/write TAPE12 rather than TAPE11 as previously. To clarify, here's output that is now occurring immediately before the failure:

--------Begin output block----------

Running exec: lblrtm

STOP LBLRTM EXIT

real 0m5.502s user 0m3.736s sys 0m1.553s Adjusting wavelength calibration by at most -0.0020449159 nm Fitting Resolution Optimal resolution found at R = 883.204173621 Fraction of points used in X^2 evaluation: 1.0 X^2 = 377.220808769 Parameter Value Fitting? Bounds

---

temperature 2.83570E+02 True 280.45 - 290.45 h2o 6.25325E+01 True 1 - 99 o2 2.12000E+05 True 50000 - 1e+06

Running exec: lblrtm

At line 214 of file ../src/util_gfortran.f90 (unit = 12, file = 'TAPE12') Fortran runtime error: Sequential READ or WRITE not allowed after EOF marker, possibly use REWIND or BACKSPACE Error: Command 'cd /Users/advisor/software/TelFit-1.2/rundir3/;sh runlblrtm_v3.sh' failed in directory /Users/advisor/software/TelFit-1.2/rundir3/

--------End output block----------

I'll have to dig a bit further to see if I can isolate where things are going wrong now. Given that MakeModel() runs to completion and constructs valid output files, that would probably imply that the error does not occur in that step, correct?

[kgullikson88]

The error you are seeing is still coming from the LBLRTM code and is probably related to the input. The LBLRTM errors are super cryptic but in my experience have always come down to a slightly wonky TAPE5, so the MakeModel() code is still the way to go. If you do a diff on the TAPE5 generated in rundir3 right before it fails and the TAPE5 generated by your makemodel script, they are probably different. I suggest the following

• Go into the code and add some print or logging statements with all of the keywords that are passed to MakeModel()
• Run the fitting code again
• Edit your makemodel script to have all of the same parameters as the set that was last printed out.

Then, your makemodel script should make the same TAPE5 as your fitting script, and it should fail. If not, something weird is going on and I'll have to think more about it...

[datlee]

At the moment, I seem to have stumbled out of the error condition I was running into. The code is being used by my student, and in the process of trying to isolate the error I realized that there was a units problem with the wavelength argument passed to the modeling code, which caused the wavelengths passed to the code to extend past the range that was defined at compile time. This appears to have solved the problem. I am not 100% confident in the solution, but so far the error has not recurred.

[kgullikson88]

Hmm okay. I really wish the LBLRTM code would throw more informative errors! I'll leave this issue open for a week or so, to see if you get the error again at some point.

FYI: I don't think it is available in version 1.2, but the version on github and on Pypi allow you to give wavelengths as astropy Quantities, and returns you a model with the same units.

[ekpass]

I seem to be experiencing a similar issue, receiving the error "At line 214 of file ../src/util_gfortran.f90 (unit = 12, file = 'TAPE12') Fortran runtime error: Sequential READ or WRITE not allowed after EOF marker, possibly use REWIND or BACKSPACE." Following the advice above, I've determined that the following parameters are causing the crash. Any insight into resolving this?

modeler = Modeler()

pressure = 641.383
temperature = 270.137
lowfreq = 4148.20714487
highfreq = 4280.32718821
angle = 33.6972
humidity = 40.227
co2 = 450.923
o3 = 0.039
n2o = 0.531054
co = 0.0212874
ch4 = 2.02968
o2 = 381349
no = 3.31851e-19
so2 = 0.0001
no2 = 0.0001
nh3 = 0.0001
hno3 = 0.00056
lat = 19.8
alt = 4.2
wavegrid = None
resolution = None
save = False
libfile = None
vac2air = False

modeler.MakeModel(pressure=pressure, temperature=temperature, lowfreq=lowfreq, highfreq=highfreq, humidity=humidity, co2=co2, o3=o3, n2o=n2o, co=co, ch4=ch4, o2=o2, no=no, so2=so2, no2=no2, nh3=nh3, hno3=hno3, angle=angle, lat=lat, alt=alt, wavegrid=wavegrid, resolution=resolution, save=save, libfile=libfile, vac2air=vac2air)