search

NOAA-PMEL / LAS

Live Access Server
https://ferret.pmel.noaa.gov/LAS/
The Unlicense
13 stars 5 forks source link

LAS scripts depend on OPeNDAP access from ferret.pmel.noaa.gov. #1759

Open noaaroland opened 5 years ago

noaaroland commented 5 years ago

If there is no local etopo2.nc and/or etopo1.nc file then the LAS scripts access these via our OPeNDAP server. There is a test to see of the server works, but the test does not seem to be able to tell that a server is returning the "shutdown page" so the scripts fail.

Maybe the test should be more robust and if it fails the fallback should be to local files. At any rate, we don't want a plot to fail just because a remote OPeNDAP server is unavailable especially when the URL is defined in the script not in the configuration.

karlmsmith commented 5 years ago

(Those are actually externally accessible, but no matter.) There is a similar issue with the Ferret/PyFerret benchmark tests. I have been thinking there needs to be a way of "saying" run only the tests that do not require remote access. (For Ferret benchmarks, maybe a "check-noremote" Makefile target - not sure if this is what is needed here as well.) Should have separate test results for this "no-remote-tests" version of the benchmark tests.

noaaroland commented 5 years ago

I clarified my issue description to indicate I meant that the URL is internal to the code not part of the configuration which would make it easier to change in the event a server was unavailable.

AnsleyManke commented 5 years ago

... I've been experimenting with installing PyFerret on my personal Windows 10 machine under Ubuntu. Using the conda install it's pretty seamless, once I figured out some details about the Linux setup.

It seems to me we should be able to solve this within Ferret or the Ferret scripts. They're trying to check whether the URL can be opened, but the issue is all the errors that the netCDF library blats out, when the scripts ask for the status of the URL. It seems to me we should be able to catch that stuff so it doesn't put us into a state that makes the LAS product return an error. Our SET REDIRECT doesn't redirect this - why not? It seems the error stuff being written somewhere besides stderr / stdout. Maybe there's another call besides trying to OPEN the url and reporting the result.

yes? let status = test_opendap("http://ferret.pmel.noaa.gov/pmel/thredds/dodsC/data/PMEL/etopo2.nc")
yes? list status
syntax error, unexpected WORD_WORD, expecting SCAN_ATTR or SCAN_DATASET or SCAN_ERROR
context: <!DOCTYPE^ html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional/
.....
karlmsmith commented 5 years ago

SET REDIRECT uses SPLIT_LIST to handle the redirection. I do not see any attempt to actually change the destination of the standard Fortran LUNs. And the Fortran LUNs are different buffers from the C stdout and stderr buffers. (And the Python ones are yet another set.) The above error messages are probably being written to the C buffers.

One of my PyFerret issues is to unify all these output destinations so everything goes to the Python buffers (so Python notebooks see all this output).