ERDDAP / erddap

ERDDAP is a scientific data server that gives users a simple, consistent way to download subsets of gridded and tabular scientific datasets in common file formats and make graphs and maps. ERDDAP is a Free and Open Source (Apache and Apache-like) Java Servlet from NOAA NMFS SWFSC Environmental Research Division (ERD).
Creative Commons Zero v1.0 Universal
78 stars 57 forks source link

Outer axis overlap between files #108

Open ralsei38 opened 10 months ago

ralsei38 commented 10 months ago

Hello, This is my first issue ever, sorry if I'm mistaking or missing out on something... (I digged into the documentation and into my configuration without finding the problem.) When I import a dataset, loading fails, and i get the following error message.

datasets.xml error on line #221 (this points to the closing tag of the dataset, nothing too specific
While trying to load datasetID=my_dataset (after 35112 ms)
java.lang.RuntimeException: datasets.xml error on or before line #221: Outer axis overlap between files.
max=20803.0 for /app/datasets_folder/my_dataset/file1.cdf
is greater than
min=20468.0 for /app/datasets_folder/my_dataset/file2.cdf
 at gov.noaa.pfel.erddap.dataset.EDD.fromXml(EDD.java:486)
 at gov.noaa.pfel.erddap.LoadDatasets.run(LoadDatasets.java:359)
Caused by: java.lang.RuntimeException: Outer axis overlap between files.
max=20803.0 for /app/datasets_folder/my_dataset/file1.cdf
is greater than
min=20468.0 for /app/datasets_folder/my_dataset/file2.cdf

 at gov.noaa.pfel.erddap.dataset.EDDGridFromFiles.updateValidateFileTable(EDDGridFromFiles.java:1322)
 at gov.noaa.pfel.erddap.dataset.EDDGridFromFiles.<init>(EDDGridFromFiles.java:1140)
 at gov.noaa.pfel.erddap.dataset.EDDGridFromNcLow.<init>(EDDGridFromNcLow.java:96)
 at gov.noaa.pfel.erddap.dataset.EDDGridFromNcFiles.<init>(EDDGridFromNcFiles.java:103)
 at gov.noaa.pfel.erddap.dataset.EDDGridFromFiles.fromXml(EDDGridFromFiles.java:302)
 at gov.noaa.pfel.erddap.dataset.EDD.fromXml(EDD.java:445)
 ... 1 more

Please don't mind the dataset and filenames, I rewrote them... The error is basically telling me "your maximum is greater than your minimum, so i throw you an error"

By reading the code i found an abnormal formulation (or an error) code here

if (ftMax.get(f - 1) > ftMin.get(f))
    throw new RuntimeException("Outer axis overlap between files.\n" +
        "max=" + ftMax.get(f-1) + " for " + dirList.get(ftDirIndex.get(f-1)) + ftFileList.get(f-1) + "\n" +
        "is greater than\n" +
        "min=" + ftMin.get(f)   + " for " + dirList.get(ftDirIndex.get(f  )) + ftFileList.get(f  ) + "\n");

shouldn't the condition be if (ftMax.get(f - 1) < ftMin.get(f)) ?

If this is not an error, I'm sorry for taking your time. Have a good day !

BobSimons commented 10 months ago

The code you referenced is correct. After the files are sorted based on the min value of the outer dimension in each file, the max value in one file must be less than the min of the next file. For example, if each file has a range of time values (unusual, but allowed), then the max value of one file (e.g., 0, 1, 2 days since 1985-01-01) should be less than the min of the next file (3, 4, 5).

I am confident that, as the error message indicates, the problem is related to the values in the outer dimension of your files. It is surprisingly common that a collection of files has an extra file, an incorrect file, or something else which isn't what you expect or want. Look at the values of the outer dimension in the 2 files that ERDDAP mentioned (e.g., with ncdump). I am confident that you will see that the values overlap. There are lots of possible problems. You'll just have to investigate and figure out what's going on with those outer dimension values. If you get stuck, tell us what the dimension is (e.g., time?), what the units are, and what the values are.

I hope that helps.