search

USF-IMARS / erddap-config

Content dir for docker-erddap incl setup.xml & dataset.xml
0 stars 1 forks source link

SSTN_2f10_d542_b535 "Outer axis overlap between files" #11

Closed 7yl4r closed 4 years ago

7yl4r commented 4 years ago 
[root@imars-dockerhost-dune ~]# sudo docker exec -it axiom-docker-erddap bash -c "cd webapps/erddap/WEB-INF/ && bash DasDds.sh -verbose"
[...]
*** constructing EDDGridFromFiles SSTN_2f10_d542_b535
sourceAxisNames=img_y, img_x
sourceDataNames=sstn_median, sstn_anom
sourceDataTypes=double, double
dir/file table doesn't exist: /erddapData/dataset/35/SSTN_2f10_d542_b535/dirTable.nc
dir/file table doesn't exist: /erddapData/dataset/35/SSTN_2f10_d542_b535/fileTable.nc
creating new dirTable and fileTable (dirTable=null?true fileTable=null?true badFileMap=null?false)
^[[5;7~doQuickRestart=false
428 files found in /srv/imars-objects/fk/MEAN_7D_VSNPP/SSTN/
regex=.*\.nc recursive=true pathRegex=.* time=69ms
old nBadFiles size=0
sortTime=1ms
old fileTable size=0   nFilesMissing=0
#0 file=/srv/imars-objects/fk/MEAN_7D_VSNPP/SSTN/VSNPP_2012019_2012021_7D_FK_SSTN.nc
#0 inserted in cache
#1 file=/srv/imars-objects/fk/MEAN_7D_VSNPP/SSTN/VSNPP_2012022_2012028_7D_FK_SSTN.nc
#1 inserted in cache
[...]
#426 file=/srv/imars-objects/fk/MEAN_7D_VSNPP/SSTN/VSNPP_2020085_2020091_7D_FK_SSTN.nc
#426 inserted in cache
#427 file=/srv/imars-objects/fk/MEAN_7D_VSNPP/SSTN/VSNPP_2020092_2020098_7D_FK_SSTN.nc
#427 inserted in cache
fileTable updated; time=105428ms
2nd sortTime=1ms

  tFileNamePA.size=428
  dirTable.nRows=1
  fileTable.nRows=428
    fileTableInMemory=false
    nUnchanged=0
    nRemoved=0 (nNoLastMod=0, nNoSize=0)
    nReadFile=428 (nDifferentModTime=0 nNew=428) readFileCumTime=1m 44s avg=244ms

*** An error occurred while trying to load SSTN_2f10_d542_b535:
java.lang.RuntimeException: datasets.xml error on or before line #2441: Outer axis overlap between files.
max=652.0 for /srv/imars-objects/fk/MEAN_7D_VSNPP/SSTN/VSNPP_2012019_2012021_7D_FK_SSTN.nc
is greater than
min=0.0 for /srv/imars-objects/fk/MEAN_7D_VSNPP/SSTN/VSNPP_2012022_2012028_7D_FK_SSTN.nc
 at gov.noaa.pfel.erddap.dataset.EDD.fromXml(EDD.java:436)
 at gov.noaa.pfel.erddap.dataset.EDD.oneFromDatasetsXml(EDD.java:517)
 at gov.noaa.pfel.erddap.dataset.EDD.testDasDds(EDD.java:11068)
 at gov.noaa.pfel.erddap.DasDds.doIt(DasDds.java:129)
 at gov.noaa.pfel.erddap.DasDds.main(DasDds.java:155)
Caused by: java.lang.RuntimeException: Outer axis overlap between files.
max=652.0 for /srv/imars-objects/fk/MEAN_7D_VSNPP/SSTN/VSNPP_2012019_2012021_7D_FK_SSTN.nc
is greater than
min=0.0 for /srv/imars-objects/fk/MEAN_7D_VSNPP/SSTN/VSNPP_2012022_2012028_7D_FK_SSTN.nc

 at gov.noaa.pfel.erddap.dataset.EDDGridFromFiles.updateValidateFileTable(EDDGridFromFiles.java:1265)
 at gov.noaa.pfel.erddap.dataset.EDDGridFromFiles.<init>(EDDGridFromFiles.java:1092)
 at gov.noaa.pfel.erddap.dataset.EDDGridFromNcLow.<init>(EDDGridFromNcLow.java:96)
 at gov.noaa.pfel.erddap.dataset.EDDGridFromNcFiles.<init>(EDDGridFromNcFiles.java:104)
 at gov.noaa.pfel.erddap.dataset.EDDGridFromFiles.fromXml(EDDGridFromFiles.java:294)
 at gov.noaa.pfel.erddap.dataset.EDD.fromXml(EDD.java:396)
 ... 4 more
7yl4r commented 4 years ago

@dotis : ERDDAP is throwing an "Outer axis overlap between files" error when trying to load your new files. I haven't seen this error before. Is there anything unusual about these files (specifically the VIIRS SSTN files) when compared with the others we have loaded into ERDDAP?

Perhaps VIIRS passes overlap and MODIS do not?

dotis commented 4 years ago

I can't tell what's happening here. From what I see, the VIIRS SSTN files are just like the SST and OC. Do the VIRRS OC files work ok? Other changes: Compared to MODIS, the resolution of the VIIRS files is higher. The fk region has now also expanded. The new dashboard files, both MODA and VSNPP, now have the following extent: 23N to 27.4N/-79.0E to -85E. Not sure if these changes will affect the transition into ERDDAP. My thought was to show images of the full extent on the FWC board, but keep the old limits for the FKNMS (24N to 26N/-79.5E to -83.5E). I think we can choose the limits on a case by case basis? Although, this might mean two sets of files in ERDDAP, one for each board.

7yl4r commented 4 years ago

The new MODIS and VIIRS images are also not working but I have not yet checked they have the same problem, it could be something else.

Your plan to use different extents should work fine, we can set the limits in grafana.

From: dotis notifications@github.com Sent: Tuesday, April 14, 2020 2:30 PM To: USF-IMARS/erddap-config erddap-config@noreply.github.com Cc: Tylar tylarmurray@mail.usf.edu; Assign assign@noreply.github.com Subject: Re: [USF-IMARS/erddap-config] SSTN_2f10_d542_b535 "Outer axis overlap between files" (#11)

This email originated from outside of USF. Do not click links or open attachments unless you recognize the sender or understand the content is safe.

I can't tell what's happening here. From what I see, the VIIRS SSTN files are just like the SST and OC. Do the VIRRS OC files work ok? Other changes: Compared to MODIS, the resolution of the VIIRS files is higher. The fk region has now also expanded. The new dashboard files, both MODA and VSNPP, now have the following extent: 23N to 27.4N/-79.0E to -85E. Not sure if these changes will affect the transition into ERDDAP. My thought was to show images of the full extent on the FWC board, but keep the old limits for the FKNMS (24N to 26N/-79.5E to -83.5E). I think we can choose the limits on a case by case basis? Although, this might mean two sets of files in ERDDAP, one for each board.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHubhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FUSF-IMARS%2Ferddap-config%2Fissues%2F11%23issuecomment-613607576&data=02%7C01%7Ctylarmurray%40usf.edu%7C4b54585f6e484218bcf008d7e0a1fe78%7C741bf7dee2e546df8d6782607df9deaa%7C0%7C0%7C637224858670150984&sdata=G%2B6cQI4jBy8gikIkOhF%2FrnUpv8njQf5HlTMk4piMQKw%3D&reserved=0, or unsubscribehttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAAIAV7SG3JEH6JSMQDNCDMLRMSTT5ANCNFSM4MH4VFYQ&data=02%7C01%7Ctylarmurray%40usf.edu%7C4b54585f6e484218bcf008d7e0a1fe78%7C741bf7dee2e546df8d6782607df9deaa%7C0%7C0%7C637224858670160972&sdata=I%2BvgsODnnaGIPpi9XwG9%2F%2Bi7O%2Bnrt5BHHYVdTjEVbmQ%3D&reserved=0.

7yl4r commented 4 years ago

Same issue for all the other new datasets too ( SST_66bb_c258_cd59, OC_2a67_61be_9fd6, SST4_aac4_99a3_6549, SST_2963_b743_ee2e ).