TDS 5.4 WMS Time Value Jump

[tsanah]

https://ds.nccs.nasa.gov/thredds/wms/bypass/CREATE-IP/reanalysis/MERRA2/mon/atmos/ta.ncml?service=WMS&version=1.3.0&request=GetCapabilities

1980-01-16T12:00:00.000Z,1980-02-15T12:00:00.000Z,1980-03-16T12:00:00.000Z/1980-07-16T12:00:00.000Z/P30DT12H,1980-08-16T12:00:00.000Z/1980-12-16T12:00:00.000Z/P30DT12H,1981-01-16T12:00:00.000Z,1981-02-15T00:00:00.000Z,1981-03-16T12:00:00.000Z/1981-07-16T12:00:00.000Z/P30DT12H,1981-08-16T12:00:00.000Z/1981-12-16T12:00:00.000Z/P30DT12H,1982-01-16T12:00:00.000Z,1982-02-15T00:00:00.000Z,1982-03-16T12:00:00.000Z/1982-07-16T12:00:00.000Z/P30DT12H,1982-08-16T12:00:00.000Z/1982-12-16T12:00:00.000Z/P30DT12H,1983-01-16T12:00:00.000Z,1983-02-15T00:00:00.000Z,1983-03-16T12:00:00.000Z/1983-07-16T12:00:00.000Z/P30DT12H,1983-08-16T12:00:00.000Z/1983-12-16T12:00:00.000Z our monthly dataset has each month from 01 to 03 -- then jump to 07 to 08 then again jump to 12

[tdrwenski]

Do I understand correctly that the data exists for every month but some months are not showing up in WMS? Is this only occurring with WMS and not other services? Can you also send the catalog or catalog entry you are using for this dataset so we can have closer look.

[tsanah]

Do I understand correctly that the data exists for every month but some months are not showing up in WMS? Is this only occurring with WMS and not other services? Can you also send the catalog or catalog entry you are using for this dataset so we can have closer look.

Thanks for looking into this. We used to have this work with THREDDS 4.6 -- after upgrading to 5.4 it seems to have problem

https://ds.nccs.nasa.gov/thredds/catalog/bypass/CREATE-IP/reanalysis/MERRA2/mon/atmos/catalog.html

we use ncml to aggregate files -- these are monthly average data

[tdrwenski]

Can you test if the bug is also present in the latest 5.5-SNAPSHOT version? We have fixed a number of bugs since 5.4, including some related to WMS.

Could you send the ncml for that aggregation?

[tsanah]

<?xml version="1.0" encoding="UTF-8"?>

[tsanah]

our system admin won't do the SNAPSHOT --- they only deploy stable one

[haileyajohnson]

Our SNAPSHOT actually is the most stable release. There are several CVEs present in TDS 5.4

[tsanah]

https://thredds.ucar.edu/thredds/catalog/grib/NCEP/RTMA/GUAM_2p5km/RTMA_GUAM_2p5km_20230914_0300.grib2/catalog.html?dataset=grib/NCEP/RTMA/GUAM_2p5km/RTMA_GUAM_2p5km_20230914_0300.grib2

This is your own data server please check opendap time and WMS time --- it does not make much sense to me

from opendap it seems like every 3 hours

ataset { Float64 time[time = 246]; } grib/NCEP/RTMA/GUAM_2p5km/RTMA_GUAM_2p5km_20230914_0300.grib2;

time[246] 0.0, 3.0, 6.0, 9.0, 12.0, 15.0, 18.0, 21.0, 24.0, 27.0, 30.0, 33.0, 36.0, 39.0, 42.0, 45.0, 48.0, 51.0, 54.0, 57.0, 60.0, 63.0, 66.0, 69.0, 72.0, 75.0, 78.0, 81.0, 84.0, 87.0, 90.0, 93.0, 96.0, 99.0, 102.0, 105.0, 108.0, 111.0, 114.0, 117.0, 120.0, 123.0, 126.0, 129.0, 132.0, 135.0, 138.0, 141.0, 144.0, 147.0, 150.0, 153.0, 156.0, 159.0, 162.0, 165.0, 168.0, 171.0, 174.0, 177.0, 180.0, 183.0, 186.0, 189.0, 192.0, 195.0, 198.0, 201.0, 204.0, 207.0, 210.0, 213.0, 216.0, 219.0, 222.0, 225.0, 228.0, 231.0, 234.0, 237.0, 240.0, 243.0, 246.0, 249.0, 252.0, 255.0, 258.0, 261.0, 264.0, 267.0, 270.0, 273.0, 276.0, 279.0, 282.0, 285.0, 288.0, 291.0, 294.0, 297.0, 300.0, 303.0, 306.0, 309.0, 312.0, 315.0, 318.0, 321.0, 324.0, 327.0, 330.0, 333.0, 336.0, 339.0, 342.0, 345.0, 348.0, 351.0, 354.0, 357.0, 360.0, 363.0, 366.0, 369.0, 372.0, 375.0, 378.0, 381.0, 384.0, 387.0, 390.0, 393.0, 396.0, 399.0, 402.0, 405.0, 408.0, 411.0, 414.0, 417.0, 420.0, 423.0, 426.0, 429.0, 432.0, 435.0, 438.0, 441.0, 444.0, 447.0, 450.0, 453.0, 456.0, 459.0, 462.0, 465.0, 468.0, 471.0, 474.0, 477.0, 480.0, 483.0, 486.0, 489.0, 492.0, 495.0, 498.0, 501.0, 504.0, 507.0, 510.0, 513.0, 516.0, 519.0, 522.0, 525.0, 528.0, 531.0, 534.0, 537.0, 540.0, 543.0, 546.0, 549.0, 552.0, 555.0, 558.0, 561.0, 564.0, 567.0, 570.0, 573.0, 576.0, 579.0, 582.0, 585.0, 588.0, 591.0, 594.0, 596.0, 600.0, 603.0, 606.0, 608.0, 612.0, 615.0, 618.0, 620.0, 624.0, 627.0, 630.0, 632.0, 636.0, 639.0, 642.0, 644.0, 648.0, 651.0, 654.0, 656.0, 660.0, 663.0, 666.0, 668.0, 672.0, 675.0, 678.0, 680.0, 684.0, 687.0, 690.0, 692.0, 696.0, 699.0, 702.0, 704.0, 708.0, 711.0, 714.0, 716.0, 720.0, 723.0, 726.0, 728.0, 732.0, 735.0

from WMS

2023-08-15T00:00:00.000Z/2023-08-17T12:00:00.000Z/PT3H,2023-08-17T18:00:00.000Z/2023-09-08T18:00:00.000Z/PT3H,2023-09-08T20:00:00.000Z,2023-09-09T00:00:00.000Z/2023-09-09T06:00:00.000Z/PT3H,2023-09-09T08:00:00.000Z,2023-09-09T12:00:00.000Z/2023-09-09T18:00:00.000Z/PT3H,2023-09-09T20:00:00.000Z,2023-09-10T00:00:00.000Z/2023-09-10T06:00:00.000Z/PT3H,2023-09-10T08:00:00.000Z,2023-09-10T12:00:00.000Z/2023-09-10T18:00:00.000Z/PT3H,2023-09-10T20:00:00.000Z,2023-09-11T00:00:00.000Z/2023-09-11T06:00:00.000Z/PT3H,2023-09-11T08:00:00.000Z,2023-09-11T12:00:00.000Z/2023-09-11T18:00:00.000Z/PT3H,2023-09-11T20:00:00.000Z,2023-09-12T00:00:00.000Z/2023-09-12T06:00:00.000Z/PT3H,2023-09-12T08:00:00.000Z,2023-09-12T12:00:00.000Z/2023-09-12T18:00:00.000Z/PT3H,2023-09-12T20:00:00.000Z,2023-09-13T00:00:00.000Z/2023-09-13T06:00:00.000Z/PT3H,2023-09-13T08:00:00.000Z,2023-09-13T12:00:00.000Z/2023-09-13T18:00:00.000Z/PT3H,2023-09-13T20:00:00.000Z,2023-09-14T00:00:00.000Z/2023-09-14T06:00:00.000Z/PT3H,2023-09-14T08:00:00.000Z,2023-09-14T12:00:00.000Z,2023-09-14T15:00:00.000Z

[tdrwenski]

I believe in the example you gave from our thredds server, it is an irregular point time array with spacing approximately 3, but not always (for example last points of 726.0, 728.0, 732.0, 735.0).

We use the edal-java library to handle WMS in TDS 5.x, something may have changed there from TDS 4.x when it was called ncWMS v 1.x. I believe what is happening is if there are only two points in a row with the same spacing it is listing them explicitly and if there are three or more points in a row with regular spacing it is listing them as a repeating interval. Here is an example:

for time = 0, 1, 4, 6, 8 (minutes since 2023-09-14T17:50:42.000Z)

would lead to the WMS representation of:

2023-09-14T17:50:42.000Z,2023-09-14T17:51:42.000Z,2023-09-14T17:54:42.000Z/2023-09-14T17:58:42.000Z/PT2M

so the first two times are listed explicitly and the last three are treated like two repeating intervals.

Hope this clarifies things a bit. Let us know if you still have questions.

[tsanah]

ok I am not familiar with your data but I know very well our data -- for this example

https://ds.nccs.nasa.gov/thredds/catalog/bypass/CREATE-IP/reanalysis/MERRA2/mon/atmos/catalog.html?dataset=bypass/CREATE-IP/reanalysis/MERRA2/mon/atmos/ta.ncml

time is monthly -- so around 30 days each time step from opendap 29234.5, 29264.5, 29294.5, 29325.0, 29355.5, 29386.0, 29416.5, 29447.5, 29478.0, 29508.5, 29539.0, 29569.5, 29600.5, 29630.0, 29659.5, 29690.0, 29720.5, 29751.0, 29781.5, 29812.5, 29843.0, 29873.5, 29904.0, 29934.5, 29965.5, 29995.0, 30024.5, 30055.0, 30085.5, 30116.0, 30146.5, 30177.5, 30208.0, 30238.5, 30269.0, 30299.5,

but from WMS

It jumps from March to July then from Aug to Dec keep this pattern repeated

1980-01-16T12:00:00.000Z,1980-02-15T12:00:00.000Z,1980-03-16T12:00:00.000Z/1980-07-16T12:00:00.000Z/P30DT12H,1980-08-16T12:00:00.000Z/1980-12-16T12:00:00.000Z/P30DT12H,1981-01-16T12:00:00.000Z,1981-02-15T00:00:00.000Z,1981-03-16T12:00:00.000Z/1981-07-16T12:00:00.000Z/P30DT12H,1981-08-16T12:00:00.000Z/1981-12-16T12:00:00.000Z/P30DT12H,1982-01-16T12:00:00.000Z,1982-02-15T00:00:00.000Z,1982-03-16T12:00:00.000Z/1982-07-16T12:00:00.000Z/P30DT12H,1982-08-16T12:00:00.000Z/1982-12-16T12:00:00.000Z/P30DT12H,1983-01-16T12:00:00.000Z,1983-02-15T00:00:00.000Z,1983-03-16T12:00:00.000Z/1983-07-16T12:00:00.000Z/P30DT12H,1983-08-16T12:00:00.000Z/1983-12-16T12:00:00.000Z

[tdrwenski]

What output are you expecting/ did you used to get in TDS 4.x?

[tsanah]

we don't have TDS 4.6 running now so I can not provide output but I am sure it has each month of each year

[tsanah]

Okay, this is your own data and on your own server
https://thredds.ucar.edu/thredds/catalog/grib/NCEP/GFS/Global_0p25deg_ana/GFS_Global_0p25deg_ana_20230915_0000.grib2/catalog.html?dataset=grib/NCEP/GFS/Global_0p25deg_ana/GFS_Global_0p25deg_ana_20230915_0000.grib2

I used an opendap client ferret to be able to confirm that it has time from 15-Sep-2023 00:00 to 18-SEP-2023 06:00

It has 14 time steps with interval of 6 hours .

but the WMS shows very wrong on time and as a result it screws up Godiva3 display --- you can not make any plot other than the first time step (15-Sep-2023 00:00)

[tdrwenski]

I found a URL query parameter that may solve your issue. If you add verbose=true to your WMS getCapabilities request, it will explicitly list each individual time instead of trying to condense it into a mix of ranges and times. For instance:

https://ds.nccs.nasa.gov/thredds/wms/bypass/CREATE-IP/reanalysis/MERRA2/mon/atmos/ta.ncml?service=WMS&version=1.3.0&request=GetCapabilities&verbose=true

Regarding the issue you mentioned on our thredds server-- I think you are hitting another bug in 5.4, which we have resolved in the latest 5.5-SNAPSHOT (running on our thredds-test server). Individual grib files should only contain one time. The full collection has all times, and seems to work correctly in Godiva here:

https://thredds-test.unidata.ucar.edu/thredds/catalog/grib/NCEP/GFS/Global_0p25deg_ana/catalog.html?dataset=grib/NCEP/GFS/Global_0p25deg_ana/TP

Let us know if you have any other questions!

[tsanah]

how to set verbose=true as default on THREDDS ? it was not like that for 4.6

[tdrwenski]

We probably don't want to make this the default since it can result in very large xml for large time-series and unfortunately there is currently no way to configure this. We could consider making this configurable if it is useful for TDS users (I believe that it was configurable in TDS 4.x but the default was to show all individual times).

[tdrwenski]

I am closing this as I think that the query parameter solves your issue. Let us know if you still have questions.