search

Unidata / LDM

The Unidata Local Data Manager (LDM) system includes network client and server programs designed for event-driven data distribution, and is the fundamental component of the Unidata Internet Data Distribution (IDD) system.
http://www.unidata.ucar.edu/software/ldm
Other
43 stars 27 forks source link

Too many ignoring too-old product WARN messages in ldmd.log in version 6.13.15 #100

Closed sebenste closed 2 years ago

sebenste commented 2 years ago

Hello,

Entries like this are currently filling up our daily logs, to the tune of a half gigabyte:

20210907T161939.690039Z some.server.somewhere.com[8475] down6.c:vetProduct:226 WARN Ignoring too-old product: 58925 20210907160339.583461 CONDUIT 184 data/nccf/com/gfs/prod/gfs.20210907/12/atmos/gfs.t12z.pgrb2.1p00.f105 !grib2/ncep/GFS/#000/202109071200F105/HGHT/40 hPa PRES! 000184

Yes, it is good to know that there is a feed delay issue. However, every single product that is late with a high-volume feed can cause the log file to swell to 1 GB or more, masking other issues. It would be nice to demote these to INFO, or have an option in registry.xml to turn this feature on or off. Unless a better way is possible that I cannot think of right now. Maybe a once an hour message of "Feedtype X is experiencing a delay of 4 minutes from some.server.com".

Gilbert

semmerson commented 2 years ago

The product was created at 16:03:39 and was received at16:19:39 -- about 16 minutes later. What is the value of the "max_latency" parameter in your LDM registry?

sebenste commented 2 years ago

421 seconds....I'll try increasing it after the severe weather in our area is over.

semmerson commented 2 years ago

We recommend 3600 seconds (and a concomitantly large product-queue) so that you can be offline for an hour while you fix something.

After increasing the "max_latency" parameter, you'll have to restart your LDM for the change to take effect.

sebenste commented 2 years ago

That's what we'll do tomorrow. Looks like the value of max_latency isn't big enough.

Gilbert

On Tue, Sep 7, 2021 at 4:59 PM Steven Emmerson @.***> wrote:

We recommend 3600 seconds (and a concomitantly large product-queue) so that you can be offline for an hour while you fix something.

After increasing the "max_latency" parameter, you'll have to restart your LDM for the change to take effect.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Unidata/LDM/issues/100#issuecomment-914655527, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFLWO2OC2CYD2OGCH4HPT63UA2DMFANCNFSM5DSX6IHA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

--

Gilbert Sebenste Consulting Meteorologist AllisonHouse, LLC