search

relaton / support

Internal repository support for Relaton
0 stars 0 forks source link

Running NIST crawler after ~year deletes data dir #42

Open kesara opened 2 months ago

kesara commented 2 months ago

The NIST crawler for bib.ietf.org was disabled due to inactivity ^1 When I manually ran it today it deleted the /data dir because GHA removed all the NIST data files. ^2.

andrew2net commented 2 months ago

@kesara the crawler removes the data before fetching updated dataset. This problem occurred because the script caught any fetching errors, so the script finished without errors even fetching failed.

The error catching is removed in relaton-nist v1.19.3. With the last version fetching errors will end the script and GHA running, so the empty data dir won't be committed.

kesara commented 2 months ago

@andrew2net thanks for the quick fix. bibxml-service has been using relaton nist v1.14.*. I'll update the Gem, hopefully things won't break.