search

NOAA-GSL / VxIngest

Other
2 stars 0 forks source link

cbimport is old version #422

Open randytpierce opened 4 days ago

randytpierce commented 4 days ago

I discovered today that the run_import,sh script on adb-cb1 was not actually importing any data and that the error was such that the exit code didn't cause the program to report an error. This happened because of a mismatch between the version of cbimport from the cbtools installation on adb-cb1 and the database version of the couchbase since the upgrade. The script uses ${HOME}/cbtools/bin/cbimport so I removed all the stuff in that cbtools and linked in the bin directory from /opt/couchbase/bin so that the tools will be exactly the same as what is in the distribution. I don't want to use an actual install dir because in the end this will need to be containerized.

I still need to investigate the build for the docker container for the import image to make sure it is using the correct tool. It probably wants to use a cbtools installation but I'll have to investigate if it is the correct version.

randytpierce commented 4 days ago

I also need to add IOProblemError to the errors that the builder politely ignores, but traces, to avoid giant traceback messages that fill up /var in the event of ingest errors reading files.

ian-noaa commented 3 days ago

Regarding the docker container - that upgrade is in PR #413.