search

pacificclimate / wx-files-service

0 stars 0 forks source link

Improved indexing #18

Closed jameshiebert closed 2 years ago

jameshiebert commented 2 years ago

This PR adds two improvements to the indexing scripts:

  1. It revises the table creation script to make database creation and population into two independent actions (for greater flexibility).
  2. It provides error handling (and logging) for the file indexing so that one problematic file does not prevent the entire run from crashing.
jameshiebert commented 2 years ago

FWIW, the skipped files are logged at the point at which they error out. But I've added a patch that collects them all and lists them in the logs at INFO level.