Add error checking before import

gbv / coli-ana

API to analyze DDC numbers

https://coli-conc.gbv.de/coli-ana/app/

MIT License

2 stars 0 forks source link

Closed nichtich closed 3 years ago

nichtich commented 3 years ago

Some decompositions have detectable errors, e.g.

https://coli-conc.gbv.de/coli-ana/dev/?notation=016.61419: second notations should have same length
https://coli-conc.gbv.de/coli-ana/app/?notation=094.2&mode=analyze: decompositions should not have loops (=second nortations should be unique)
...

Better detect these cases in convert.js and skip them with a message. Dev instance database could still include all numbers for error inspection.

stefandesu commented 3 years ago

Currently, main and dev are using the same database, but I agree that we should change that.

I will:

Add error detection to the script, and skip those cases by default.
Add a --ignore-errors flag (or similar) that can be used to import those cases anyway. (We can use this in dev.)
Use a separate database for the dev instance.
Reimport all data.

stefandesu commented 3 years ago

Data is currently being reimported, should be done in about 20 minutes. 👍