search

gbv / jskos-server

Web service to access JSKOS data
https://coli-conc.gbv.de/api/
MIT License
6 stars 4 forks source link

import script should tell number of omitted records #206

Closed nichtich closed 1 year ago

nichtich commented 1 year ago

npm run import concepts file.ndjson will silently ignore all concepts from file.nsjon that don't pass validation. Better give a message such as $n concepts not imported.

stefandesu commented 1 year ago

As far as I can see, we need to check the number of errors in this line: https://github.com/gbv/jskos-server/blob/356847d0182cb873f636b5d20b8d649a12152a70/services/concepts.js#L245

prepareAndCheckConcepts also returns a list of errors which is simply ignored during bulk import. We could add a log output to say N concepts not imported due to validation errors, or even add the URIs of the concepts. (I don't think it's easily possible to give a line number in the source file though.)

Edit: Note that this output will only be shown when importing through the CLI. In theory, bulk import can be performed through the web API as well, but in that case, we would need to report the number of errors in some response header or something. 🤔

nichtich commented 1 year ago

N concepts not imported due to validation errors should be enough. For web API the return already includes the URI of successfully imported entities, so the client is able to detect errors.