Open gabrielmbmb opened 4 weeks ago
@gabrielmbmb which version is this? I believe we normally strive fro providing the index of the specific record. Not sure if it would be best to just continue because it might be a small warning you end up missing and then you suddenly don't sync the expected data.
2.0.1
@frascuchon I think we should deal with this by implementing a parameter in the log
method of DatasetRecords
that defines how it deals with errors. Something like on_error
which takes a literal of raise
, skip
, return
.
raise
(default) - raise an error on the recordskip
- skip and warn on the bad recordsreturn
- return the bad records from DatasetRecords.log
so they can be processed and resent or just printed.We should also improve the logging so that it logs the record index relative to the whole dataset, not the batch.
@nataliaElv I think this is something we could easily fit into 2.2
.
Is your feature request related to a problem? Please describe. When sending to the server many records using
dataset.records.log
it can happen that one record contains a not valid value for some reason. When this happens anUnprocessableEntityError
is raised that stops the logging loop, which is really annoying.Describe the solution you'd like Instead of raising the exception, I would just let the users know that one of the records couldn't be registered in the server, and continue sending records to the server. If possible, I would let the users know in this warning the index of the record that failed, so they can go to the list of provided records to the method
dataset.records.log
and check it.