Closed bondjimbond closed 6 years ago
@bondjimbond how many total items in your fetcher's input?
You mean how many items total exist in the CSV file? Around 395 for this run, 999 for the previous run. Both runs had this result.
OK, thanks, I'll take a look. Does the error happen with other values of --limit
such as 20, 30, etc.?
Confirmed: other values create same result.
-- limit X results in error message, and number of packages generated = X+1.
OK, thanks. I suspect someone, probably me, introduced an error in the --limit
option sometime in the recent past.
Hm. I'm beginning to suspect this is an issue with a third-party module, either the CSV reader or the progress bar library, because I checkout out code all the way back to March of this year and the error occurs there too (and it's impossible it would have gone undetected since March). I'll see if I can replicate this on Windows, which does not use the progress bar, tomorrow at work.
Plot thickens. Just tested this on Windows. No errors, but although I specified a limit of 10 (-l 10
) MIK produced 15 packages.
Continuing to investigate. This error doesn't happen with OAI-PMH toolchains.
You got 15 packages out of a limit 10? Very weird. Every run I've tried has been x+1.
Another thing on the Limit topic... This may merit its own issue, depending on your response. I'm wondering about what would be considered "expected behaviour". If I set --limit 5 in my test run, and objects at lines 1 and 4 are "problem records", MIK will skip over them and generate packages for 2, 3, 5, 6, 7, and 8 (because of the x+1 problem).
My expectation was that MIK should produce packages for only 2, 3, and 5, showing that objects 1 and 4 errored out... that is, my interpretation of the --limit 5 parameter is "Try the first five objects and return the results". MIK's interpretation is "Run MIK, and return 5 successful packages."
Both interpretations are valid, but what would you think is the average user's expectation?
Closing, thanks for testing.
Here's what happens whenever I run MIK with a limit option.
And 11 packages are created, instead of the 10 that are requested.