CoEDL / data-loader

0 stars 0 forks source link

failed data load #4

Open nthieberger opened 1 year ago

nthieberger commented 1 year ago

I tried loading 421 gb to an SSD and got the following error. The item it stopped at has no files, is that a problem? I tried removing it but got the same error

Preparing the target device Device ready for loading Processing the data to be loaded. Data processed Building the index. Generated the index for item: ROHP1/861 Generated the index for item: ROHP1/8610 Generated the index for item: ROHP1/8611 Generated the index for item: ROHP1/862 Generated the index for item: ROHP1/863 Generated the index for item: ROHP1/865 Generated the index for item: ROHP1/867 Generated the index for item: ROHP1/868 Generated the index for item: ROHP1/869 Generated the index for item: ROHP1/VAIB Generated the index for item: ROHP2/92502 Generated the index for item: ROHP2/92503 Generated the index for item: ROHP2/92504 Generated the index for item: ROHP2/92505 Generated the index for item: ROHP2/92506 Generated the index for item: ROHP2/92507 Generated the index for item: ROHP2/92508 Generated the index for item: ROHP2/92509 Generated the index for item: ROHP2/92510 Generated the index for item: ROHP2/92511 Generated the index for item: ROHP2/92512 Generated the index for item: ROHP2/92513 Generated the index for item: ROHP2/92514 Generated the index for item: ROHP2/92516 Generated the index for item: ROHP2/92517 Generated the index for item: ROHP2/92518 Generated the index for item: ROHP2/92519 Generated the index for item: ROHP2/92520 Generated the index for item: ROHP2/92521 this.collection.languages.language.forEach is not a function

marcolarosa commented 1 year ago

A number of bugs have been identified and squashed. In addition, the loader now skips items when an error occurs and continues on with the remaining items. Update to https://github.com/CoEDL/data-loader/releases/tag/v2.1.2 to get this latest code.