search

NIAID-Data-Ecosystem / nde-crawlers

Harvesting infrastructure to collect and standardize dataset and computational tool metadata
Apache License 2.0
0 stars 0 forks source link

Fix dataverse plugin #126

Closed NikkiBytes closed 3 months ago

NikkiBytes commented 4 months ago

Updated the dataverse plugin code to avoid 403 errors. Refactored with a new sleep for each 1000 data records and added in some memory logging.