search

ePADD / epadd

ePADD is a software package developed by Stanford University's Special Collections & University Archives that supports archival processes around the appraisal, ingest, processing, discovery, and delivery of email archives.
https://www.epaddproject.org
111 stars 24 forks source link

Adopt BagIt specification for epadd folders #226

Closed peterchanws closed 6 years ago

peterchanws commented 6 years ago

epadd-standalone-v6-18Jun archive: full Bush Mac OS 10.13 Chrome

workflow lead to the issue

  1. index full Bush successfully in appraisal module
  2. exported full Bush archive to disk
  3. change to processing module
  4. import full bush to processing - epadd hanged here and system was reboot
  5. epadd try to load archive in appraisal module for ~1 hr without success screen shot 2018-06-21 at 12 59 25 pm epadd.log

epadd-standalone-v6-18Jun archive: full Bush Windows 10 Chrome

workflow lead to the issue

  1. index full Bush successfully in appraisal module
  2. exported full Bush archive to disk
  3. change to processing module
  4. import full bush to processing - waiting for ~1 hr without success;
  5. Restart the machine; take 1 hr 10 min to load archive in appraisal module
  6. Import archive to processing module takes 30 min.
peterchanws commented 6 years ago

Using 32GB RAM and i7 5820 K cpu epadd-standalone-v6-21June archive: full Bush Windows 10 Chrome

Time taken to load archive in processing module: 4:31 min

peterchanws commented 6 years ago

Using 16GB RAM and dual-core Intel Core i7 epadd-standalone-v6-beta-13June-non-stemmed archive: full Bush Mac OS 10.13 Chrome

Time taken to load archive in processing module: 51 sec

peterchanws commented 6 years ago

Add "Verify Checksum" under ePADD mode in Help. After users click "Verify Checksum", system will verify checksum in the bag and at the same time will prompt "This operation require a powerful machine and will take time. Working" during the calculation.

peterchanws commented 6 years ago

Bush full 16GB Win 10, laptop ver6 beta Jun27: Took 2 hrs 23 min to finish indexing ver5.1: Took 2 hrs 7 min to finish indexing

peterchanws commented 6 years ago

@chinuhub need to talk to NLPY on the exclusion of metadata file in their processing

peterchanws commented 6 years ago

ver v6-beta1-3July Took 10 min to add full Bush to processing