search

ePADD / epadd

ePADD is a software package developed by Stanford University's Special Collections & University Archives that supports archival processes around the appraisal, ingest, processing, discovery, and delivery of email archives.
https://www.epaddproject.org
111 stars 24 forks source link

Export from Appraisal to Processing not working #439

Closed kantonelli closed 1 week ago

kantonelli commented 1 year ago

Platform details:

I attempted to export from the Appraisal module to Processing. When I clicked the "Export" button, it went gray and nothing else happened.

I typically launch ePADD using the CLI (in order to allocate more RAM), and there was no readout at all when I clicked "Export". I tried launching via the executable file but the results were the same. I also tried exporting to multiple different directories and drives with large amounts of open space but the results were still the same.

I did try re-downloading ePADD version 9 and the export worked normally in that version.

jfarwer commented 1 year ago

Could you please try version 10.0.2 (the latest release - just published)?

kantonelli commented 1 year ago

I deleted my last comment soon after writing once I realized it was irrelevant - sorry for any confusion.

The "Export" button does work now, which is encouraging, and I think resolves my initial issue, so thank you for that!

The export did not complete, though - it said there was not enough space on the disk. I'm not sure why I would be getting that error since I was exporting to a disk with >3TB free space. Does there need to be a certain amount of free space available in the home folder in addition to the space in the export destination?

This is from epadd.log:

07 Feb 15:35:28 SimpleSessions INFO - The time spent in calculating metadata is 1675802128693 07 Feb 15:35:28 Indexer INFO - Closing indexer handles 07 Feb 15:53:30 Util ERROR - Error trying to export archive java.io.IOException: There is not enough space on the disk at sun.nio.ch.FileDispatcherImpl.pwrite0(Native Method) ~[?:?] at sun.nio.ch.FileDispatcherImpl.pwrite(FileDispatcherImpl.java:74) ~[?:?] at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:134) ~[?:?] at sun.nio.ch.IOUtil.write(IOUtil.java:80) ~[?:?] at sun.nio.ch.IOUtil.write(IOUtil.java:71) ~[?:?] at sun.nio.ch.FileChannelImpl.writeInternal(FileChannelImpl.java:1020) ~[?:?] at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:1001) ~[?:?] at sun.nio.ch.FileChannelImpl.transferFromFileChannel(FileChannelImpl.java:864) ~[?:?] at sun.nio.ch.FileChannelImpl.transferFrom(FileChannelImpl.java:937) ~[?:?] at org.apache.commons.io.FileUtils.doCopyFile(FileUtils.java:1147) ~[commons-io-2.4.jar:2.4] at org.apache.commons.io.FileUtils.doCopyDirectory(FileUtils.java:1428) ~[commons-io-2.4.jar:2.4] at org.apache.commons.io.FileUtils.doCopyDirectory(FileUtils.java:1426) ~[commons-io-2.4.jar:2.4] at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:1389) ~[commons-io-2.4.jar:2.4] at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:1261) ~[commons-io-2.4.jar:2.4] at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:1230) ~[commons-io-2.4.jar:2.4] at org.apache.commons.io.FileUtils.moveDirectory(FileUtils.java:2755) ~[commons-io-2.4.jar:2.4] at edu.stanford.muse.index.ArchiveReaderWriter.saveArchive(ArchiveReaderWriter.java:413) ~[classes/:?] at edu.stanford.muse.index.Archive.export(Archive.java:1758) ~[classes/:?] at org.apache.jsp.ajax.async.export_002dfrom_002dappraisal_jsp.exportFromAppraisal(export_002dfrom_002dappraisal_jsp.java:79) ~[?:?] at org.apache.jsp.ajax.async.export_002dfrom_002dappraisal_jsp$1.onStart(export_002dfrom_002dappraisal_jsp.java:228) ~[?:?] at edu.stanford.epadd.util.OperationInfo.lambda$run$0(OperationInfo.java:61) ~[classes/:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:577) ~[?:?] at java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?] at java.lang.Thread.run(Thread.java:1589) ~[?:?]

jfarwer commented 1 year ago

This is a good point. The archive is copied to the default tmp directory before going to the export destination. I think that tmp directory is defined in the TMP environment variable which is probably something like C:\Users\\AppData\Local\Temp. This is not ideal, we will think about how to improve that.

kantonelli commented 1 year ago

I was able to get more memory on my C drive, and when I re-tried the export it was successful. The step of copying to the tmp directory must have been my (secondary) issue. Thanks for your help. For now, I might recommend adding some language to the User Guide to make space needs clearer.

jfarwer commented 1 year ago

Thanks for the feedback, that's helpful. Yes, we should add something to the user guide to clarify. I hope we will have a better solution for this in the next release removing the need for this extra space on the drive with the user's home directory.

jfarwer commented 1 week ago

The location of the tmp directory has been changed in version 11.0.1. When exporting to an external hard drive data is no longer temporarily stored in the home directory.