search

ESPRI-Mod / synda

ESGF Downloader (this is a deprecated repository, the tool has now moved to https://github.com/ESGF/esgf-download)
https://espri-mod.github.io/synda/
21 stars 11 forks source link

synda daemon crashes during synda remove #81

Closed jgs900 closed 3 years ago

jgs900 commented 7 years ago

Every time I do a 'synda remove' the daemon crashes with 'no such file or directory' - Synda 3.8

eg:

410 file(s) will be removed. After this operation, 1.0 TB of disk space will be freed. Do you want to continue? [y/N] y 410 file(s) removed Remove empty folders and files.. (may take some time)

Error occured at 2017-10-31 16:17:55.436366

Traceback (most recent call last): File "/bin/synda", line 174, in status=sdtiaction.actionsargs.subcommand File "/usr/share/python/synda/sdt/bin/sdtiaction.py", line 378, in remove return sdremove.run(args,stream) File "/usr/share/python/synda/sdt/bin/sdremove.py", line 33, in run run_remote(args,stream) File "/usr/share/python/synda/sdt/bin/sdremove.py", line 81, in run_remote remove_helper(args,metadata) File "/usr/share/python/synda/sdt/bin/sdremove.py", line 116, in remove_helper remove(metadata,(not args.keep_data)) File "/usr/share/python/synda/sdt/bin/sdremove.py", line 154, in remove sdcleanup.part_cleanup(paths) # remove paths File "/usr/share/python/synda/sdt/bin/sdcleanup.py", line 69, in part_cleanup remove_empty_files(p) File "/usr/share/python/synda/sdt/bin/sdcleanup.py", line 31, in remove_empty_files for name in os.listdir(p): OSError: [Errno 2] No such file or directory: '/g/data1/ua6/synda_tests/cmip5/output1/CSIRO-BOM/ACCESS1-0/amip/mon/atmos/Amon/r1i1p1/v20120727/clw'

hot007 commented 7 years ago

Hi Jon,

Silly question but I notice you're trying to remove from '/g/data1', does your synda VM contain the appropriate slink to /g/data1a? Just checking it is a recognised path.

cheers Claire

jgs900 commented 7 years ago

Thanks Claire - the path synda removes from should be the same path it created in the synda install step, and is based on the data directory setting given to synda - if the root directory was wrong, things should have failed in a much earlier step I'd think

painter1 commented 6 years ago

This is similar to my issue #71, except that I have not seen an error message, and I see the problem only when a Synda download job is active.

pjournou-ipsl commented 3 years ago

In the future 3.4 release (2021), the features based on the daemon will be deprecated. A new feature for asynchronous downloads will be implemented.