search

MI-DPLA / combine

Combine /kämˌbīn/ - Metadata Aggregator Platform
MIT License
26 stars 11 forks source link

Job delete, remove static upload payloads #394

Closed ghukill closed 5 years ago

ghukill commented 5 years ago

If performing large static uploads, the directory /home/combine/data/combine/static_uploads can grow to be quite large as the "temporary" payloads are not removed when the job is deleted:

root@db97c89c10df:/home/combine/data/combine/static_uploads# du -h
1.9G    ./8d37ee46-830b-4dbd-83e7-1c523d5f59c7
1.2G    ./ac0b22b2-eb13-4cad-9d84-df1f06e78a47/ngram1
1.9G    ./ac0b22b2-eb13-4cad-9d84-df1f06e78a47/ocr
1.8G    ./ac0b22b2-eb13-4cad-9d84-df1f06e78a47/metadata
4.8G    ./ac0b22b2-eb13-4cad-9d84-df1f06e78a47
3.1M    ./c3a10c77-7f66-4ba1-8358-a7bf119ce2a0
988M    ./dd0dbece-9278-42dd-86c4-8fceadc81d0e
3.1M    ./b6dc66c3-14ed-49d5-a158-c265bf3dc737
988M    ./28496763-47de-463a-814e-309c577ee3e0
3.1M    ./f2ec5263-67f3-450f-92e4-9c4c4e757361
8.6G    .

These are stored in the job_details as payload_dir and could be / should be removed when the Job is deleted from Combine.

ghukill commented 5 years ago

Done.