search

microsoft / onefuzz

A self-hosted Fuzzing-As-A-Service platform
MIT License
2.82k stars 199 forks source link

The agent should delete output files after they are uploaded to Azure #3447

Open DrChat opened 1 year ago

DrChat commented 1 year ago

As stated, for any output files uploaded from a task, the 1F agent should delete the source files from disk after the azcopy sync operation has finished. We need this because we produce some pretty large files from our generic_analysis task, and they have the potential to fill up the OS disk if they aren't cleaned up after they have been uploaded.

The most likely place that this could be implemented is in SyncedDir::sync, adding an additional delete_src: bool parameter.

Also note that it does not appear that the azcopy team will implement this functionality: https://github.com/Azure/azure-storage-azcopy/issues/990

AB#164737

DrChat commented 1 year ago

Per discussion with @chkeita, this functionality would have to be opt-in due to the possibility that existing tasks probably expect outputs to still exist after they are uploaded.