cleaning up the processing queue for specific files.

[mallorybobalice]

hi there.

sooo, we periodically see issues where an unexpected set of files ends up in the sandbox

took the advice of using something ala

curl -d "option=name&argument=^.*xxx.yyy$" http://xxx:8080/api/tasks/extendedsearch/ >dis2.txt
curl -d "option=name&argument=^.*xxx.yyy2$" http://xxx:8080/api/tasks/extendedsearch/ >>dis2.txt

clean up dis2 a bit 

while read p; do
  curl http://xxx:8080/api/tasks/delete/$p/
done <dis2.txt

problem is, this looks to as per the API only do things that have a report in mongo. That's a bit of a problem given at times the queue is also full of the same files (or rather, some have started processing, others are reported waiting for processing) .

select target,status from tasks limit 2; +-------------------------------------------------------------------------------------------------------------+----------+ | target | status | +-------------------------------------------------------------------------------------------------------------+----------+ | /home/blablablabla/somefile.pdf | reported |

what's the safe way to delete items matching a target query, from this table, and are there associated tables to delete things from?

ummm ... and ideally the associated sample, bson, malheur report. Is there an alternative API, or what do we do with mysql/cuckoo, tasks table? guess i could delete the matching analyses folders and malheur report txt files but it's not really ideal...

Thanks mb.

ps hopefully no-one will helpfully suggest python cuckoo --clean =) . bit dramatic to wipe all the samples

[mallorybobalice]

I get the sense i should be moving whatever i have above to get task ids, from a poorly written bash script to python (which also makes cleaning up json responses redunant)

then importing retention.py and calling a function from there maybe (looks to have cleanup functions for file, mongorep (redunant in my case). after deleting the mysql row in an appropriate way that doesn't make the db explode.

[mallorybobalice]

nevermind, did

update tasks
set status='failed_analysis' 
where target like '.....' 

then python cuckoo.py --failed-clean.

good enough for me for the pending analysis or reporting tasks .