openaustralia / morph

Take the hassle out of web scraping
https://morph.io
GNU Affero General Public License v3.0
461 stars 74 forks source link

Intermediate container cleanup should be done at the end #1042

Closed mlandauer closed 8 years ago

mlandauer commented 8 years ago

Currently it's easily possible for an intermediate container not to get cleaned up properly. Instead why don't we tag all intermediate containers with the run id and marking the type of container it is and then do the cleanup at the end. That way we can more consistently ensure that things will get properly cleaned up if the background job runs to completion.

mlandauer commented 8 years ago

I think when I said intermediate container I think I meant intermediate image. Hmmm....

We now don't generate intermediate images so much of this problem should automatically go away. Closing this.