Open carlos-jenkins opened 8 years ago
Thinking about implementation of this enhancement, one option could be:
DockerNode
to a __del__
method.Another options was to split the destroy()
with a new cleanup()
. In this way, the topology can be stopped, in this case the containers stopped, then all containers removed in the cleaning process.
Currently, the unbuild of a topology is linear. Before a new suite starts, the previous needs to be unbuild. Depending of the topology being built, the unbuild step can be a large part of the time spent executing the suite.
Given the fact that topology_docker can run multiple suites in parallel, we can differ the unbuild of the topologies until some heuristic or event happen. In this way, the data of the test execution is available sooner.
If this mode is enabled, the unbuild will be differed to a specific "garbage collection" subprocess that will trigger the stop and removal of all the containers upon request or activation by an event or heuristic.
At the implementation details, the plugin can submit to the garbage collector all the topology objects to be discarted. The garbage collector process will query from time to time for resources, and decide to kick in if some parameter of disk space, number of containers, ram or processing is matched.
As part of this optimization, it is important to consider that the current implementation of the stop command is blocking:
https://github.com/HPENetworking/topology_docker/blob/master/lib/topology_docker/node.py#L188 https://github.com/HPENetworking/topology_docker/blob/master/lib/topology_docker/platform.py#L211
An optimization could be that once the topology is submitted to the garbage collector, the garbage collector subprocess request stop of all the containers, but do not wait or block until done. Once the event trigger the removal, the garbage collector will then wait if not stopped and then remove them. Wait and removal of the container can be done in different threads (consider the GIL), and until all of them finish then the garbage collector will be done.