Threading for AWS state deletion

[james-berkheimer]

Terminating both EC2 instances and Elasticache clusters takes time with the clusters taking a very long time. I should investigate ways to background these tasks so that the user can move onto to other tasks.

[james-berkheimer]

Some example code to illustrate what I want to do:

import os
import multiprocessing
from tqdm import tqdm

def delete_resource(resource):
    # Simulate long deletion process
    os.remove(resource)
    return 1

resources = [self.aws_state_path, key_pair_path]  # Add all resources to delete here

with multiprocessing.Pool() as pool:
    with tqdm(total=len(resources)) as pbar:
        for i, _ in enumerate(pool.imap_unordered(delete_resource, resources)):
            pbar.update()