aristanetworks / ansible-cvp

Ansible modules for Arista CloudVision
http://cvp.avd.sh
Apache License 2.0
66 stars 61 forks source link

Feature Request - Add multi threading support for get and update calls towards CV #370

Closed noredistribution closed 7 months ago

noredistribution commented 3 years ago

Request Type

Detailed Description

Is your feature request related to a problem? Please describe. Today all get calls towards CV are sent in series, most specifically the getConfigletByName.do and the updateConfiglet.do and addNoteToConfiglet.do, this can take minutes if you have 100+ configlets, with concurrent futures we can parallelize these calls and reduce the playbook execution time to just a few seconds

Possible Implementation

a quick test with cvprac

from cvprac.cvp_client import CvpClient
import ssl
from concurrent.futures import ThreadPoolExecutor
ssl._create_default_https_context = ssl._create_unverified_context
import requests.packages.urllib3
requests.packages.urllib3.disable_warnings()

# Create connection to CloudVision
clnt = CvpClient()
clnt.connect(nodes=['mycvp'], username="arista",password="arista")

import time
from functools import wraps

def get_list_of_configlets(configlets):
    """
    Create a thread pool and download specified urls
    """

    futures_list = []
    results = []

    with ThreadPoolExecutor(max_workers=40) as executor:
        for configlet in configlets:
            futures = executor.submit(clnt.api.get_configlet_by_name, configlet)
            futures_list.append(futures)

        for future in futures_list:
            try:
                result = future.result(timeout=60)
                results.append(result)
            except Exception:
                results.append(None)
    return results

if __name__ == "__main__":
    with open("configlet_list.txt") as f:
        configlets = f.read().splitlines()
    results = get_list_of_configlets(configlets)
    for result in results:
        print(result)

This takes a few seconds instead of 1+ minute when doing the calls in series

titom73 commented 2 years ago

First test available in cv_facts_v3 PR #430