search

redhat-performance / quads

:calendar: The infrastructure deployment time machine
https://quads.dev
GNU General Public License v3.0
86 stars 36 forks source link

Implement asyncio for move_and_rebuild_hosts.py #256

Closed sadsfae closed 5 years ago

sadsfae commented 5 years ago

With large deployments we're bottlenecked on parsing move_and_rebuild_hosts.py for every single host among a few hundred or more.

This is an RFE to implement the asyncio library (available in Python3) to do this more concurrently.

https://docs.python.org/3/library/asyncio.html

A logical place to start is here:

https://github.com/redhat-performance/quads/blob/master/quads/tools/move_and_rebuild_hosts.py#L21

Care or intelligent locking/sleep should be taken into consideration when interacting with JunOS devices from multiple sessions however, perhaps shuffling them towards the bottom of the queue if one is active to the same IP address / device using the queue module:

https://docs.python.org/3/library/asyncio-queue.html#asyncio-queues

sadsfae commented 5 years ago

Update: this has been promoted forward to be in our 1.1.0 release scope because it's just better to accomplish a proper asyncio implementation (and actually less change) than to hobble together the second part of https://github.com/redhat-performance/quads/issues/277

Even though python3-paramiko is not asyncio compatible we will not be using asyncio tasks for the network automation portion of QUADS as to not step on multiple connections / changes to the same switching.

grafuls commented 5 years ago

WIP under: https://review.gerrithub.io/c/redhat-performance/quads/+/464191