Possible gRPC issue with Arista SSoT

[qduk]

Environment

• Python version: Unsure
• Nautobot version: 2.1.9
• nautobot-ssot version: 2.5.0

Expected Behavior

Running the Arista SSoT successfully populates data into Nautobot.

Observed Behavior

A gRPC error is raised during the sync.

Steps to Reproduce

1. Install SSoT app with Arista Integration
2. Run sync job

[qduk]

Traceback:

[2024-05-09 23:53:26,378: WARNING/ForkPoolWorker-30] Exception ignored in:
[2024-05-09 23:53:26,382: WARNING/ForkPoolWorker-30] <function _Rendezvous.__del__ at 0x7fdf5d99e480>
[2024-05-09 23:53:26,386: WARNING/ForkPoolWorker-30] Traceback (most recent call last):
[2024-05-09 23:53:26,388: WARNING/ForkPoolWorker-30]   File "/usr/local/lib/python3.11/site-packages/grpc/_channel.py", line 561, in __del__
[2024-05-09 23:53:26,394: WARNING/ForkPoolWorker-30] with self._state.condition:
[2[2024-05-09 23:53:26,445: WARNING/ForkPoolWorker-30] AttributeError
[2024-05-09 23:53:26,447: WARNING/ForkPoolWorker-30] :
[2024-05-09 23:53:26,450: WARNING/ForkPoolWorker-30] '_MultiThreadedRendezvous' object has no attribute '_state'
[2024-05-09 23:53:26,452: WARNING/ForkPoolWorker-30] Exception ignored in:
[2024-05-09 23:53:26,454: WARNING/ForkPoolWorker-30] <function _Rendezvous.__del__ at 0x7fdf5d99e480>
[2024-05-09 23:53:26,457: WARNING/ForkPoolWorker-30] Traceback (most recent call last):
[2024-05-09 23:53:26,459: WARNING/ForkPoolWorker-30]   File "/usr/local/lib/python3.11/site-packages/grpc/_channel.py", line 561, in __del__
[2024-05-09 23:53:26,463: WARNING/ForkPoolWorker-30]
[2024-05-09 23:53:26,465: WARNING/ForkPoolWorker-30] with self._state.condition:
[2024-05-09 23:53:26,516: WARNING/ForkPoolWorker-30] AttributeError
[2024-05-09 23:53:26,518: WARNING/ForkPoolWorker-30] :
[2024-05-09 23:53:26,521: WARNING/ForkPoolWorker-30] '_Rendezvous' object has no attribute '_state'
[2024-05-09 23:53:26,544: ERROR/ForkPoolWorker-30] Task nautobot.extras.jobs.run_job[a2a6ea55-fc2f-440e-89c1-8c82761c563b] raised unexpected: RpcError()
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/celery/app/trace.py", line 477, in trace_task
    R = retval = fun(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/celery/app/trace.py", line 760, in __protected_call__
    return self.run(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/nautobot/extras/jobs.py", line 1136, in run_job
    result = job(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/nautobot/extras/jobs.py", line 149, in __call__
    return self.run(*args, **deserialized_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/source/nautobot_ssot/integrations/aristacv/jobs.py", line 140, in run
    super().run(dryrun=self.dryrun, memory_profiling=self.memory_profiling, *args, **kwargs)
  File "/source/nautobot_ssot/jobs/base.py", line 317, in run
    self.sync_data(memory_profiling)
  File "/source/nautobot_ssot/jobs/base.py", line 136, in sync_data
    self.load_source_adapter()
  File "/source/nautobot_ssot/integrations/aristacv/jobs.py", line 119, in load_source_adapter
    self.source_adapter.load()
  File "/source/nautobot_ssot/integrations/aristacv/diffsync/adapters/cloudvision.py", line 267, in load
    self.load_devices()
  File "/source/nautobot_ssot/integrations/aristacv/diffsync/adapters/cloudvision.py", line 72, in load_devices
    for index, dev in enumerate(cloudvision.get_devices(client=self.conn.comm_channel, import_active=True), start=1):
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/source/nautobot_ssot/integrations/aristacv/utils/cloudvision.py", line 270, in get_devices
    for resp in responses:
  File "/usr/local/lib/python3.11/site-packages/grpc/_channel.py", line 543, in __next__
    return self._next()
           ^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/grpc/_channel.py", line 969, in _next
    raise self
grpc.RpcError

This was an error brought to me by @chambersh1129 while at Cisco Live. I need dig into it more but I wanted to get the issue open. @chambersh1129 can you provide the python version you are using?

[chambersh1129]

Python Version: 3.11.9

PLUGINS_CONFIG Arista SSoT Snippet:

PLUGINS_CONFIG = {
    "nautobot_ssot": {
        "enable_aristacv": True,
        "aristacv_cvp_token": os.getenv("NAUTOBOT_ARISTACV_TOKEN", ""),
        "aristacv_cvaas_url": "www.arista.io:443",
        "aristacv_verify": is_truthy(os.getenv("NAUTOBOT_ARISTACV_VERIFY", True)),
        "aristacv_from_cloudvision_default_site": "cloudvision_imported",
        "aristacv_from_cloudvision_default_device_role": "network",
        "aristacv_from_cloudvision_default_device_role_color": "ff0000",
        "aristacv_delete_devices_on_sync": is_truthy(os.getenv("NAUTOBOT_ARISTACV_DELETE_ON_SYNC", False)),
        "aristacv_apply_import_tag": is_truthy(os.getenv("NAUTOBOT_ARISTACV_IMPORT_TAG", True)),
        "aristacv_import_active": is_truthy(os.getenv("NAUTOBOT_ARISTACV_IMPORT_ACTIVE", False)),
        "aristacv_create_controller": is_truthy(os.getenv("NAUTOBOT_ARISTACV_CREATE_CONTROLLER", False)),
        "aristacv_controller_site": os.getenv("NAUTOBOT_ARISTACV_CONTROLLER_SITE", ""),
        "aristacv_hostname_patterns": [],  # scrubbed
        "aristacv_site_mappings": {},  # scrubbed
        "aristacv_role_mappings": {},  # scrubbed
    },
}