Support for custom threadpool

pynetbox version

v7.3.4

NetBox version

v3.6.0

Feature type

Change to existing functionality

Proposed functionality

The ability to inject a custom threadpool implementation into the Api object so users of have a more control over the behaviour.

The threadpool can be an extra, optional, argument to the Api constructor and usage would require a single line change

    def concurrent_get(self, ret, page_size, page_offsets):
        futures_to_results = []
           with self.threadpool(max_workers=4) as pool:
                ...

(from https://github.com/netbox-community/pynetbox/blob/d62cb871ca64cd38ae496292d0ecb6d84e257d70/pynetbox/core/query.py#L261C1-L263C59)

Use case

I have a custom API that uses netbox as a backend through pynetbox. This API has tracing support, which generally works well with the automatic requests (the python lib) instrumentation from open telemetry. However the tracing breaks when enabling the threading feature, as the trace id needs to be injected into the thread (contextvar I believe).

The threadpool can still be overwritten but it highly depends on internal usage and naming:

    import pynetbox.core.query

    # replace threadpool with a tracing enabled one
    import myapp.tracing.netbox_futures as nf

and netbox_futures.py

import concurrent.futures
from concurrent.futures import *  # noqa
from typing import Callable

from opentelemetry import context as otel_context
from opentelemetry.trace import get_tracer

class ThreadPoolExecutor(concurrent.futures.ThreadPoolExecutor):  # type: ignore
  ...

External dependencies

None

netbox-community / pynetbox