Open ktbyers opened 9 months ago
Do NOT comment here--except for adding: 1)new discussion topics, 2)links to the GH issues where the discussion topics are actually addressed
Inventory + Async Discussion: https://github.com/nornir-automation/nornir/issues/880
@dbarrosop @dgarros @ogenstad Adding a few people to goals thread.
- Maintain compatibilty with current plugin system. In other words, current sync plugins and sync world should not care nor be broken by addition of async support).
I'd extend this point (or add a new one) that users should be able to pick and mix sync/async plugins. For instance, this should work (pseudocode ahead):
def my_grouped_task(task):
task.run(task=netmiko_send_command)
task.run(task=async_http_request)
nr = InitNornir(inventory=ShinyNewAsyncInventoryPlugin)
nr.run(task=netmiko_send_command)
nr.run(task=async_http_request)
nr.run(task=my_grouped_task)
The example will probably end up requiring awaits and maybe even using run_async
instead of run but that should be fine. My only point is about being able to mix them.
DISCLAIMER: this opinion of mine may change if this means nornir-core becomes too complex and/or unmaintainable.
Do we allow mixing of the sync-world and async world (albeit with blocking)?
I think we should, specially if nornir's API ends up separating sync and async methods (i.e. run
vs run_async
although I am not suggesting we should). Even if we don't, if the user cares about non blocking code I'd say they need to do some due diligence and make sure plugins are non-blocking too (maybe we could log warnings). The upside to this is that users that don't care about non-blocking code could leverage new and old plugins seamlessly (let's assume "modern" plugins may only come in async fashion).
Can plugins incrementally add async support in some way, or does the plugin need to register a new endpoint?
I'd say new plugins should be needed. I.e. NetBoxInventoryPluginAsync
vs NetBoxInventoryPlugin
. Mostly because it will probably make things simpler for nornir-core and both Sync and Async plugins can share as much code as they want without us having to make any assumptions (i.e. http calls are different but the whole post-processing of data is shared). It also makes explicit to the user whether they are using Sync or Async code. Although hopefully that'd be properly documented by the plugin and the user would take the time to read the docs. Because that's what users do, don't they? :P
Are any fundamental Nornir behaviors changed (outside the adding of async inventory and async runners)? The biggest one I see is the sequential nature of multiple nr.run() calls.
Probably Processors
which are mostly IO-bound and transform_functions.
Discussion on high-level Nornir async architecture.
How sync and async worlds interact Inventory Runner Tasks Error Handling Tests Document A bunch of others.