Improves the runtime by addressing some shortcomings that cause issues in rare situations and implementing features that provide quality of life improvements:
Splits the single-threaded worker into two threads: one thread does work, while one consumes incoming messages. This prevents the rx buffer from filling up and blocking other parts of the system while the worker is working on a long job.
Added read receipts to the waiting messages to create a "happens-before" relationship with certain messages. This ensures eventual consistency with idle worker tracking through the system and avoids a previously allowed race condition. Moreover, the manager can now always adjust their idle count to an accurate amount.
Workers now send messages instantly rather than waiting until their current job has finished or enters an awaited state. This allows for child tasks to start quicker and log messages to appear instantly.
Improved Serialization
Circuit serialization now uses dill internally (even if called with pickle) for gates that cannot be pickled normally
Circuit serialization uses pickle (even if called with dill) to pickle gates that can be pickled normally
Workflow serialization now uses dill internally, allowing lambdas to be passed as pass arguments (and more)
Additionally, workflows using dill now allows passes defined in __main__ to be used without exporting to another file
RuntimeTasks only deserialize fnargs at destination now rather than at every node in the runtime
RuntimeTasks use dill to serialize fnargs allowing lambdas to be passed to children tasks (and more)
Compilers export their sys path to the runtime, allowing normally unreachable files to be found, depending on file system similarities
Fixes: #181
Fixes: Attached servers and workers importing numpy before setting BLAS threads to 1
Fixes: Workflow copy constructor not setting all necessary variables
Address: #211
TODO:
[ ] Evaluate performance hit with using dill
[x] Allow BLAS thread settings to be configurable on server launch
Improves the runtime by addressing some shortcomings that cause issues in rare situations and implementing features that provide quality of life improvements:
__main__to be used without exporting to another fileTODO: