search

pikers / piker

(e2e) foss trading for non-tinas
GNU Affero General Public License v3.0
103 stars 17 forks source link

FSP engine task re-syncs should not re-attach to shm #359

Closed goodboy closed 2 years ago

goodboy commented 2 years ago

The main error is:

Traceback (most recent call last):
  File "/home/goodboy/repos/tractor/tractor/_actor.py", line 193, in _invoke
  File "/home/goodboy/repos/piker/piker/fsp/_engine.py", line 319, in cascade
  File "/home/goodboy/repos/piker/310/lib/python3.10/site-packages/trio/_core/_run.py", line 813, in __aexit__
  File "/home/goodboy/repos/piker/piker/fsp/_engine.py", line 212, in fsp_compute
  File "/home/goodboy/repos/piker/piker/fsp/_volume.py", line 292, in flow_rates
  File "/home/goodboy/repos/piker/piker/fsp/_api.py", line 138, in get_shm
  File "/home/goodboy/repos/piker/piker/data/_sharedmem.py", line 566, in attach_shm_array
  File "/home/goodboy/repos/piker/piker/data/_sharedmem.py", line 556, in attach_shm_array
  File "/usr/lib/python3.10/multiprocessing/shared_memory.py", line 103, in __init__
OSError: [Errno 24] Too many open files: '/xmrbtc.binance.fsp.dolla_vlm.chart.15d82439-b7a1-40f0-8e24-9f67e6103523'

The source issue for this #332 but this will likely be the right workaround for that until someone can hack that issue directly in the stdlib.

the main issue is that when the underlying fsp function is run it will re-attach to any dependee shms declared in the @fsp decorator wherein the underlying Fsp.get_shm() recalls attach_shm() instead of using a cached version..

funny enough, there's already a comment about doing this 😂