I was testing the behavior of using two processes to write the dict, even with SHARED_MEMORY_USE_LOCK=1, I got exceptions
Python 3.9.16. Mac OS
run the two python file in separate terminal with SHARED_MEMORY_USE_LOCK=1 set.
export SHARED_MEMORY_USE_LOCK=1 && python shm_process_safe.py
shm_process_saft.py:
import time
from shared_memory_dict import SharedMemoryDict
smd = SharedMemoryDict(name='tokens', size=1024)
smd['a'] = 'xx'
time.sleep(2)
for i in range(1000000):
smd['b'] = str(i) + 'yy'
print(smd)
# try:
# smd.shm.close()
# smd.shm.unlink()
# del smd
# except Exception as e:
# pass
in another terminal, run the other python file also with SHARED_MEMORY_USE_LOCK=1 set.
export SHARED_MEMORY_USE_LOCK=1 && python shm_process_safe2.py
shm_process_safe2.py:
import time
from shared_memory_dict import SharedMemoryDict
smd = SharedMemoryDict(name='tokens', size=1024)
print(smd)
time.sleep(1)
# smd['b']= 'yy'
print(smd)
for i in range(1000000):
smd['b'] = str(i) + 'xx'
print(smd)
# try:
# smd.shm.close()
# smd.shm.unlink()
# del smd
# except Exception as e:
# pass
Result:
Got following errors:
export SHARED_MEMORY_USE_LOCK=1 && python shm_process_safe.py
Traceback (most recent call last):
File "/Users/xx/work/code_snipet/shm/shm_process_safe.py", line 7, in <module>
smd['b'] = str(i) + 'yy'
File "/Users/xx/.virtualenvs/octopus/lib/python3.9/site-packages/shared_memory_dict/dict.py", line 95, in __setitem__
with self._modify_db() as db:
File "/usr/local/Cellar/python@3.9/3.9.16/Frameworks/Python.framework/Versions/3.9/lib/python3.9/contextlib.py", line 119, in __enter__
return next(self.gen)
File "/Users/xx/.virtualenvs/octopus/lib/python3.9/site-packages/shared_memory_dict/dict.py", line 87, in _modify_db
db = self._read_memory()
File "/Users/xx/.virtualenvs/octopus/lib/python3.9/site-packages/shared_memory_dict/dict.py", line 184, in _read_memory
return self._serializer.loads(self._memory_block.buf.tobytes())
File "/Users/xx/.virtualenvs/octopus/lib/python3.9/site-packages/shared_memory_dict/serializers.py", line 50, in loads
return pickle.loads(data)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x94 in position 6: invalid start byte
/usr/local/Cellar/python@3.9/3.9.16/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
I was testing the behavior of using two processes to write the dict, even with SHARED_MEMORY_USE_LOCK=1, I got exceptions Python 3.9.16. Mac OS run the two python file in separate terminal with SHARED_MEMORY_USE_LOCK=1 set.
export SHARED_MEMORY_USE_LOCK=1 && python shm_process_safe.py
shm_process_saft.py:in another terminal, run the other python file also with SHARED_MEMORY_USE_LOCK=1 set.
export SHARED_MEMORY_USE_LOCK=1 && python shm_process_safe2.py
shm_process_safe2.py:Result:
Got following errors: