DataLoader worker (pid 2991): Bus error.

[Mirandl]

Hi, Thank you for your great work! When running your code, I got this error:

`Running TCMR on each person tracklet... 0%| | 0/5 [00:00<?, ?it/s]ERROR: Unexpected bus error encountered in worker. This might be caused by insufficient shared memory (shm). 0%| | 0/5 [00:02<?, ?it/s] Traceback (most recent call last): File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 779, in _try_get_data data = self._data_queue.get(timeout=timeout) File "/usr/lib/python3.6/multiprocessing/queues.py", line 104, in get if not self._poll(timeout): File "/usr/lib/python3.6/multiprocessing/connection.py", line 257, in poll return self._poll(timeout) File "/usr/lib/python3.6/multiprocessing/connection.py", line 414, in _poll r = wait([self], timeout) File "/usr/lib/python3.6/multiprocessing/connection.py", line 911, in wait ready = selector.select(timeout) File "/usr/lib/python3.6/selectors.py", line 376, in select fd_event_list = self._poll.poll(timeout) File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/signal_handling.py", line 66, in handler _error_if_any_worker_fails() RuntimeError: DataLoader worker (pid 2991) is killed by signal: Bus error. It is possible that dataloader's workers are out of shared memory. Please try to raise your shared memory limit.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/root/data/meilin/TCMR/demo.py", line 377, in main(args) File "/root/data/meilin/TCMR/demo.py", line 157, in main for i, batch in enumerate(crop_dataloader): File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 363, in next data = self._next_data() File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 974, in _next_data idx, data = self._get_data() File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 941, in _get_data success, data = self._try_get_data() File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 792, in _try_get_data raise RuntimeError('DataLoader worker (pid(s) {}) exited unexpectedly'.format(pids_str)) RuntimeError: DataLoader worker (pid(s) 2991) exited unexpectedly

Process finished with exit code 1 ` It seems the num_workers need to be adjusted, but I found it's no use... Can you guide me a little bit for this! Thank you!

[hongsukchoi]

the shared memory error message usually indicates that RAM (cpu memory) is insufficient. I remember that normally the experiment took around 50GB.

Try increasing RAM (more/biggr RAM cards), or though not recommended, make swap memory in the disk

[Mirandl]

Hi, thank you very much for your timely reply.

I have tried this, but it still makes the same error. My memory is 61GB and share memory is 64MB. I use 21 CPU and 1 GPU. Should I continue increasing them?

[hongsukchoi]

first check the exaxt required memory by htop. I guess at least 128g is safe!