indigo-dc / udocker

A basic user tool to execute simple docker containers in batch or interactive systems without root privileges.
https://indigo-dc.github.io/udocker/
Apache License 2.0
1.32k stars 133 forks source link

/dev/shm is missing or read-only such that necessary semaphores cannot be written #411

Closed yimingsun12138 closed 5 months ago

yimingsun12138 commented 8 months ago

Hi, thank you for your amazing tools!

I am running jupyterlab container on Goocle Colab using the following code:

!pip install udocker
!sudo -u knight udocker install
!sudo -u knight udocker pull mrdoge/jupyterlab
!sudo -u knight screen -d -m udocker run -p 8787:8888 mrdoge/jupyterlab
!sleep 120
!sudo -u knight udocker ps

After logging into the jupyterlab notebook using ngrok proxy, I installed some python packages:

pip install scvelo
pip install Cython
pip install velocyto

Then I create a new notbook and begin my work:

import sys
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
import loompy
import velocyto as vcy
import logging
from sklearn.svm import SVR
from sklearn.linear_model import LinearRegression
from statsmodels.nonparametric.smoothers_lowess import lowess
from scipy.interpolate import interp1d

logging.basicConfig(stream=sys.stdout, format='%(asctime)s - %(levelname)s - %(message)s', level=logging.DEBUG)
%matplotlib inline
plt.rcParams['pdf.fonttype'] = 42

The log show that:

/home/knight/miniconda3/envs/jupyterlab/lib/python3.10/site-packages/numba/np/ufunc/parallel.py:324: UserWarning: Could not obtain multiprocessing lock due to OS level error: [Errno 2] No such file or directory
A likely cause of this problem is '/dev/shm' is missing orread-only such that necessary semaphores cannot be written.
*** The responsibility of ensuring multiprocessing safe access to this initialization sequence/module import is deferred to the user! ***

My problem is that whether udocker support the shm as docker does (--shm-size tag)? Is there anything I can do to make the parallel runtime possible in my jupyterlab?

jppgks commented 7 months ago

@yimingsun12138 did you find a solution for this?

jorge-lip commented 5 months ago

Maybe try something like:

!pip install udocker
!udocker --allow-root install
!udocker --allow-root pull mrdoge/jupyterlab
!udocker --allow-root create --name=JLAB mrdoge/jupyterlab
!udocker --allow-root run --entrypoint="" JLAB tini -g -- jupyter lab --allow-root --ip=0.0.0.0 --port=8789 --no-browser & ngrok http 8789
yimingsun12138 commented 5 months ago

Maybe try something like:

!pip install udocker
!udocker --allow-root install
!udocker --allow-root pull mrdoge/jupyterlab
!udocker --allow-root create --name=JLAB mrdoge/jupyterlab
!udocker --allow-root run --entrypoint="" JLAB tini -g -- jupyter lab --allow-root --ip=0.0.0.0 --port=8789 --no-browser & ngrok http 8789

I'm not sure what happened, but ever since the recent update of udocker, I haven't encountered this problem again.

jorge-lip commented 5 months ago

Great ! thanks for the feedback.