Deploying IsoProt using Singularity

[ssamberkar]

Hi,

We (myself and our institute IT) have been trying to deploy IsoProt using Singularity due to Docker posing a security concern, requiring the user being root. While the build was successful, I was unable to log in the container to launch the ipynb notebooks.

Secondly, are there any specific hardware requirements for IsoProt to run?

Thanks.

[jgriss]

Hi @ssamberkar,

Thanks for your interest in IsoProt! We never tested IsoProt with singularity.

Can you give us some more details about your setup so we can try to replicate it?

• Do you run IsoProt on a server or on a local computer?
• How did you set the port forwarding?
• Can you maybe even share the singularity command you used to launch the container?
• How did you try to access IsoProt?

IsoProt does not have any specific hardware requirements. Depending on the size of your dataset you will require sufficient amount of RAM. But there are no requirements that would prevent IsoProt from starting.

Kind regards, Johannes

[ssamberkar]

Hi Johannes,

This is what I tried -

singularity pull docker://protprotocols/isoprot:latest

I then run Jupyter notebook on the .sif image file. Jupyter notebook was invoked however, as I mentioned previously, I couldn't log into the container.

As for RAM requirements, will 32GB on a 64GB Macbook Pro suffice to analyse 3x batches of 10 samples from a TMT experiment?

Best, Sandeep

[jgriss]

Hi Sandeep,

The looks all fine. Could you share the second command that you used to then run the Jupyter notebook with me?

The reason is that you need to forward the port from the singularity container to your local computer. This is most likely the reason why you cannot access the notebook server.

From your reply I guess that you saw on the console that the Jupyter notebook server started? Could you maybe share this output with me?

Kind regards, Johannes

[ssamberkar]

Hi Johannes,

Sorry for a delayed reply, I built the container using Singularity like so: singularity pull docker://protprotocols/isoprot:release-0.2

I then fired Jupyter on my remote machine jupyter notebook -no-browser --port=8090 isoprot_release-0.2.sif

Forwarded this port to my local machine, like so, ssh -N -f -L localhost:9090:localhost:8090 user@remotehost

This is the error I got on the console of my remote machine (HPC), [I 08:36:27.073 NotebookApp] 302 GET / (127.0.0.1) 1.15ms [W 08:36:38.594 NotebookApp] 400 GET /api/contents/isoprot_release-0.2.sif?type=file&format=text&_=1590996995355 (127.0.0.1): /mnt/gpfs2/data/molonc/samberkar/tools/Isoprot/isoprot_release-0.2.sif is not UTF-8 encoded [W 08:36:38.594 NotebookApp] /mnt/gpfs2/data/molonc/samberkar/tools/Isoprot/isoprot_release-0.2.sif is not UTF-8 encoded [W 08:36:38.595 NotebookApp] 400 GET /api/contents/isoprot_release-0.2.sif?type=file&format=text&_=1590996995355 (127.0.0.1) 3164.72ms referer=http://localhost:9090/edit/isoprot_release-0.2.sif [W 08:36:56.775 NotebookApp] 404 GET /static/edit/js/main.min.js.map (127.0.0.1) 28.95ms referer=None [W 08:36:58.451 NotebookApp] 404 GET /static/edit/js/main.min.js.map (127.0.0.1) 3.46ms referer=None [W 08:37:01.683 NotebookApp] 400 GET /api/contents/isoprot_release-0.2.sif?type=file&format=text&_=1590997018427 (127.0.0.1): /mnt/gpfs2/data/molonc/samberkar/tools/Isoprot/isoprot_release-0.2.sif is not UTF-8 encoded [W 08:37:01.683 NotebookApp] /mnt/gpfs2/data/molonc/samberkar/tools/Isoprot/isoprot_release-0.2.sif is not UTF-8 encoded [W 08:37:01.684 NotebookApp] 400 GET /api/contents/isoprot_release-0.2.sif?type=file&format=text&_=1590997018427 (127.0.0.1) 3174.55ms referer=http://localhost:9090/edit/isoprot_release-0.2.sif

And on the web browser, as I obtained from Chrome console DevTools failed to load SourceMap: Could not load content for chrome-extension://ekhagklcjbdpajgpjgmbionohlpdbjgc/browser-polyfill.js.map: HTTP error: status code 404, net::ERR_UNKNOWN_URL_SCHEME DevTools failed to load SourceMap: Could not load content for chrome-extension://gighmmpiobklfepjocnamgkkbiglidom/include.preload.js.map: HTTP error: status code 404, net::ERR_UNKNOWN_URL_SCHEME DevTools failed to load SourceMap: Could not load content for chrome-extension://gighmmpiobklfepjocnamgkkbiglidom/include.postload.js.map: HTTP error: status code 404, net::ERR_UNKNOWN_URL_SCHEME DevTools failed to load SourceMap: Could not load content for http://localhost:9090/static/edit/js/main.min.js.map: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE main.min.js?v=e3253dc1d2a1f445711555bd8ea58ada:46764 Loaded moment locale en main.min.js?v=e3253dc1d2a1f445711555bd8ea58ada:9837 GET http://localhost:9090/api/contents/isoprot_release-0.2.sif?type=file&format=text&_=1590997018427 400 (bad format) send @ main.min.js?v=e3253dc1d2a1f445711555bd8ea58ada:9837 ajax @ main.min.js?v=e3253dc1d2a1f445711555bd8ea58ada:9434 ajax @ main.min.js?v=e3253dc1d2a1f445711555bd8ea58ada:35508 (anonymous) @ main.min.js?v=e3253dc1d2a1f445711555bd8ea58ada:35548 promising_ajax @ main.min.js?v=e3253dc1d2a1f445711555bd8ea58ada:35540 Contents.get @ contents.js?v=20200601083502:102 Editor.load @ main.min.js?v=e3253dc1d2a1f445711555bd8ea58ada:45689 (anonymous) @ main.min.js?v=e3253dc1d2a1f445711555bd8ea58ada:47353 execCb @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:1693 check @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:881 (anonymous) @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:1136 (anonymous) @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:134 (anonymous) @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:1186 each @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:59 emit @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:1185 check @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:936 enable @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:1173 init @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:786 callGetModule @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:1200 completeLoad @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:1587 onScriptLoad @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:1714 load (async) req.load @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:1939 load @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:1682 load @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:832 fetch @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:822 check @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:854 enable @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:1173 enable @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:1554 (anonymous) @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:1158 (anonymous) @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:134 each @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:59 enable @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:1110 init @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:786 (anonymous) @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:1457 setTimeout (async) req.nextTick @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:1812 localRequire @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:1446 requirejs @ require.js?v=951f856e81496aaeec2e71a1c2c0d51f:1794 (anonymous) @ main.min.js?v=e3253dc1d2a1f445711555bd8ea58ada:47271 main.min.js?v=e3253dc1d2a1f445711555bd8ea58ada:35442 {readyState: 4, getResponseHeader: ƒ, getAllResponseHeaders: ƒ, setRequestHeader: ƒ, overrideMimeType: ƒ, …} main.min.js?v=e3253dc1d2a1f445711555bd8ea58ada:35444 API request failed (400): /mnt/gpfs2/data/molonc/samberkar/tools/Isoprot/isoprot_release-0.2.sif is not UTF-8 encoded main.min.js?v=e3253dc1d2a1f445711555bd8ea58ada:45705 Error loading: XhrError: /mnt/gpfs2/data/molonc/samberkar/tools/Isoprot/isoprot_release-0.2.sif is not UTF-8 encoded at wrap_ajax_error (http://localhost:9090/static/edit/js/main.min.js?v=e3253dc1d2a1f445711555bd8ea58ada:35490:29) at Object.settings.error (http://localhost:9090/static/edit/js/main.min.js?v=e3253dc1d2a1f445711555bd8ea58ada:35546:24) at fire (http://localhost:9090/static/edit/js/main.min.js?v=e3253dc1d2a1f445711555bd8ea58ada:3291:31) at Object.fireWith [as rejectWith] (http://localhost:9090/static/edit/js/main.min.js?v=e3253dc1d2a1f445711555bd8ea58ada:3421:7) at done (http://localhost:9090/static/edit/js/main.min.js?v=e3253dc1d2a1f445711555bd8ea58ada:9535:14) at XMLHttpRequest.<anonymous> (http://localhost:9090/static/edit/js/main.min.js?v=e

[jgriss]

Hi @ssamberkar,

Looking at your commands it seems that you are trying to launch the singularity image through jupyter. This cannot work. Jupyter does not know how to run singularity containers.

Instead, you actually have to run the singularity container. The easiest way is to use:

singularity run docker://protprotocols/isoprot:release-0.2

After a quick test on my machine this worked and I was able to access the Jupyter server. I did not test the complete workflow though. Therefore, I cannot confirm that everything is working. Especially, since singularity threw quite a few errors when building the image.

But this is the method to run a docker image through singularity.