search

ChileanVirtualObservatory / jovial.chivo.cl

A JupyterHub server over high-availability technologies
http://jovial.chivo.cl
2 stars 1 forks source link

Sqlite error when using Ipython #12

Open maray opened 7 years ago

maray commented 7 years ago

I get a disk I/O error when running IPython. I can actually create and modify files in lustre without problem, but maybe sqlite use a file API that is not compatible with Lustre. This is not a huge problem right now, but I think we need to investigate why this is happening to understand more complicated lustre problems in the future.

maray@jupyter-maray:~$ ipython
[TerminalIPythonApp] ERROR | Failed to open SQLite history /home/maray/.ipython/profile_default/history.sqlite (disk I/O error).
[TerminalIPythonApp] ERROR | History file was moved to /home/maray/.ipython/profile_default/history-corrupt.sqlite and a new file c
reated.
[TerminalIPythonApp] ERROR | Failed to open SQLite history /home/maray/.ipython/profile_default/history.sqlite (disk I/O error).
[TerminalIPythonApp] ERROR | History file was moved to /home/maray/.ipython/profile_default/history-corrupt.sqlite and a new file c
reated.
[TerminalIPythonApp] ERROR | Failed to open SQLite history /home/maray/.ipython/profile_default/history.sqlite (disk I/O error).
[TerminalIPythonApp] ERROR | Failed to load history too many times, history will not be saved.
mosoriob commented 7 years ago

Yes, it is related with lustre. http://lustre-discuss.clusterfs.narkive.com/COjnLwRK/lustre-sqlite

maray commented 7 years ago

I know that mounting with different locking options may cause performance degradation (as we discussed with @sirspock), but we are not thinking here on highly concurrent storage mods, but more like bulk data transfer, so I do not see the problem. If we add #13 to the mix, I think we need to do this for the nodes and the dockers... Assigning the problem to @madiazp and @sirspock