search

jupyter / help

:sparkles: Need some help or have some questions? Please visit our Discourse page.
https://discourse.jupyter.org
291 stars 97 forks source link

database is locked #150

Open tburnett opened 7 years ago

tburnett commented 7 years ago

My nbsignatures.db file is suddenly locked. It looks like what was reported here https://github.com/jupyter/nbformat/issues/52 but that seems to have been resolved, and the work-around from that discussion does not seem to be relevant. I've been using Jupyter for a long time, made no changes that I"m aware of when this suddenly stopped me in my tracks. I tried simply removing the file: when I try to create a new notebook, it makes a new zero-length one, which is again locked, as was described in that previous issue: rhel6-64c:~/guburnett/.local/share/jupyter[532]$[I 13:18:58.937 NotebookApp] Creating new notebook in /transients_P302 [E 13:19:04.008 NotebookApp] Error while saving file: transients_P302/Untitled.ipynb database is locked

rhel6-64c:~/guburnett/.local/share/jupyter[532]$sqlite3 nbsignatures.db SQLite version 3.13.0 2016-05-18 10:57:30 Enter ".help" for usage hints. sqlite> .schema Error: database is locked

This is version 4.1.0

takluyver commented 7 years ago
tburnett commented 7 years ago

Thanks for the note. I now recall that I tried to create a PDF via File/Download as, which invokes nbformat; that presumably triggered the change. I am indeed on NFS at SLAC, and had version nbformat version 4.0.1, now updated to 4.3.0. SoI can run Jupyter again, and see that IPKernelApp is working around the SQLite failures with both the history and signatures database files, apparently now a feature. So nothing is trusted now, and I have no history. At least I can work.

takluyver commented 7 years ago

If you have a writable partition which isn't on NFS, you can configure both the history and trust databases to be stored on there. With nbformat 4.3, you can also configure it to store the notebook signatures in some other way (e.g. a separate database server), but this would be a bit more effort. Unfortunately we don't have a good default that works with NFS.

avatar101 commented 6 years ago

@takluyver I'm also facing the same error of database getting locked while creating a new file and I can't figure a way to solve it. The error log:[I 15:16:34.430 NotebookApp] Creating new notebook in [E 15:16:39.542 NotebookApp] Error while saving file: Untitled.ipynb database is locked Traceback (most recent call last): File "/work/ollie/muali/miniconda3/lib/python3.6/site-packages/notebook/services/contents/filemanager.py", line 421, in save self.check_and_sign(nb, path) File "/work/ollie/muali/miniconda3/lib/python3.6/site-packages/notebook/services/contents/manager.py", line 440, in check_and_sign self.notary.sign(nb) File "/work/ollie/muali/miniconda3/lib/python3.6/site-packages/nbformat/sign.py", line 449, in sign self.store.store_signature(signature, self.algorithm) File "/work/ollie/muali/miniconda3/lib/python3.6/site-packages/nbformat/sign.py", line 211, in store_signature """, (algorithm, digest, datetime.utcnow()) sqlite3.OperationalError: database is locked [W 15:16:39.547 NotebookApp] Unexpected error while saving file: Untitled.ipynb database is locked [E 15:16:39.548 NotebookApp] { "Host": "localhost:15695", "User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0", "Accept": "application/json, text/javascript, /; q=0.01", "Accept-Language": "en-US,en;q=0.5", "Accept-Encoding": "gzip, deflate", "Referer": "http://localhost:15695/tree?", "Content-Type": "application/json", "X-Xsrftoken": "2|5f550496|5a47d6631499b635ef4bd559d1d4ad2d|1525766829", "X-Requested-With": "XMLHttpRequest", "Content-Length": "19", "Cookie": "_xsrf=2|5f550496|5a47d6631499b635ef4bd559d1d4ad2d|1525766829; username-localhost-15695=\"2|1:0|10:1525766838|24:username-localhost-15695|44:MDgyMWM5NjFhMjkxNDVkNTk0NTA5MjIwZjkzZDZmYWQ=|bfcff19897b4783e5192d53a59ebebcf04cc303bd0b7ce9de750d8eed9c32e66\"", "Connection": "keep-alive" } [E 15:16:39.548 NotebookApp] 500 POST /api/contents (127.0.0.1) 5119.43ms referer=http://localhost:15695/tree?

I'm using SQLite version 3.19.3 2017-06-08 14:26:16 and notebook version 4.3.0 and it's a BeeGFS based file system. Earlier, I also got disk I/O error and resolved it as mentioned here This is not the first time I faced database lock error. On previous ocassions, it would work by just restarting the jupyter notebook. This time around that doesn't seem to work. Thanks in advance,

avatar101 commented 6 years ago

Seems like for now I have worked my way around the error by deleting the nbsignatures.db and notebook_secret files at ~/.local/share/jupyter. But, then it gave me the familiar disk I/O error. So, I worked my way around it by following the steps above in the link where I deleted my default ipython profile at .ipython/profile_default and created a new profile and added the steps c = get_config() #gets the configuration c.HistoryManager.hist_file='/tmp/ipython_hist.sqlite' #changes history file writing to tmp folder to get rid of the the disk I/O. Now, on starting the notebook I get this message [W 16:11:34.577 NotebookApp] The signatures database cannot be opened; maybe it is corrupted or encrypted. You may need to rerun your notebooks to ensure that they are trusted to run Javascript. The old signatures database has been renamed to /home-nfs/ollie/muali/.local/share/jupyter/nbsignatures.db.bak and a new one has been created. So, from what I understand is that it has created a new signatures file. Can this lead to a problem in the future? Update: I have wrong across the same error. My database is locked again while saving the new notebook I created. Here is the log: [I 16:32:44.066 NotebookApp] Saving file at /python_notebook_ollie/netcdf_files.ipynb [E 16:32:49.120 NotebookApp] Error while saving file: python_notebook_ollie/netcdf_files.ipynb database is locked Traceback (most recent call last): File "/work/ollie/muali/miniconda3/lib/python3.6/site-packages/notebook/services/contents/filemanager.py", line 421, in save self.check_and_sign(nb, path) File "/work/ollie/muali/miniconda3/lib/python3.6/site-packages/notebook/services/contents/manager.py", line 440, in check_and_sign self.notary.sign(nb) File "/work/ollie/muali/miniconda3/lib/python3.6/site-packages/nbformat/sign.py", line 449, in sign self.store.store_signature(signature, self.algorithm) File "/work/ollie/muali/miniconda3/lib/python3.6/site-packages/nbformat/sign.py", line 219, in store_signature self.db.commit() sqlite3.OperationalError: database is locked [W 16:32:49.125 NotebookApp] Unexpected error while saving file: python_notebook_ollie/netcdf_files.ipynb database is locked [E 16:32:49.126 NotebookApp] { "Host": "localhost:15695", "User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0", "Accept": "application/json, text/javascript, /; q=0.01", "Accept-Language": "en-US,en;q=0.5", "Accept-Encoding": "gzip, deflate", "Referer": "http://localhost:15695/notebooks/python_notebook_ollie/netcdf_files.ipynb", "Content-Type": "application/json", "X-Xsrftoken": "2|5f550496|5a47d6631499b635ef4bd559d1d4ad2d|1525766829", "X-Requested-With": "XMLHttpRequest", "Content-Length": "9549", "Cookie": "_xsrf=2|5f550496|5a47d6631499b635ef4bd559d1d4ad2d|1525766829; username-localhost-15695=\"2|1:0|10:1525766838|24:username-localhost-15695|44:MDgyMWM5NjFhMjkxNDVkNTk0NTA5MjIwZjkzZDZmYWQ=|bfcff19897b4783e5192d53a59ebebcf04cc303bd0b7ce9de750d8eed9c32e66\"", "Connection": "keep-alive" } [E 16:32:49.127 NotebookApp] 500 PUT /api/contents/python_notebook_ollie/netcdf_files.ipynb (127.0.0.1) 5061.97ms referer=http://localhost:15695/notebooks/python_notebook_ollie/netcdf_files.ipynb [I 16:33:10.767 NotebookApp] Saving file at /python_notebook_ollie/netcdf_files.ipynb [E 16:33:15.810 NotebookApp] Error while saving file: python_notebook_ollie/netcdf_files.ipynb database is locked Traceback (most recent call last): File "/work/ollie/muali/miniconda3/lib/python3.6/site-packages/notebook/services/contents/filemanager.py", line 421, in save self.check_and_sign(nb, path) File "/work/ollie/muali/miniconda3/lib/python3.6/site-packages/notebook/services/contents/manager.py", line 440, in check_and_sign self.notary.sign(nb) File "/work/ollie/muali/miniconda3/lib/python3.6/site-packages/nbformat/sign.py", line 449, in sign self.store.store_signature(signature, self.algorithm) File "/work/ollie/muali/miniconda3/lib/python3.6/site-packages/nbformat/sign.py", line 219, in store_signature self.db.commit() sqlite3.OperationalError: database is locked [W 16:33:15.812 NotebookApp] Unexpected error while saving file: python_notebook_ollie/netcdf_files.ipynb database is locked [E 16:33:15.813 NotebookApp] { "Host": "localhost:15695", "User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0", "Accept": "application/json, text/javascript, /; q=0.01", "Accept-Language": "en-US,en;q=0.5", "Accept-Encoding": "gzip, deflate", "Referer": "http://localhost:15695/notebooks/python_notebook_ollie/netcdf_files.ipynb", "Content-Type": "application/json", "X-Xsrftoken": "2|5f550496|5a47d6631499b635ef4bd559d1d4ad2d|1525766829", "X-Requested-With": "XMLHttpRequest", "Content-Length": "9891", "Cookie": "_xsrf=2|5f550496|5a47d6631499b635ef4bd559d1d4ad2d|1525766829; username-localhost-15695=\"2|1:0|10:1525766838|24:username-localhost-15695|44:MDgyMWM5NjFhMjkxNDVkNTk0NTA5MjIwZjkzZDZmYWQ=|bfcff19897b4783e5192d53a59ebebcf04cc303bd0b7ce9de750d8eed9c32e66\"", "Connection": "keep-alive" } [E 16:33:15.813 NotebookApp] 500 PUT /api/contents/python_notebook_ollie/netcdf_files.ipynb (127.0.0.1) 5047.83ms referer=http://localhost:15695/notebooks/python_notebook_ollie/netcdf_files.ipynb

thomasaarholt commented 5 years ago

I am experiencing this when running the notebook over a ssh tunnel to a supercomputer.

jupyter notebook --no-browser --port=8889 # remotely
ssh -N -f -L localhost:8888:localhost:8889 thomas@myserver.com #locally

This is the terminal error. Could anyone provide insight?

[W 18:37:22.516 NotebookApp] Unexpected error while saving file: prismatic_gcc5/bin/Untitled.ipynb database is locked
[E 18:37:22.516 NotebookApp] {
      "X-Xsrftoken": "2|6d27e608|74db06c4a34632d80e009e4a62af0de4|1547154594",
      "Dnt": "1",
      "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36",
      "Content-Type": "application/json",
      "Origin": "http://localhost:8889",
      "Content-Length": "2900",
      "Accept-Encoding": "gzip, deflate, br",
      "Host": "localhost:8889",
      "Connection": "keep-alive",
      "Accept-Language": "en-US,en;q=0.9,nb;q=0.8,en-GB;q=0.7",
      "Cookie": "_xsrf=2|6d27e608|74db06c4a34632d80e009e4a62af0de4|1547154594; username-localhost-8888=\"2|1:0|10:1547625528|23:username-localhost-8888|44:M2Y3ODBiZWUyZjgwNDIxZDk3YmRlNGY0ZWE1ZWYyNmQ=|2d64d71cc7ed4f240f9bfba13277ce5e9a2bbffd403bab1021928de7adca5ccc\"; username-localhost-8889=\"2|1:0|10:1548091166|23:username-localhost-8889|44:MDVjMjc2ODA0MGUyNGI2NDllYjEyYjg4NWM2OGU4ZDg=|d74b5eae3d1a8cbae03f1c19676079b9d84f0ef7106f7d98b81b045a589e9efc\"",
      "X-Requested-With": "XMLHttpRequest",
      "Referer": "http://localhost:8889/notebooks/prismatic_gcc5/bin/Untitled.ipynb",
      "Accept": "application/json, text/javascript, */*; q=0.01"
    }