Open ayushr2 opened 2 years ago
willhuang1997
commented
2 years ago Hi @ayushr2 , thanks for reporting this. I tried reproducing it on my mac system but I couldn't quite reproduce it. Is there a reason why you're putting in the root folder? That seems a little odd.
ayushr2
commented
2 years ago @willhuang1997, this reproduces on Linux. If you don't have access to a Linux machine, you should be able to repro using Docker.
$ docker run --rm -it ubuntu bash
root@e5155b1e3e15:/# apt update
root@e5155b1e3e15:/# apt install -y python3-pip
root@e5155b1e3e15:/# pip install streamlit
root@e5155b1e3e15:/# touch /foo.py
root@e5155b1e3e15:/# streamlit run /foo.py
Traceback (most recent call last):
File "/usr/local/bin/streamlit", line 8, in <module>
sys.exit(main())
...
line 406, in _raise_error
raise OSError(err, os.strerror(err))
FileNotFoundError: [Errno 2] No such file or directoryIs there a reason why you're putting in the root folder? That seems a little odd.
I work on gVisor. We have multiple customers who are running streamlit under this setting. We were investigating this failure and noted that this is not a gVisor bug, because it happens on Linux too. And that proc(5) filesystem is not compatible to be recursively watched using notify the way watchdog package does it.
So on streamlit side, we should probably avoid recursively watching into /proc directory (or any other procfs mounts).
kajarenc
commented
2 years ago Hello, @ayushr2, and thanks for the detailed response!
I think this is the same bug, we discussed here https://github.com/streamlit/streamlit/issues/4842
Yes, we not support running streamlit from the OS root directory since 1.10 .
We recommend specifying WORKDIR in Dockerfile https://docs.streamlit.io/knowledge-base/tutorials/deploy/docker
CC: @snehankekre it would be great to add a paragraph in documentation, that says that we don't support running streamlit from the root of OS
vdonato
commented
2 years ago Hm, so I think that even with a workaround, we'll want to fix this rather than add an item to our docs about it (although the docs item is necessary until a fix is merged).
I do think doing so is relative low priority given that we do have an easy workaround for it, though.
ayushr2
commented
2 years ago Can the doc update be made in the interim, so people don't try to run streamlit in this way?
As for the fix, I guess it has to do with avoiding procfs mounts while adding watchers recursively.
snehankekre
commented
2 years ago CC: @snehankekre it would be great to add a paragraph in documentation, that says that we don't support running streamlit from the root of OS Yes, we do not support running streamlit from the OS root directory since 1.10 .
Does this issue affect Linux, macOS, and Windows, or just Linux distros?
snehankekre
commented
2 years ago In docs PR #482, I propose to add the following paragraph to Main concepts:
As of Streamlit version 1.10.0 and higher, Streamlit apps cannot be run from the root directory of Unix-like operating systems, including Linux and macOS. If you try to run a Streamlit app from the root directory, Streamlit will throw a
FileNotFoundError: [Errno 2] No such file or directory error.For more information, see GitHub issue #5239.If you are using Streamlit version 1.10.0 or higher, your main script should live in a directory other than the root directory. When using Docker, you can use the
WORKDIRcommand to specify the directory where your main script lives. For an example of how to do this, read Create a Dockerfile.
I've also added a callout to our Docker and Kubernetes guides:
Please let me know if these changes are accurate and sufficient or if it needs improving. cc @kajarenc @ayushr2
kajarenc
commented
2 years ago @snehankekre thank you very much, I think it looks good!
Another option to mitigate this issue could be to wrap this error and raise a more descriptive error message in core library in case fixing this completely would be problematic (i think that the most probably case in our situation)
CC: @vdonato
ayushr2
commented
2 years ago As of Streamlit version 1.10.0 and higher, Streamlit apps cannot be run from the root directory of Unix-like operating systems, including Linux and macOS.
@snehankekre I think @willhuang1997 confirmed above (in the second comment) that he can't reproduce this on his MacOS system. I am only aware of this issue occurring on Linux.
snehankekre
commented
2 years ago I think @willhuang1997 confirmed above (in the second comment) that he can't reproduce this on his MacOS system. I am only aware of this issue occurring on Linux.
@willhuang1997 can you confirm you have write-access to the root directory on your macOS machine? Were you not able to repro because you have read-only access to root (and thus can't write a .py there), or is it the case you have write-access but are still not able to repro?
On my work macOS machine, I have read-only access to root. That's why I ask to confirm. Want to confirm whether this bug affects Unix-like operating systems or just Linux.
willhuang1997
commented
2 years ago Hey @snehankekre , I only had read access on Mac and that's why it's only reproducible on Linux.
snehankekre
commented
2 years ago @ayushr2 @willhuang1997 Thanks for confirming. I was able to get a macOS VM running on my personal Linux box. Found out that since the release of Catalina, the root volume on macOS hasn't been writeable. As such, it doesn't make sense to include it in this note, as users will not be able to write .py files to root to begin with.
On #487, I've removed mentions of macOS from the note and say that running Streamlit apps from root isn't possible in Linux distributions.
mlanett
commented
1 year ago This sounds like a watchdog problem, not a streamlit problem.
I wouldn't expect anyone deploying via Docker (e.g. a read-only production-like situation) to be using watchdog.
Summary
Running something like
streamlit run /foo.pycrashes.Steps to reproduce
Expected behavior:
This crash should not happen since there is not documented constraints of where the python file can be placed.
Actual behavior:
Crashes as shown in reproducer.
Is this a regression?
I am not sure. Have not bisected across versions.
Debug info
Technical details of crash
The reason this crash happens is because streamlit tries to recursively add inotify watchers on everything inside the parent directory of the python file. In this case, the parent directory is root itself. As a result, streamlit attempts to add inotify watchers in
/procfilesystem. This fails becausewatchdog/observerspackage uses a directory file descriptor to read the contents of each directory. At some point, it comes across/proc/<selfpid>/task/<selftid>/fddirectory.getdents64(2)results on this directory naturally includes the directory file descriptor which was used forgetdents64(2). Then the directory FD is closed. Butwatchdogpackage remembers the entry and tries to add inotify watcher on that entry. But that entry no longer exists because the directory FD has been closed. Here are some strace logs (which were collected using gvisor) which show this phenomenon happening: