search

galaxyproject / galaxy

Data intensive science for everyone.
https://galaxyproject.org
Other
1.39k stars 1k forks source link

/api/remote_files fails when library_import_dir targets cvmfs mount #12385

Open innovate-invent opened 3 years ago

innovate-invent commented 3 years ago

Describe the bug I have configured Galaxies library_import_dir config option to point at /cvmfs. Opening the GUI menu to import a directory into a library, the file listing never appears. Opening the browser inspector shows that the api request for the folder listing times out (http://localhost:49164/api/remote_files?target=importdir&format=jstree&disable=folders).

I have also tried specifying /cvmfs/genomes.pseudomonas.com/ and /cvmfs/genomes.pseudomonas.com/sequences as well as ensuring that the repository is mounted by listing the files outside of Galaxy.

Changing library_import_dir to any other folder outside of cvmfs does successfully list the directory contents for import.

No error appears in the log despite the request timing out.

Galaxy Version and/or server at which you observed the bug Galaxy Version: 21.01 Commit: ae3c49a07d234d810070ec085c72fefb333fdfa7

To Reproduce Steps to reproduce the behavior: Install CVMFS client and mount a repo. Set library_import_dir to /cvmfs and restart. Go to the import menu for a shared data folder.

Expected behavior It should list the cvmfs repo contents.

Screenshots image

Additional context If the server is attempting to do a full recursive listing of the import dir, that is not good as there are likely millions of files at the deepest level. I'd recommend a breadth first search at each recursion depth with a upper limit. When the limit is reached you should rerequest a recursive listing of the folder when the user tries to expand it.

innovate-invent commented 2 years ago

It looks like the UI for this has significantly changed in 21.09. I can now import paths from CVMFS by pasting the full path with file:// prefix. I am not sure what direction the UI is going in and this issue may no longer be relevant.