search

amschaal / bioshare

MIT License
6 stars 2 forks source link

Improve list directory performance #30

Closed amschaal closed 7 years ago

amschaal commented 7 years ago

Listing a directory of thousands of files can time out. Try to improve performance, maybe by eliminating stats calls for every single file. Try to get file stats all at once if possible.

amschaal commented 7 years ago

Interesting discussion: https://code.activestate.com/lists/python-list/315077/

amschaal commented 7 years ago

Turns out that checking whether the file was a text file (for preview purposes) was by far the biggest slowdown. Disabled the check, so now a user will have to try to preview to find out if it is possible! Major speed improvement though.