search

iboboc / relay

Automatically exported from code.google.com/p/relay
MIT License
0 stars 0 forks source link

CRITICAL: move Image/Document thumbnails from db to filesystem #2

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago 
The current image/document thumbnail system stores thumbnail images as binary 
BLOBs in the MySQL database.  This is undesirable both due to the potential for 
database bloat and because, as currently implemented, folders with large number 
of files HAMMER the database with potentially thousands of single-row SELECT 
statements at once.

Instead, the thumbnailer should use a parallel directory to the /filestore 
directory, called /thumbnails.  The thumbnails should be stored in /thumbnails 
exactly as the files are in /filestore; for example 
/filestore/folder/subfolder/document.pdf should have its thumbnail stored in 
/thumbnails/folder/subfolder/document.pdf.png.

Also, Relay currently generates or retrieves thumbnails for all files it can 
see at once.  This behavior does NOT scale well with large sets of files.  
Instead, the thumbnailer should only be called when the file in question is 
selected - it should first check for the existence of a thumbnail, then 
generate and store one if one was not found (and the file is of a type that 
should be thumbnailed).

There should be a control at the bottom of the "File Information" dialog to 
manually regenerate a thumbnail for a given file, and administrators of a given 
folder should have a control to generate thumbnails for all files in that 
folder (possibly an entry in the folder's context menu, which only shows up if 
the logged in user has admin rights on that folder).

The easiest way to do most of this is probably simply to have the image source 
for a thumbnail image point to a php script; the script would echo the image 
data out to the browser with type image/png as well as doing the necessary 
maintenance in the background.

Original issue reported on code.google.com by jim.salt...@gmail.com on 10 Jun 2011 at 8:22

GoogleCodeExporter commented 8 years ago 
This issue causes such a performance hit that Relay is effectively unusable 
with 1,000 files or more - just opening the interface after logging in can take 
upwards of 60 seconds on an unloaded, reasonably fast server.

Original comment by jim.salt...@gmail.com on 10 Jun 2011 at 8:27

GoogleCodeExporter commented 8 years ago 
To resolve the performance aspects of this issue, we want to make sure that the 
MySQL db never needs to be queried for anything but user login, and checking 
permissions on a virtual folder.  This avoids the issue of hundreds or 
thousands of concurrent SELECTs being performed when viewing large filesets.

Original comment by jim.salt...@gmail.com on 11 Jun 2011 at 8:39

GoogleCodeExporter commented 8 years ago

Original comment by jim.salt...@gmail.com on 11 Jun 2011 at 9:11

GoogleCodeExporter commented 8 years ago

Original comment by jim.salt...@gmail.com on 17 Jun 2011 at 3:08

GoogleCodeExporter commented 8 years ago 
fixed in Relay 1.5.1

Original comment by jim.salt...@gmail.com on 10 Jul 2011 at 5:25