Displaying directory with a large number of files freezes the system

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Create a directory with a large number of files (I tried to display
Doxygen documentation directory of live555 project which has 7773 files).

2. Click on the applet's button and navigate to that directory.

Expected: user can abort directory scanning if it takes too long by moving
the mouse away from the directory entry in the menu.

Observed: Unable to abort scan, unable to switch to other windows via Alt-Tab.

What version of the product are you using? On what operating system?
Ubuntu 8.1

Original issue reported on code.google.com by jackhab on 13 Jan 2009 at 9:49

[GoogleCodeExporter]

Yeah... IO is currently not async :-o I am trying async IO in a branch, with 
little
success so far :-( It seems leaving (with the mouse) the active menu item does 
not
trigger the signal to cancel the IO operation.

I am working on it though...

Original comment by bertol...@gmail.com on 15 Jan 2009 at 11:59

• Changed state: Started

[GoogleCodeExporter]

Issue 58 has been merged into this issue.

Original comment by bertol...@gmail.com on 29 Mar 2009 at 8:38

[GoogleCodeExporter]

Issue 54 has been merged into this issue.

Original comment by bertol...@gmail.com on 29 Mar 2009 at 8:39

[GoogleCodeExporter]

Breaking directory contents into ~20 element long segments and a ``next items''
submenu at the bottom might help the situation. Like in kde 3.5.x.

Original comment by mtm...@gmail.com on 22 May 2009 at 4:07

[GoogleCodeExporter]

IO will still have to be asynch though (which is the current problem). The 
bottleneck
is the IO, not making the widgets. Showing only 20 widgets would not speed 
things up
as g_file_enumerate_children will still read the entire contents of the 
directory.

I'm a little low on motivation these days, but think this is a pretty important 
"bug"
am still (slowly) working on it.

Original comment by bertol...@gmail.com on 22 May 2009 at 4:14

• Added labels: Priority-High
• Removed labels: Priority-Medium

[GoogleCodeExporter]

I didn't look at the code before I wrote the comment, but now I see that all
properties are queried at once.

I'm not very familiar with the GFile interface, but I'm sure there is a way to 
get
only the list of files in a directory and query their properties on demand. 
Getting
only the filenames with the g_file_enumerate_children(), and then using
g_file_query_info() on the results one by one might work. This way the attribute
gathering can be interrupted at any time (eg. after N items). Of course the
enumerator must be remembered somewhere to continue the query if the user 
selects the
``next items'' submenu...

BTW I like your applet very much, I used kde primarily because of the
filesystem-browsing-from-menu feature, but it disappeared in 4.x, and they are 
not
very motivated to reintroduce it :(

Original comment by mtm...@gmail.com on 22 May 2009 at 5:08

[GoogleCodeExporter]

The properties g_file_query_info() can return depend on what was specified in
g_file_enumerate_children(). The actual IO happens somewhere in 
enumerate_children.
query_info just retrieves data from the resulting data-structure (populated by
enumerate_children).

Any way you cut it though, the IO has to be asynch because otherwise the 
interface
blocks (does not respond) while the IO is happening. In general IO should 
always be
asynch, especially if there's a gui involved.

I have an asynch branch going, and the problem isn't doing the IO 
asynchronously, its
stopping (before it finishes completely). I was hoping the menu-item's 
"deselect"
signal could be used to cancel the IO, but it can't. So I have to find some 
other way
to cancel it :-(

As for the KDE version, it was actually the inspiration for this one :-) Ironic 
that
now thins are sort-of the other way around. I suspect the motivation is low 
because
searching/tagging is much cooler these days than remembering where your shit 
is. I
bet it wouldn't be that hard to port to KDE/QT4 though. You should do it!. 
Don't let
lack of knowledge stop you; I knew exactly 0.0 about glib/gtk and that stuff 
before I
started this project (now I know just slightly more). 

Original comment by bertol...@gmail.com on 22 May 2009 at 5:27

[GoogleCodeExporter]

Issue 70 has been merged into this issue.

Original comment by bertol...@gmail.com on 5 Nov 2009 at 5:32

[GoogleCodeExporter]

So, after some testing and measuring, I appears I wasn't quite right. I/O isn't
always the bottleneck. Specifically, if a directory has a large number of small
files, I/O is relatively quick. Drawing the widgets is this slowest part (by 
far).
This is gtk actually drawing the widgets, not calling gtk_widget_show() or
gtk_widget_show_all().

So, the case for partially populating the menu and including a "more" button is 
valid
after all...

Original comment by bertol...@gmail.com on 23 Nov 2009 at 4:18

[GoogleCodeExporter]

Issue 77 has been merged into this issue.

Original comment by bertol...@gmail.com on 2 Feb 2010 at 7:35

[GoogleCodeExporter]

Part of the problem is that just hovering over a full directory freezes the 
applet, 
and there is not way to avoid it.  My Downloads directory is near the top of 
the menu, 
and is almost always too full, and this makes the applet almost unusable.  

Two possible hacks around this (before the cleaner async solution) would be 
either (a) 
adding an option do remove some fixed locations from the listing (with a 
.ignore file 
or some similar mechanism) or (b) waiting for an onclick, instead of just 
hovering, to 
read the directory and open the submenu.  

Original comment by yoav.gol...@gmail.com on 29 Mar 2010 at 9:09

[GoogleCodeExporter]

Hey,

Implementing option a properly would be not much less work than making IO 
async. As
for option b, gtk+ menus don't allow for that as far as I can tell. If a menu 
item
has a submenu, it's activated automatically.

:-(

Original comment by bertol...@gmail.com on 29 Mar 2010 at 9:26

[GoogleCodeExporter]

Issue 83 has been merged into this issue.

Original comment by bertol...@gmail.com on 16 Jun 2010 at 3:45

[GoogleCodeExporter]

Hello,

Just curious ... have you lost interest in working on this?  I really like this 
little applet ... if this big dirs bite issue were to get sorted out ... wow!

Original comment by charles....@gmail.com on 20 Sep 2010 at 3:31

[GoogleCodeExporter]

I second the above comment. This applet is awesome. But issue 52 is a 
dealbreaker for me.

Original comment by redneb8...@gmail.com on 20 Sep 2010 at 3:55

[GoogleCodeExporter]

Hi there,

Admittedly I have not worked on this all summer long. Now that that's over I'll 
have another look at this soon.

BTW, not that its a solution, but this seems to me much less of an issue with 
ext4...

Original comment by bertol...@gmail.com on 21 Sep 2010 at 2:58

[GoogleCodeExporter]

Have you looked at the code for cairo-dock's "quick-browser" applet?  It 
implements similar functionality ... perhaps it might give you an idea or two!  
Gotta luv gpl

Original comment by charles....@gmail.com on 21 Sep 2010 at 4:50