Heether / font-manager

Automatically exported from code.google.com/p/font-manager
GNU General Public License v3.0
0 stars 0 forks source link

Adding font directory does not cause rescan. #1

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago
What steps will reproduce the problem?
1. Start font-manager
2. Select 'Preferences' button
3. Press '+' button to add a new scan directory (I added '/')
4. Observe new directory added
5. Press 'Apply Changes'
6. -- Observe: New directory is not scanned --
7. Exit and restart font-manager
8. -- Observe: New directory is not scanned --

What is the expected output? What do you see instead?

I expected to see my entire system re-scanned for fonts.
I tried this because Gnome Character Map showed fonts that were not found
by the font-manager initial scan.

What version of the product are you using? On what operating system? What
desktop environment?

font-manager 0.3 on freshly updated F11.

Please provide any additional information below.

- How do I force a font rescan?  Is there a font-manager file somewhere I
can delete?  Adding a "Force Rescan Now" button would be useful...

Original issue reported on code.google.com by FlyM...@gmail.com on 18 Sep 2009 at 11:00

GoogleCodeExporter commented 9 years ago
Correction to step 2: Press 'Set application preferences' button.

Original comment by FlyM...@gmail.com on 18 Sep 2009 at 11:26

GoogleCodeExporter commented 9 years ago
Update: It seems the scan is not recursive.  All font dirs must be added one at 
a time!

Please either make recursive scanning the default mode, or add a "Scan 
Recursively"
checkbox when adding a font directory.

I have over 100 directories of fonts on my system!  It was easier to bypass the 
GUI
to add all the dirs by manually editing ~/.FontManager/config/dirs.conf.

Original comment by FlyM...@gmail.com on 18 Sep 2009 at 11:51

GoogleCodeExporter commented 9 years ago
Here's the command I used to automate finding and adding all my TTF directories:

$ locate -i ttf | grep -i [.]ttf$ | xargs -l1 -i dirname "{}" | sort | uniq | 
xargs
-i echo "  <dir>{}</dir>" >> ~/.FontManager/config/dirs.conf

Then I edited the file to move the "</fontconfig>" line to the end of the file.

Original comment by FlyM...@gmail.com on 19 Sep 2009 at 12:21

GoogleCodeExporter commented 9 years ago
The prior comment assumes you have run 'updatedb' since your last fonts were 
added.

Original comment by FlyM...@gmail.com on 19 Sep 2009 at 12:28

GoogleCodeExporter commented 9 years ago
Thanks for taking the time to report this.

The fact that the application allowed adding '/' as a directory to scan is a 
huge
flaw. I'm sorry to say I just never considered adding the entire filesystem.

Asking fontconfig to scan thousands of directories is a terrible idea, it would
probably bring your system to it's knees if it did.

Any directory added to the search path IS scanned recursively by fontconfig, 
not sure
how deep it will go, but from my tests I can gaurantee at least 20 directories 
deep.

This issue has been resolved in 0.4 by blacklisting most 'system' paths. 
Actually
some of your suggestions are already implemented, maybe not the way you 
hoped... but
still. ;-)

To answer some of your questions, even though they may not apply for the new 
version
anymore. Forcing a rescan on 0.3 was just a matter of restarting the app.

0.4 includes a 'refresh' button and it also automatically refreshes when 
folders are
added or fonts are installed or removed. That said this function is a little 
flaky at
this point, because of what it's attempting to do, and restarting the app may 
still
be necessary if it fails to update. 

I'll make sure to note this in the README and Changelog.

I've also added an 'Autoscan' feature which crawls a few common directories and 
adds
any folders containing font files ('.ttf', '.ttc', '.otf', '.pfb', '.pfa', 
'.pfm',
'.afm', '.bdf').

At this point, the included directories in the scan are:

/home/<your user>
/media
/mnt
/opt

I think those are sensible defaults.

If you think other directories belong in that list, please list them and why.

And thanks again for taking the time to file these issues.

Original comment by JerryCasiano on 21 Sep 2009 at 8:05

GoogleCodeExporter commented 9 years ago
Make the scan list configurable, so I can add '/' to it.

If something could take a long time, simply pop up a Yes/No dialog before 
starting
the search: "Top-level directory searches can take a long time.  Are you sure 
you
want to continue?"  You should also display an "Abort" button during the scan, 
so
impatient users may move on.

Now, if you want to be REALLY smart, you would try to avoid the need to do your 
own
directory scan, by checking to see if a current filesystem scan ALREADY EXISTS! 
Common apps that do filesystem scans include updatedb(locate) and Beagle.

For locate, simply run 'locate -S' and grab the database from the first line of
output, then check the date on that file.  If it exists and is 'new enough' 
(less
than a week or so), use 'locate', and you will have full-disk search results 
within
seconds.

I run updatedb as a cron job every night, so I always have a current scan 
available.
 More Linux distros provide the cron job ready to go, but leave it off by default.  

It is also trivially easy to run 'updatedb' manually.  It uses an optimized
filesystem scan that should be much faster than a simple directory tree search.

To find all font files, just do the following:

   locate -i --regex "\.ttf$" "\.ttc$" "\.otf$" "\.pfb$" "\.pfa$" "\.pfm$" "\.afm$"
"\.bdf$"

Note the use of "-i": I have many font files that have the suffix in CAPS.  
Remember,
the font system itself doesn't care about the case of the font file names.

If the user has no filesystem scan available, then you may want to recommend 
that the
user runs 'updatedb', and only do the scan yourself if the users refuses.

On my system, a full scan of over 500 GB takes under 5 minutes.  And I'm still
running ext3 (ext4 and btrfs may be faster).  The home systems people are 
buying now
are probably faster.

How long to you expect font-manager to exist?  Allow for future systems to be 
much
faster than what you have on your desk now.

Original comment by FlyM...@gmail.com on 22 Sep 2009 at 12:10

GoogleCodeExporter commented 9 years ago
I'll try to go in order here.

I'm sorry, but I won't be adding system paths ( paths where users probably 
shouldn't
be keeping personal files to begin with ), besides I think the current setup 
will be
fine for most users. Users with more specific needs are very likely to know 
exactly
how to accomplish it themselves.

Using system calls is something I would like to avoid, as much as possible. 
Using system calls that may require root priviledges ( updatedb ) is out of the
question, as is asking the user to open a terminal for any reason.

The scan is already case insensitive and it only runs when the user requests it,
which I expect won't be very often for most users. This is one reason I haven't
invested too much time into it, but we'll see...

Your last suggestion is interesting, and probably what Microsoft was thinking 
as they
coded Vista. I'm actually a fan of the opposite point of view, meaning that you 
try
to make something run well on older hardware, knowing that newer hardware will 
not
only have no problems with it but run it much faster. If I had anything older 
than a
dual core, I would be testing on that. ;-)

Original comment by JerryCasiano on 22 Sep 2009 at 3:06

GoogleCodeExporter commented 9 years ago
You may have your logic backwards:  Sure, software should perform well on slow
systems, though such performance may require making it possible to bypass "the 
big
stuff" when needed.  Your take on it is that software should be "limited" just
because it *may* be run on a slow system.  That's not correct.

Just because something MAY take a long time, doesn't mean it should be 
outlawed. 
Instead, give the user a warning (popup) and a way out (abort button).  Such a 
remedy
will permit your app to work well on all systems, both now and in the future.

There are many situations where your "selective paths" solution will fail, and 
slow
scans WILL occur, such as when the filesystem itself is slow, a common 
situation for
external mounts.  Even fast systems could benefit from an Abort button, no 
matter
what default search path set you choose.  In some situations, WHAT you scan
(filesystem type) matters more than WHERE you scan.

For example, network mounts.  Even on an otherwise fast system, a scan could 
take
forever if the network is congested or the fileserver is loaded.  The user 
should be
able to include or exclude such mounts, no matter where they are, even if they 
fall
within your default path list.

Another example would be old external USB 1.1 drives.  I still have such a 
drive I
use only for streaming MP3s, since that's about all it is fast enough to do.  
Others
may put their font libraries on such drives.

Then there are font collections on CD or DVD.  Worse yet would be a mount of an 
ftp
site via FTPFS, or any other Internet mount.  But sometimes, you really need to 
do
such things, and I've done them for fonts.

The only way to protect the user from slowdowns of all kinds will be for your 
app to
alert the user when the scan is about to cross filesystem boundaries, and to
continuously monitor the scan rate.  Is this your plan?

A user could be surprised by a slow scan no matter how hard you try to prevent 
it. 
Clearly, a general means is needed to detect and handle the "OMG, this scan is 
slow!"
situation.  This cannot be avoided by hard-wiring the scan defaults.

The user needs a way to determine if a scan is going "too slowly".  The most 
generic
way to do this is to provide a progress bar and a live estimate of the time 
required
to complete the scan.  Unfortunately, it is not always fast or easy to 
determine the
length of the progress bar (how many directories will be scanned) before 
actually
scanning them.  There are many tricks available to help with this, but most of 
them
need a ton of memory, and can cause a slowdown if swap is active.

During a scan, it may be useful to show the current directory being scanned, 
and the
name of the last font added from that directory.  This would permit the user to 
see
where a slowdown is occurring, even without a progress bar.  Together with an 
abort
button, a user could quickly tailor the scan to cover as much as possible as 
fast as
possible.

If you truly want to protect the user from long scan times, I've shared two 
ways you
could do it.  If you aren't *that* interested (I wouldn't be), then provide a 
README
and an Abort button, and let the users do whatever they think is necessary.

Sensible defaults are great, so long as they can be easily modified.

Bonus thoughts:
1.  Only the FIRST "deep" scan need ever be slow.  Rescans can be optimized to 
trim
larger sub-trees that contain no fonts.
2.  If you track the time for the first deep scan, you can use that as a guess 
for
the next scan, and provide a progress bar.
3.  Do "standard" scans in the foreground, and let "deep" scans run in the 
background.
4.  Use multiple threads for all scans, which will let the disk driver and the 
disk
controller optimize disk access (*always* faster than a single-threaded 
recursive
descent scan, with a minimal memory penalty and no CPU penalty).

Please don't think I'm overly criticizing!  Your font-manager app falls under 
the
"Insanely Great" category, and has already been a HUGE benefit to me.  I'm just
trying to push it a little bit further to cover more of my font management 
needs.

Original comment by FlyM...@gmail.com on 22 Sep 2009 at 6:15

GoogleCodeExporter commented 9 years ago
I don't mind criticism. :-)

I don't think in terms of limiting this application because of slower machines, 
after
all it's only intended to provide basic font management, it's not like I'm 
working on
some revolutionary application here that can actually make use of the latest and
greatest tech.

You raise some valid points and my intention was not to dismiss them, but my not
wanting to allow configuration for this feature at this point is not related to
possible slowdowns in any way, it's directly related to the amount of time and 
effort
I'm willing to expend on this feature right now. Later it will depend on how 
many
users have any use for it, the percentage of users that have problems with it 
and the
problems they actually encounter, that's yet to be determined. 

At this point I haven't even implemented any way to show progress or abort the
operation... though I considered it.. :-/ 

There's too many core functions which I'm not happy with yet that need more
attention, truth is, huge chunks of code will need to be re-written, some of 
them in
C, before this app ever sees a 1.0 release. I'm hoping with time there will be 
more
interest in contributing.

Your point about network mounts and the like, is a good point, these if present 
will
no doubt cause MAJOR slowdown, the fact that there's no configuration possible, 
no
way to abort the operation without killing the app and no progress only 
aggravates this. 

Fortunately, users with such setups can just manually add folders.

In any case, this is the beauty of Open Source, if a feature is important to 
someone,
and they are able to improve it, they can grab the source, hack on it all they 
want
and provide a patch. So when I say, I'm not going to do that, it just means 
right now
and it doesn't necessarily mean I wouldn't accept a patch that does do it. ;-)

But like I said, not only do I not mind criticism but I very much appreciate 
posts
like yours which not only provide good ideas but also tend to bring up details 
which
are very easy to overlook.

Original comment by JerryCasiano on 22 Sep 2009 at 8:42

GoogleCodeExporter commented 9 years ago

Original comment by JerryCasiano on 24 Sep 2009 at 11:06