scanner wrongly detects some symlinks as directories

                                                                                   [          ]

Issue migrated (2015-06-05) from old issue tracker http://mirrorbrain.org/issues/issue141

Title    scanner wrongly detects some symlinks as directories
 Priority   bug           Status      resolved
Superseder              Nosy List     dagobert, poeml
Assigned To poeml        Keywords     scanner

                            Files
File name            Uploaded              Type     Edit Remove

scanner.patch poeml, 2014-02-01.23:55:06 text/x-patch

msg504 (view) Author: dagobert Date: 2014-02-01.17:07:42

Some symlinks are wrongly detected as directories during scanning, e.g.

mirror-test# rsync rsync://mirror.opencsw.org/opencsw-full/beanie/i386/5.11/| less drwxr-xr-x 3,773 2014/02/01 15:44:00 . lrwxrwxrwx 65 2013/01/10 18:18:49 389_admin-1.1.30,REV=2013.01.07- SunOS5.10-i386-CSW.pkg.gz ...

mirror-test# mb scan ircam -v | less Sat Feb 1 18:07:22 2014 ircam: starting Sat Feb 1 18:07:24 2014 ircam: total files before scan: 76622 ircam: rsync excludes: */.~tmp~/ /.~tmp~/ ... ircam: rsync dir: 755 6 Thu Jan 10 15:58:46 2013 beanie/sparc ircam: rsync dir: 755 6 Thu Jan 10 15:58:38 2013 beanie/i386 ircam: rsync dir: 755 3757 Sat Feb 1 09:44:33 2014 beanie/sparc/5.11 ircam: rsync dir: 755 3501 Sat Feb 1 09:44:14 2014 beanie/sparc/5.9 ircam: rsync dir: 755 3756 Sat Feb 1 09:44:23 2014 beanie/sparc/5.10 ircam: rsync dir: 755 5 Sat Feb 1 09:44:05 2014 beanie/sparc/5.8 ircam: rsync dir: 777 63 Thu Jan 10 18:18:52 2013 beanie/sparc/5.11/rb_xapian-1.2.7,REV=2011.11.07-SunOS5.9-sparc-CSW.pkg.gz ...

msg508 (view) Author: poeml Date: 2014-02-01.23:20:50

Maybe only the logging is broken. Trying to reproduce the issue, I find that the scanner doesn't enter the symlinks into the database and it doesn't try to follow the "directories".

There are 3771 files in the directory, and only 24 of those are not symlinks. But the scanner finds them correctly. It doesn't try to follow false directories and is finishes scanning in less than 1 second:

poeml@doozer2 ~ % rsync rsync://mirror.opencsw.org/opencsw-full/beanie/i386/5.11/ | grep -v lrwxrwxrwx | wc -l 25 poeml@doozer2 ~ % mb scan csw Sun Feb 2 00:17:04 2014 opencsw: starting Sun Feb 2 00:17:04 2014 opencsw: total files before scan: 24 Sun Feb 2 00:17:04 2014 opencsw: scanned 24 files (37/s) in 0s Sun Feb 2 00:17:04 2014 opencsw: files to be purged: 0 Sun Feb 2 00:17:04 2014 opencsw: total files after scan: 24 (delta: 0) Sun Feb 2 00:17:04 2014 opencsw: purged old files in 0s. Sun Feb 2 00:17:04 2014 opencsw: done. Completed in 0 seconds poeml@doozer2 ~ %

msg509 (view) Author: poeml Date: 2014-02-01.23:55:06

proposed patch

msg510 (view) Author: dagobert Date: 2014-02-02.03:34:18

The patch seems to work, thanks!

msg511 (view) Author: poeml Date: 2014-02-02.11:57:35

fixed in r8357. Thanks!

History
         Date           User   Action             Args
2014-02-02 11:57:35 poeml    set    status: chatting -> resolved
                                      messages: + msg511
2014-02-02 03:34:18 dagobert set    messages: + msg510
2014-02-01 23:55:06 poeml    set    files: + scanner.patch
                                      messages: + msg509
                                      status: unread -> chatting
                                      assignedto: poeml
2014-02-01 23:20:50 poeml    set    messages: + msg508
                                      keyword: + scanner
                                      nosy: + poeml
2014-02-01 17:07:42 dagobert create

poeml / mirrorbrain