search

nqxcode / laravel-lucene-search

Laravel 4.2, 5.* package for full-text search over Eloquent models based on ZF2 Lucene.
73 stars 28 forks source link

Getting errors while running search:rebuild #13

Closed teeli closed 9 years ago

teeli commented 9 years ago

I'm getting a bunch of errors running search:rebuild on Laravel 4.2 and laravel-lucene-search 1.1.3

Any ideas what could be causing this and how to fix it?

/var/www/html$ php artisan search:rebuild

  [ZendSearch\Lucene\Exception\RuntimeException]                                  
  File '/var/www/html/app/storage/lucene-search/index/_25j.cfs' is not readable.  

  [ZendSearch\Lucene\Exception\InvalidArgumentException]                          
  File '/var/www/html/app/storage/lucene-search/index/_25j.cfs' is not readable.  

search:rebuild

PHP Fatal error:  Uncaught exception 'ZendSearch\Lucene\Exception\InvalidArgumentException' with message 'File '/var/www/html/app/storage/lucene-search/index/_25j.cfs' is not readable.' in /var/www/html/vendor/nqxcode/zendsearch/library/ZendSearch/Lucene/Storage/File/Filesystem.php:45
Stack trace:
#0 /var/www/html/vendor/nqxcode/zendsearch/library/ZendSearch/Lucene/Storage/Directory/Filesystem.php(348): ZendSearch\Lucene\Storage\File\Filesystem->__construct('/var/www/html/a...')
#1 /var/www/html/vendor/nqxcode/zendsearch/library/ZendSearch/Lucene/Index/SegmentInfo.php(267): ZendSearch\Lucene\Storage\Directory\Filesystem->getFileObject('_25j.cfs')
#2 /var/www/html/vendor/nqxcode/zendsearch/library/ZendSearch/Lucene/Index/Writer.php(513): ZendSearch\Lucene\Index\SegmentInfo->__construct(Object(ZendSearch\Lucene\Storage\Directory\Filesystem), '_25j', 1, 1, NULL, 1, true)
#3 /var/www/html/vendor/nqxcode/zendsearch/library/ZendSearch/Lucene/Index/Writer.php(747): ZendSearch\Lucene\Index\Writer->_updateSegments()
#4 /var/www/html/ve in /var/www/html/vendor/nqxcode/zendsearch/library/ZendSearch/Lucene/Index/Writer.php on line 590

and occasianlly this:


/var/www/html$ php artisan search:rebuild

  [ZendSearch\Lucene\Exception\RuntimeException]  
  Can't delete file:                              

search:rebuild
nqxcode commented 9 years ago

I assume that this problem arises because files of search index are created and changed by different users, for example by "http-server user" (www-data) and by "user in terminal".

Possible solution: make available to record the storage folder (or storage/lucene-search) for all users. Run the following command to make it: sudo chmod a+w -R /var/www/html/app/storage

teeli commented 9 years ago

That might be one reason, but all the directories were 777 and files 666. Also it happened while running as root.

I was wondering if it could have something to do with the fact that I have 1 master server, that does the rebuilding, and shares the index over NFS to 6 other load balanced servers. And if it is, do you have any suggestions how to handle multiple servers?

nqxcode commented 9 years ago

It is a problem arises in case of multiple "non-locking access" to files of search index.

The kernel of search (ZendSearch) used flock() for "locking access":

// ZendSearch\Lucene\Storage\File\Filesystem.php

class Filesystem extends AbstractFile
{
    // ...

    public function lock($lockType, $nonBlockingLock = false)
    {
        if ($nonBlockingLock) {
            return flock($this->_fileHandle, $lockType | LOCK_NB);
        } else {
            return flock($this->_fileHandle, $lockType);
        }
    }

    // ...
}

But flock() will NOT work on NFS (see page: http://us3.php.net/manual/en/function.flock.php#107939)

There are some solutions:

  1. True solution. Fork "ZendSearch" and rewrite implementation of lock() in "ZendSearch\Lucene\Storage\File\Filesystem.php". Use this fork in "laravel-lucene-search". 2.Very brute solution. Create own "ZendSearch\Lucene\Storage\File\Filesystem" class with alternate implementation of lock() which will boot instead of original.
teeli commented 9 years ago

Thanks. That would explain it. I'll have to try a workaround for it.