Search Crawlers on different web servers

[mikebigun]

Description of Problem

There are few issues reported to us related to search functionality. After troubleshooting it was detected that search crawlers are failing due to corrupted/missing files in a App_Data\Search folder. The errors are happening for the Evoq customers that are on web farm infrastructure setup. Apart of Site Crawler, there are also File Crawler and Url Crawler configured that uses lucene search engine API implemented on a Platform. It happened, that customer has crawlers configured to run on a different web servers. Because of that, files are getting modified and attempted to be scanned by different machines. It causes errors and search doesn't work as expected. This can be fixed by configuring all crawlers to be run on a same web server. Apart of that, to avoid same problems in a newer version, it is recommended to prevent admins to setup different machines for search crawlers.

Proposed Solution

Solution which is proposed here is to warn admin when he is attempting to setup crawlers like mentioned above. For that, it would be good to have a Host/Portal setting SchedulersToRunOnSameWebServer with a list of schedulers that we recommend to run on a same machine. On a scheduler configuration save, we will be checking if server that is going to be saved matches to the server of other schedulers in the list. If not, warning message will be displayed. This should not block admin to save it in other way, but supposed to notify only. To use this feature, we need this validation to be implemented in a core part.

Dear community, please let me know whether proposed solution makes sense and looks good to you to be implemented here?

Affected version

• [*] 10.0.0 nightly build
• [*] 9.4.1 nightly build
• [*] 9.4.0 release candidate
• [*] 9.3.2 latest supported release

[valadas]

I do not see a problem with that solution....