OpenSimSearch add-on module for Open Simulator
Requirements
The webserver needs PHP support with support for CURL and XMLRPC. NOTE: Not all webservers may have CURL and XMLRPC support for PHP.
About the files
README - The file you are currently reading
bin/OpenSimSearch.Modules.dll - A pre-compiled module you can use
OpenSimSearch/ - Source code for OpenSim search module
webroot/*.php - Files to install on webserver
webroot/sql/ossearch.sql - This will create the needed database tables webroot/sql/update-*.sql - Files used to update an older installation
webroot/wiredux/ - Files to add to an installation of wiredux
How it works
If you are looking for a detailed explanation of how the add-on search system works you should study the contents of the files which accompany this README file. What follows is a general overview of the search package to give you an idea of how the pieces fit together in case you have any problems.
There are three main components to OpenSimSearch which are a database, a DLL file, and several PHP files.
Most of the tables in the search database contain details on items which can appear in the results of a search. The exception is the hostsregister table. The hostsregister table contains the list of OpenSim instances which will be checked periodically for information about searchable items.
When an OpenSim instance starts it accesses the register.php script (using the URL of the data_services entry in OpenSim.ini) to record the host address and port of the instance in the hostsregister table of the database. The host and port entries are used by parser.php to retrieve the data to be indexed from the running instances.
Take note of the last part of the previous paragraph where it stated that parser.php is used to index the data from an OpenSim instance and not from a region. The parser retrieves data from an OpenSim instance. If the instance is running a single region the data for a single region will be updated. If the instance is running multiple regions the data for all regions hosted by that instance will be indexed.
OpenSim instances use the data snapshot module to create an XML based record of all searchable items in the regions they host. The XML record is retrieved by parser.php using a URL created from the host, port and the query string "?method=collector" (eg. http://127.0.0.1:9001/?method=collector). The parser.php file will get one host/port pair from the hostregister table each time it is called. It will parse the XML data from the OpenSim instance and save the data to the various tables of the search database.
The query.php file is the heart of the search process. It receives an XML message from the DLL based on the search request originating in a viewer. The XML message is parsed to determine the type of search being performed, SQL queries are built and executed to retrieve the data from the database tables, and finally, the results are sent to the DLL file as an XML message for any final processing before being passed along to the viewer for display.
Compiling the module
The easiest way to create the DLL file needed by OpenSim is to add the OpenSimSearch C# source file to the source tree of OpenSim so it will be compiled at the same time that OpenSim is compiled.
Copy the OpenSimSearch directory in to the addon-modules directory of your OpenSim source tree. After you have done that, compile OpenSim in the usual way (runprebuild and nant) to create the DLL file. When nant finishes, you will have an OpenSimSearch.Modules.dll file in the main bin directory of your OpenSim source tree along with a matching .pdb (or .mdb) file. These two files will be copied to the bin directory of your OpenSim instances during the installation or update steps.
First time installation
The first installation step is the creation of a database that will hold the data to be searched. If you have already installed the add-on profile module you should use the osprofile database as the search and profile modules share some tables in common.
If you don't have the osprofile module installed you will need to create a new database that will hold the search tables. Once you have know the name of the database you will be using for the search data you can use ossearch.sql (located in the webroot/sql directory) to create the required tables in the database. The name of this database will be needed in a later step when you configure one of the PHP files.
Copy the PHP files (located in the webroot directory) to your webserver. Remember to set the ownership and permissions on the files so the webserver may access the files. Use a text editor to open databaseinfo.php and enter the name or IP address of the database server, the name of the database, and the user name and password required to access the database.
The next part of the installation process is to set up and configure your OpenSim instances.
Copy the two OpenSimSearch.Modules files created during the compile steps (above) to the bin directory of each of your OpenSim instances. The next part of the installation process requires some changes to the OpenSim.ini in all of your OpenSim instances.
Add the following lines to all OpenSim.ini files:
[Search] Module = "OpenSimSearch"; //This line is optional and shows the default. ;/////////////////////////////////////////////////////////////////////////// ;// The SearchURL is important. It must be correct or search won't work. // ;// // ;// Change the URL to point to the query.php file on your own web server // SearchURL = "http://192.168.0.1/query.php" ;// // ;///////////////////////////////////////////////////////////////////////////
NOTE: You do not need to indent the above lines when adding them to your own OpenSim.ini files. The lines were indented to offset them from the rest of the text in this file.
Uncomment and edit the SearchURL line so it contains the correct address for the webserver and needed path to where the query.php file was installed on your webserver.
The last changes to be made to the OpenSim.ini file are in the [DataSnapshot] section. Change index_sims to true. You can have data_exposure set to all but it is better to leave it on minimum so users can control what items can appear in search results by using the "Show In Search" checkboxes. You can leave default_snapshot_period commented out or you can uncomment it and set it to whatever value you wish. The section on optimizing the configuration will help you to decide an appropriate value for this setting.
The last change for OpenSim.ini is the setting for data_services. This line must be uncommented and contain a valid URL for your webserver and the path to the register.php file which you installed earlier. If you do not enter a valid URL the OS instance will not get listed in the hostsregister table and search data for the regions hosted by the OS instance will not be recorded or updated. After entering the URL, exit and save the updated OpenSim.ini file.
Updating an existing installation
Updating an existing installation of ossearch is just a matter of copying a few files to the same places where you had previously installed the files.
Copy all of the PHP files (located in the webroot directory) EXCEPT for databaseinfo.php to the directory on your webserver where you place the previous copies. If you also copy databaseinfo.php when copying the other PHP files you will have to edit databaseinfo.php and reset the information used to connect to the database.
Copy the two OpenSimSearch.Modules files created during compilation to the bin directory of each of your OpenSim instances.
Finally, execute the ossearch-migrations.sql file located in the webroot/sql directory. This step is very important to make certain your database tables are up-to-date.
Configuration
With everything in place, the next step is to set up a task on your webserver (or some other machine) which will invoke the URL to the parser.php file on your webserver on a regular basis to ensure the contents of the database are kept up-to-date. For a machine running Linux you can set up a cron job. For a machine running Windows you can use Windows Scheduler.
See the section on optimizing the configuration to help you decide how often the parser.php should be run.
Optimizing the configuration
When you change what items are to be found in search by clicking the checkbox "Show In Search" or by removing an item that was set to show in search results there is a delay before the change is reflected in the database tables. There are four main factors that affect the length of this delay. They are: the number of OpenSim instances, the value of default_snapshot_period used by the DataSnapshot module, the time between calls to parser.php, and the number of OpenSim instances processed each time parser.php is run. You can't easily control the number of instances but you can control the other factors.
To explain how the factors affect the delay take a grid with 100 regions. If you have one region per instance you have 100 instances. Using the default settings and calling parser once an hour it would take 100 hours, or more than four days, for changes to appear in search results. A more realistic setup is one where you have an average of 4 regions run by each OpenSim instance. This reduces the delay to 25 hours. This is a rather long delay for a relatively small number of regions and instances. The delay can be reduced substantially.
A simple way to reduce the delay is to run parser.php more frequently. If parser.php is run once every 15 minutes instead of once an hour (60 minutes) the delay is reduced by a factor of 4 from 25 hours to 6.25 hours. Much better but still a long delay. The delay can be reduced even further but to do so requires a change to the parser.php file.
Near the end of the parser.php file is a SQL query that ends with "LIMIT 0,1". It is the value "1" which is important. That value limits the number of OS instances that will be processed each time parser.php is run. If the value is changed from 1 to 3 then three OS instances will be processed on each run of the parser.php file. This reduces the delay by a factor of 3 to just over 2 hours. This is a much better delay than the original value of 25 hours.
For those of you who like math, the amount of delay can be expressed using the following simple formula: delay = # instances * time between runs of parser / limit value in parser
Three factors affecting the delay have been discussed but earlier it was stated that there are four factors. The fourth factor is the value of default_snapshot_period value located in the [DataSnapshot] section of the OpenSim.ini file. This setting is specified in seconds and controls how often the data used by parser.php will be updated. This setting doesn't have any direct impact on the delay between updates of the database but if it is set incorrecty it can efffectively increase the delay between database updates.
The example grid setup discussed earlier was adjusted to reduce the update delay to just over 2 hours (2 hours and 5 minutes to be more precise). If the value of default_snapshot_period is set to a value slightly greater than the calculated delay it would be possible for parser.php to be called twice before the data it retrieves would have been updated. This would turn a delay of two hours into a delay of four hours. The proper setting is one that is less than the delay calculated from the other three factors discussed earlier.
Given a delay of just over 2 hours, a good value for default_snapshot_period would be 7200 (2 hours expressed in seconds). By keeping the value of this setting close to, but less than, the delay between when parser.php is used to get the data for an instance will minimize any overhead imposed on the OpenSim instance when it creates the snapshot of searchable items while, at the same time, ensures the data will have been updated by the next time the parser.php is run to update the database.
A final comment about the setting for the time between runs of parser.php and the value in the limit statement in that file. Avoid running parser.php too frequently or setting the value in the LIMIT statement too high. Doing either can cause unnecessary overhead or high loads on the webserver used to run parser.php, or on the database server while it updates all the tables with the latest information from the OpenSim instances.
Additional Information
A few words about event listings and the events database table. Support is included for events but the event listings need to be created using an external webpage.
The category for an event is stored as a number. The numbers for the categories are as follows: 0 - Any (NOTE: Event information will show "Unspecified") 18- Discussion 19- Sports 20- Live Music 22- Commercial 23- Nightlife/Entertainment 24- Games/Contests 25- Pageants 26- Education 27- Arts and Culture 28- Charity/Support Groups 29- Miscellaneous
The dateUTC field in the database is a timestamp for the event in UTC time. The duration field is the amount of time the event will run in seconds.
The covercharge field is a boolean. Set it to 0 if there is no cover charge for the event. When covercharge is not 0, the amount is in the coveramount field. (It seems silly to require the boolean but this has been left in to avoid any compatability issues.)
The globalPos field is the location of the event as a global grid coordinate. The format is "x,y,z". where x and y are the grid X and Y positions (times 256) plus the X and Y offset within the region named by the simname field.
The eventflags field is 0 for a PG event, 1 for Mature, and 2 for Adult.