consolidate MWS on pine and make it configurable

[kohlhase]

spun out of #258:

I think we need a drupal-level module that configures the MWS service and sets it up on a machine.

Here are some pine instructions for MWS that we want to keep and build on.

runs on pine at http://mathhub.info:24367/.
# build
cd mws/build
make
cd ../..

# run
./mws/build/mwsd -I HARVEST-DIR -m 8081 -r &
./mws/build/restd -p 8082 -m 8081

There is also some MMT stuff that is relevant, but I do not quite understand how things work there.

api.web.SearchServer is the generic search interface that wraps aroung the MathWebSearch class.
It expects, e.g.,
SERVER/:search?theory=URI&format=pvs
with the query in the body.

and some more

"mws is running on pine, I restart it with "sudo systemctl restart mwsd_mws_oaf", the log file is in /var/log/mwsd_mws_oaf.log, the config is in /usr/local/etc/mws/config, the data directory is /var/lib/mws/mws_oaf2, it's supposed to run on port 24367. 

[kohlhase]

@a-hambasan and @Jazzpirate can help on this as well, they have played with MWS

[kohlhase]

The drupal module should probably only contain the config, where the MWS process is running how to restart, ... I believe the config which libraries are harvested and how, should be either

• in META-INF/MANIFEST.MF of the various archives: if it contains a directive it is harvested
• or in the site config .../MathHub/meta/inf/config/MathHub/config.mcf. For that we would need to have a build target for MWS harvesting.

I think I like the latter better. This makes the question of what gets indexed a question for the site admin, and not for the archive maintainer. In particular, different archives can be indexed or not on different MathHubs. There may still be a role for META-INF/MANIFEST.MF, it could say how things should be harvested.

[kohlhase]

@m-iancu do you have any advice on this?

[m-iancu]

@kohlhase One comment/question would be regarding MWS vs TEMA search -- the latter is a lot more difficult to set up and not sure how maintained. The second is whether its better to integrate directly with MMT to make use of the api.web.SearchServer class (but both config.mcf and MANIFEST.MF routes should allow for that so this should be fine).

Also, since this feels more instance-specific rather than library-specific I would suggest config.mcf rather than MANIFEST.MF -- then two different MathHub instances could serve the same library (e.g. urtheories or examples) but one might provide search and the other not -- or use TEMA instead of MWS, etc..)

[kohlhase]

thanks for raising the tema/mws question, I feel we will need (eventually) tema, since I want to (eventually) also serve GenCS and co.

And your comments confirms my suspicion that config.mcf is the way to go.