Stack Verse Mapper

Index any and all references to Bible verses in Stack Exchange sites using the public data dumps and inject matches from this index into search results for each site using a user-script.

Support for Biblical Hermeneutics, Christianity, Judaism, Islam, History, and Skeptics is planned for the user-scipt out of the box, but you will be able to add other sites manually if you rebuild the index. Once the data-dump is processes and an index is generated, the index data will be saved locally by the browser so no third party site is involved.

Status

While not the first time such a feature has been discussed, the impetus to start this project was this meta post. Programming work has only just begun. The concept is being mapped out and organized into issues. Overall status can be tracked from the milestones page.

Contributing

Anyone with ideas is invited to participate in the issue tracker—adding how they think it should work or discussing implementation details.

Anyone that would like to contribute code is invited to fork this repo and extend it as they are able.

Usage

So far the system is non-functional. If, as a programmer, you would like to play with the backend pieces as they come along, read on…

Requirements

To use:

• Web browser with userscript manager (Greasemonkey for Firefox, Tampermorkey for Chrome, etc.)

To build, tinker, or regenerate the index:

(These are projected as some of this isn't implemented yet and actual dependencies will depend on who implements what. See the discussion in issue #5.)

• make, bash, curl, 7z, gzip, jq
• nodejs, npm

Installation & Usage

Eventually, the end user UI should just be a userscript or browser extention installation away.

Development Setup & Usage

Clone this repository (or your own fork of it if you plan to contribute).

git clone git@github.com:alerque/stack-verse-mapper.git

To get a quick and dirty sample of running a query, run

make demo

To download the data dumps and build indexes for all enabled sites:

make all

That will take a long time because it has to download all the relevant data dumps. The downloads will be preserved, so future index rebuilds will be faster.

If you'd like to build just one site, specify the site prefix (the part before the .stackexchange.com in the site's URL), e.g.:

make hermeneutics

Each site generated will produce a queryable index of posts with references in <site>-index.json.

Once indexes are generated, you can search from the command line by passing a site and a query to the search script:

./bin/demo.js hermeneutics 'Rev 5:1'