search

ericleasemorgan / reader

Distant Reader, a tool for using & understanding a corpus
GNU General Public License v2.0
20 stars 7 forks source link

Containerized release and use of LANL and Microsoft. #112

Open molikd opened 4 years ago

molikd commented 4 years ago

Our current plan in order to take advantage of all resources given to us is to auto-build #34 three containers (I think, need confirmation):

  1. Web Server/Job Submission Container (reader-ws)
  2. A Job Running Container (reader-j)
  3. Solr Index container (reader-s)

This means that for a given set of resources the web server will submit a job, which will spin up a container of the job container type, both will interact with Solr.

Solr meanwhile will have to be configured to be cross datacenter. #101

We will build two distant readers to start West and East, later we could add an India cluster. reader-ws will be deployed to Jetstream in East and Microsoft in West, reader-j will run on Slurm, Jetstream in East and LANL in West, Solr will run on Jetstream in East and Microsoft in West.

This means that for me, what I should do is use the future West implementation as the test bed for containerization. Maybe starting with a simple https://azure.microsoft.com/en-us/resources/templates/slurm/ Slurm implementation in the West.

ericleasemorgan commented 4 years ago

Dave, on your mark. Get set. Go.

Do you need anything from me?

molikd commented 4 years ago

Yes, are the software dependencies up-to-date on the README.md?

ericleasemorgan commented 4 years ago

Yes, they are up-to-date, but there is no need for Arivata. Okay?