soundscape-community / authoring-tool

MIT License
3 stars 1 forks source link

Move from Azure services to self-hosted Docker containers #4

Open steinbro opened 11 months ago

steinbro commented 11 months ago

We'd like to take the authoring tool, which historically ran on Azure services at Microsoft, and instead run them on the same Docker-based server infrastructure we use for the Soundscape tile server. This exercise will involve translating the setup instructions in the README file into instructions Docker can understand, as well as replacing any Azure-specific functionality. (Some Azure services, like their Maps API, we probably want to keep -- this is more about managed infrastructure like databases and storage.)

Docker containers are basically lightweight virtual machines. A Dockerfile defines the set of commands to run to go from a fresh OS installation to a running server. A docker-compose.yml file describes a list of Docker containers, and how they should be configured to talk to one another. Both Docker and docker-compose are software you can install and run on your local machine for development.

The Docker files we have for the Soundscape tile server should be a good reference point to start out. For example, I expect the Dockerfile for the backend will look fairly similar to our Dockerfile.tilesrv, since it is taking a base OS image, installing system and Python packages, then running a server. There are other examples online for deploying a generic Node server, which we'll need for the frontend

Our docker-compose.yml file also has some similarities to what we'll need. It's spinning up a Postgres container, a Soundscape tile server container, and a Soundscape ingest server container. For the authoring tool I expect we'll wind up having containers like so:

  1. Database (just a postgres image)
  2. Blob storage (perhaps a Ceph image)
  3. Backend (from a new Dockerfile you create)
  4. Frontend (from a new Dockerfile you create) (edit: @sheldonej clarified the frontend is static, so this would just be an additional command to run when creating the backend image, no need for a separate container)
RDMurray commented 11 months ago

I think Ceph is very heavy wait, and it is a lot more than object storage. other options are:

steinbro commented 11 months ago

Thanks for pointing that out -- I only know Ceph in the abstract, as having an S3-compatible API you can self-host. But, as Azure Blob Storage is itself not S3-compatible, Ceph wouldn't be a drop-in replacement, anyway. And, if we want to use something like Azure Blob Storage for backups and the like, maybe there's no need to swap it out at all.

I realize this task is mixing two separate goals: containerizing the authoring tool, and removing cloud dependencies. The first step is to just make Dockerfiles for the frontend and backend, and a docker-compose.yml file that ties them together. Once that's in place, we can then go through the individual underlying dependencies (Postgres, Blob Storage, authentication, etc.) and see what we can swap out.