goat-community / goat

This is the home of Geo Open Accessibility Tool (GOAT)
GNU General Public License v3.0
89 stars 47 forks source link

Deploy API and Client to K8s using TravisCI #291

Closed tareksamni closed 4 years ago

tareksamni commented 4 years ago

@EPajares I'm missing env vars needed for the client. Could you please send it to me? We can merge this when it's approved. I've commented out the auto deployment of client till I get the env vars list from you 👍

majkshkurti commented 4 years ago

@tareksamni These are the env vars for the client:

For geoserver and api endpoints we excpect to be accessible as {baseUrl}/api and /geoserver so for now we don't have to add those as env vars. Maybe we can change this later. I tried to do the same thing for mapproxy but their docker image exposes the service without a context ex. /mapproxy, and i was forced to expose it to a different port so thats why i added it in the .env variable. Thanks.

tareksamni commented 4 years ago

@majkshkurti Should I deploy geoserver as well to the k8s cluster?

majkshkurti commented 4 years ago

@tareksamni Yes. In fact we need to deploy all the services that are in the docker-compose.yaml file to k8s cluster except "traefik" that we don't need in production since we are going to use a ingress LB, right ?

majkshkurti commented 4 years ago

@tareksamni I also want to say that geoserver, print and mapproxy services have some initial and persistant data which is mounted when the container is started. You can have a look at the docker-compose to check the corresponding folders paths. Although we can discuss how to manage this initial data since the it might be different. Do we need to store it somewhere else ?

tareksamni commented 4 years ago

That docker-compose file is definitely helping me to understand how these services are interconnected. Thanks for mentioning it.

Since the data is already in the git repo. Should we just build new docker images inheriting form the original ones + adding the data from the git repo to it?

majkshkurti commented 4 years ago

I am not sure. I have to ask @EPajares because the data might have some information we don't want to expose it in the future. Before we were just copying it manually or clone the git repo to the server and run the docker-compose directly there. What do you think is the best practice for this case ?

tareksamni commented 4 years ago

I need to understand how this data is structured, how frequent it's being changed and how confidential is it. So that I can decide what way to implement/integrate it.

majkshkurti commented 4 years ago

The data is structured in different folders as it is now in the git repo. Geoserver has an UI so the files will be generated directly into where the data is stored, so we aren't changing it manually during development. We have to think here only for the init storage.
DB .SQL files could change quite often, as it will be further development in those files. Print and Mapproxy data will not change often. Regarding the confidentiality, i would say that an external storage like (ex S3 Bucket, or Digital Ocean equivalent) protected with an Access key will be sufficient.

EPajares commented 4 years ago

Concerning the database: As Majk said the folders in the database folder (https://github.com/goat-community/goat/tree/development/app/database) can change very frequently. I do think the folders data_preparation, database_functions, scripts and the setup_goat.py can be fetched from the related git repo. The data folder and the config files could be confidential therefore they should be saved somewhere else.

If all these files are put together we can create the database with tables etc. on the server.