mainstream g3W site and publish outputs

[gubuntu]

Admire set up this site a while ago http://maps.kartoza.com:8082/en/map/waterberg/qdjango/1/

[NyakudyaA]

Admire set up this site a while ago http://maps.kartoza.com:8082/en/map/waterberg/qdjango/1/

@lucernae I fixed this up so it's now running again.

[NyakudyaA]

@lucernae Can you tackle this task this week?. The deadline for this is coming up this Friday

[NyakudyaA]

@lucernae We need the following

• [x] 1. Migrate to rancher 2
• [x] 2. Upgrade cluster to use the latest images
• [x] 3. configure SSL
• [ ] 4. strategy for sharing syncing data. We need to share qgis projects and some data. Do we use btsync?

[lucernae]

@NyakudyaA are we supposed to have specific domain name for this since you mention SSL?

[NyakudyaA]

@NyakudyaA are we supposed to have specific domain name for this since you mention SSL?

It can be under kartoza ie (https://maps.wbr.kartoza.com) for now unless @gubuntu has a domain for this already

[lucernae]

Since it seems everyone is blocked by this, I deployed the app with legacy recipes here in Rancher 2's shared cluster:

https://maps.wbr.kartoza.com

Progress

• [x] Use rancher 2 shared cluster
• [ ] Migrate the node?
• [x] Latest images (g3w-suite-dev:latest)
• [x] Data migrated
• [x] SSL setup letsencrypt
• [ ] Sync data strategy
• [ ] Develop the helm charts
• [x] Upstream changes

Comments

Regarding node migration

The old service were deployed in a shared node in maps.kartoza.com. We need to decide from these options:

• Migrate over maps.kartoza.com (with all of it's different service inside the node). Potentially take longer time to do if every service in the node are different software stack.
• Create a new node for WBR specifically. Cause extra costs (extra server)
• Let it sit in kartoza shared cluster. We don't care which node is hosting the service, the data and containers can reside in any node as scheduled by kubernetes. It's possible that it sits in a node we provision for STI, for example. Currently this is what happen and k8s schedule it. To a CX41 machine we provision for Zimbabwe SDI.

Regarding Helm charts

Currently I deployed it using ad-hoc deployment. Manual modification of docker-compose recipes in the maps.kartoza.com server. I will store this manifests for safe keeping in kartoza/devops. However proper better deployment would be to create a helm chart for g3w-suite then publish necessary changes for the docker images back to g3w-suite. Then for subsequent g3w-suite deployment, we can be confident that we can deploy it easily using helm chart.

Since it seems we need the service online ASAP, I deployed it using ad-hoc deployment. Then if permitted, make the helm charts. When the charts are ready, we swap the deployment (fairly trivial to do if the image version and data doesn't change).

Regarding Upstream Changes

The current g3w-suite recipes relies on development images and host bind mount. We need to suggest a better generic approach to them to avoid future breaking changes from the recipes and images. As what we already do in SPCGeoNode deployment.

Regarding data migration

I think we need to setup a session for this. It's easier to do it by using Rancher's abilities to share k8s environment. But I need to demonstrate it first. For now, we can lists are possible data to migrate. Here's what I found from just reading the recipes:

• django docker settings
• postgis db
• shared-volume (I'm not sure what it contains)

If you think that there are any other files unmentioned, then fill me in.

Regarding Data Sync

If this is for us, I think we can just use rsync or FileZilla in general, or VSCode. If this is for client, I'm not sure at the moment. I haven't tried btsync yet in k8s, there might be something we need to change regarding the network environment as btsync is peer to peer. Minio is also possible if you may consider it.

[gubuntu]

@NyakudyaA how are we doing here? As much as possible of our layers must be available via this portal this coming week. Also, the portal needs to be public i.e. anyone can see and interact with the map without having to register or log in.

Use of service endpoints and downloading should be free and easy, but should ideally require registration. Registration should if possible include a question about why the person wants the data

[NyakudyaA]

@NyakudyaA how are we doing here? As much as possible of our layers must be available via this portal this coming week. Also, the portal needs to be public i.e. anyone can see and interact with the map without having to register or log in.

Use of service endpoints and downloading should be free and easy, but should ideally require registration. Registration should if possible include a question about why the person wants the data

An update on this @zacharlie has been testing the workflow and has encountered some bugs. Some of them have been rectified by Rizky. I will be working on this tomorrow to try to degus the issues raised by Charlie and the login issue you have mentioned above

[lucernae]

Status update in this issue so we remember the state:

1. Created maps.wbr.kartoza.com and staging.maps.wbr.kartoza.com piggybacking in kartoza-shared cluster (no new node provisioned). Deployed in Rancher 2 with LetsEncrypt SSL
2. No helm charts created yet (not enough time yet). Manifests stored in kartoza/devops
3. No data sync strategy has been decided yet

[zacharlie]

Some site performance issues are outlined in #91, #92, & #93

[zacharlie]

Once performance issues have been addressed we will need to evaluate whether the current infrastructure is suitable for the platform needs. If we are intending to use the platform for large numbers of users or for marketing purposes we will likely need to increase resources to ensure that the service is highly performant.

[gubuntu]

@NyakudyaA @tharanathkartoza was all the above taken into account when doing the recent migration to R2/k8s?

Please close if all done