search

kartoza / kartoza-rancher-catalogue

This is our catalogue of Rancher recipes
GNU General Public License v3.0
19 stars 16 forks source link

btsync-data and btsync-media containers keep failing #16

Open dacodekid opened 6 years ago

dacodekid commented 6 years ago

Trying to help a (non technical) friend to get this software up & running. Within last 14 hrs period, I've tried to provision a brand new server and reinstall Geonode with QGIS Backend and GeoSAFE several times with inconsistent results (almost every time). But the only consistent issue was that both btsync-data and btsync-media containers keep failing regardless of my reinstalling efforts.

  1. When I try a container setup with plainVagrant and Ubuntu 16.04 box (installed Rancher with supported Docker), both inasafe containers also failed 99% of the time and the qgis-server gets downgraded whenever I restart the Vagrant box.

  2. Then I tried the same steps in a DigitalOcean droplet (tried/switched several times between Ubuntu & Debian). Most of the time got 502 Gateway Error from nginx but finally was able to make it run.

Screenshot from Local Vagrant machine image

Screenshot from DigitalOcean container image

You can see that in both instances, btsync containers failed to run (I had to stop them because they keep the CPU usage in around 100%).

Below is what Rancher's log shows for both containers.

image

Please note that if I restart the DigitalOcean droplet, then again... both inasafe instances start to fail and the qgis instance gets downgraded.

The software is running without those containers but not sure what they bring to the table (unfortunately I'm not a GEO / GIS person).

Would really appreciate if you could shed some light on the right path as I'm not sure whether it is a bug to report here or a SO question. If you need further log / screenshot and let me know where / how to collect them I would gladly provide them.

dacodekid commented 6 years ago

Here are the logs for bysync containers

Logs: geosafe-btsync-data-1 (Standard Errors)

3/4/2018 10:11:17 PMgroupadd: GID '0' already exists
3/4/2018 10:11:17 PMuseradd: user 'root' already exists
3/4/2018 10:11:38 PMgroupadd: GID '0' already exists
3/4/2018 10:11:38 PMuseradd: user 'root' already exists
3/4/2018 10:28:52 PMgroupadd: GID '0' already exists
3/4/2018 10:28:52 PMuseradd: user 'root' already exists
3/4/2018 10:29:05 PMgroupadd: GID '0' already exists
3/4/2018 10:29:05 PMuseradd: user 'root' already exists
3/4/2018 10:29:20 PMgroupadd: GID '0' already exists

Logs: geosafe-btsync-media-1 (Standard Errors)

3/4/2018 10:24:05 PMgroupadd: GID '0' already exists
3/4/2018 10:24:05 PMuseradd: user 'root' already exists
3/4/2018 10:24:20 PMgroupadd: GID '0' already exists
3/4/2018 10:24:20 PMuseradd: user 'root' already exists
3/4/2018 10:24:22 PMgroupadd: GID '0' already exists
3/4/2018 10:24:22 PMuseradd: user 'root' already exists
3/4/2018 10:24:35 PMgroupadd: GID '0' already exists
3/4/2018 10:24:35 PMuseradd: user 'root' already exists
3/4/2018 10:24:37 PMgroupadd: GID '0' already exists
3/4/2018 10:24:37 PMuseradd: user 'root' already exists
dacodekid commented 6 years ago

Thought to give a shot and upgrade them both with the SECRET KEY (it was empty) and the containers started working (really have no idea how/why though).

image

dacodekid commented 6 years ago

Finally figured out the local (Vagrant) installation issues and (sort of) fixed the btsync issue as well on both Local & Server (DO Droplet) environments. This is what I did.

  1. On Vagrant, instead of rolling out the Geosafe environment from Kartoza catalogue, I tried to pull each Docker images and noticed that inasafe container failed with no available space error regardless of tries. This is, I believe, is due to Vagrant box's default of size of 10GB and so Docker allocates 10GB for rolling out containers (in addition to my 4GB Swap memory space).

So instead of default Debian Box (9.3), I tried bento/debian-9.3 which creates 64GB base box. With this space, insafe container is happy and pulled without issues. Since DigitalOcean base droplet provides 30GB space, that would be a problem either.

  1. Looked into Kartoza's source code, I realized that neither btsync containers have any dependencies directly with the app and they merely provides backup for django media files - and without a SECRET KEY they kept on failing.

So I cloned this repo and removed both btsync containers from docker-compose.yml file and referenced my repo in Rancher's catalogue.

Now it takes only about 3-4 minutes top to orchestrate entire geosafe - Geonode with QGIS Backend and GeoSAFE installation. 👍

==============

However with provided default values (Django, Celery or INASAFE ??), we're facing this issue. I've tried this installation several times both in Local & DO Droplets with same issue - which I believe something to do with CONFIG Values.

Could anyone plz able to guide us what issue it could be?

timlinux commented 6 years ago

So short version:

If you do not use btsync (now called resilio sync), simply remove these containers from your stack.

Longer answer:

If you install Resilio sync and create two shares (one for media and one for dbdumps) and then insert those keys into the place provided when orchestrating your geonode.

The backups will then get synced to your desktop where you can back them up.

See https://www.resilio.com

Note: Resilio is not free software and may require the purchase of a license for non personal use.

In the future we will make other sync protocols e.g. drop box available.