Deployment should go after Docker image pushes - Githubissues

hemilabs / blockscout

Blockchain explorer for Ethereum based network and a tool for inspecting and analyzing EVM based blockchains.

http://docs.blockscout.com

GNU General Public License v3.0

2 stars 1 forks source link

Deployment should go after Docker image pushes #16

Open gabmontes opened 3 months ago

gabmontes commented 3 months ago

@dhidalgX it looks like the deployment action defined in blockscout_deployment.yml may run before the Docker images are build and pushed to the DockerHub repository (see publish-docker-image-every-push.yml).

dhidalgX commented 3 months ago

@gabmontes we'll do the necessary fixes... going to remove on: push from the deployment action: https://github.com/hemilabs/blockscout/blob/85c0a398fabf94af2dddfb6638d718e134c7a649/.github/workflows/blockscout_deployment.yml#L5

dhidalgX commented 3 months ago

Also i've noticed some issues while generating the backups for postgres DB's,

Volumes are being created under user: 2000:2000 that means they are not being picked up by the current back up solution: https://github.com/hemilabs/blockscout/blob/85c0a398fabf94af2dddfb6638d718e134c7a649/docker-compose/services/stats.yml#L16

We'll need to update the back up action to pick up this volumes...

dhidalgX commented 3 months ago

pg_dump and pg_restoreto back up/restore postgress_db is being handle manually... We'll automate this process.

dhidalgX commented 3 months ago

During the last update we noticed some logs regarding PostgresSQL version upgrade from 14 to 15...

2024-06-04 23:25:20.738 UTC [1] FATAL:  database files are incompatible with server
2024-06-04 23:25:20.738 UTC [1] DETAIL:  The data directory was initialized by PostgreSQL version 14, which is not compatible with this version 15.7 (Debian 15.7-1.pgdg120+1).
chmod: changing permissions of '/var/run/postgresql': Operation not permitted

After this changes and because of the points mentioned above... postgresDB was not able to restored completely.. we revert back to previous commit of blockscout and still indexing... once fully indexed we can try again to deploy the latest upgrade...

dhidalgX commented 3 months ago

@gabmontes @jcvernaleo here the workaround for latest block explorer deployment:

Since we are upgrading from postgres:14 to postgres:15:

We needed to build explorer from scratch this mean not using the github action:

run docker compose down --remove-orphans to stop everything from old setup, postgres:14
delete the old volumes to avoid permission denied from new deployment postgres:15
clone/pull latest commit.
docker compose up -d with new postgres:15
restore db's from pg_dump.

Note:

I think the issue here is... trying to overwrite the actual deployment .... which in this case delivers errors from new postgres:15 with access denied to database volumes and database files incompatible.
Need to include this type of deployments in the action (Upgrades)
We need to revisit the actual backup process, and automate some of the tasks Backup Status:

Backup	Status	Detail
Volumes/Config/Repo	Automated	During deployment through Github action
Instance backup/Snapshot	Automated	Through Backup Schedule in VULTR
Postgress DB	Manual	Using pg_dump (We need to automate this with a cron or something else... )