ubiquity / ubiquibot

Putting the 'A' in 'DAO'
https://github.com/marketplace/ubiquibot
MIT License
16 stars 59 forks source link

DB backup #865

Closed rndquu closed 7 months ago

rndquu commented 9 months ago

We should periodically backup the bot's prod DB in case something goes wrong.

What should be done:

  1. Create a private github repository for DB backups
  2. Create a workflow that will dump (both schema and data) the bot's prod DB once a day (example)
  3. Try to recover the DB from a generated dump in order to check that it works
0x4007 commented 9 months ago

You can add my script to the yml file and it should work. The only modification required is that we need to stuff the database password on the interactive prompt.

It'll be beneficial to share it here but I'm posting from mobile.

rndquu commented 9 months ago

You can add my script to the yml file and it should work. The only modification required is that we need to stuff the database password on the interactive prompt.

It'll be beneficial to share it here but I'm posting from mobile.

What do you mean by "my script"? Could you share a link?

PhantomCracker commented 8 months ago

But @rndquu how can we test if we do not have schema and data? We can do a test with some mockup data maybe? Can you provide something?

0x4007 commented 8 months ago

From our conversation a couple of weeks ago

#!/bin/bash
CURRENT_TIMESTAMP=$(date +%s)

# pg_dump
docker \
    run \
    --rm \
    -it \
    -v \
    ~/Downloads:/backup \
    postgres:15.1 \
    pg_dump \
    -h \
    db.wfzpewmlyiozupulbuur.supabase.co \
    -p \
    5432 \
    -d \
    postgres \
    -U \
    postgres \
    -F \
    c \
    -f \
    /backup/"$CURRENT_TIMESTAMP".backup

# pg_restore
docker \
    run \
    --rm \
    -it \
    -v \
    ~/Downloads:/backup \
    postgres:15.1 \
    pg_restore \
    --no-owner \
    --no-acl \
    -F \
    c \
    -f \
    /backup/"$CURRENT_TIMESTAMP".sql \
    /backup/"$CURRENT_TIMESTAMP".backup

# diff
# diff \
# ~/Downloads/"$BEFORE_TIMESTAMP".sql \
# ~/Downloads/"$CURRENT_TIMESTAMP".sql \
# > \
# ~/Downloads/difference.diff
rndquu commented 8 months ago

From our conversation a couple of weeks ago

Using docker container for a single pg_dump command seems to be an overkill. I think it is simpler either to apt install postgresql (with pg_dump) in a github runner either to supabase db dump --db-url "$supabase_db_url" -f data.sql --data-only --use-copy (as described here).

rndquu commented 8 months ago

But @rndquu how can we test if we do not have schema and data? We can do a test with some mockup data maybe? Can you provide something?

You can test on your own supabase remote instance DB

PhantomCracker commented 8 months ago

Thank you @rndquu for all the provided information 👍

ubiquibot[bot] commented 7 months ago

@gitcoindev The time limit for this bounty is on Tue, 21 Nov 2023 09:45:21 UTC

ubiquibot[bot] commented 7 months ago

These linked pull requests are closed: #882

ubiquibot[bot] commented 7 months ago

@gitcoindev - Releasing the bounty back to dev pool because the allocated duration already ended! Last activity time: Thu Oct 19 2023 21:52:04 GMT+0000 (Coordinated Universal Time)

rndquu commented 7 months ago

@Keyrxng Could you self assign again?

gitcoindev commented 7 months ago

Hi @rndquu the bot confused me here -) this is for @Keyrxng right?

rndquu commented 7 months ago

Hi @rndquu the bot confused me here -) this is for @Keyrxng right?

Yes, sorry for bothering you

gitcoindev commented 7 months ago

Hi @rndquu the bot confused me here -) this is for @Keyrxng right?

Yes, sorry for bothering you

No worries, thank you for clarifying this!

Keyrxng commented 7 months ago

/start

ubiquibot[bot] commented 7 months ago

Skipping /start because it is disabled on this repo

Keyrxng commented 7 months ago

@rndquu unable to reassign atm mate

ubiquibot[bot] commented 7 months ago

@Keyrxng The time limit for this bounty is on Fri, 24 Nov 2023 03:17:47 UTC

rndquu commented 7 months ago

Closing this as completed because dumps are created fine https://github.com/ubiquity/ubiquibot-db-backups

ubiquibot[bot] commented 7 months ago

Permit generation skipped because bounty amount after penalty is 0

If you've enjoyed your experience in the DevPool, we'd appreciate your support. Follow Ubiquity on GitHub and star this repo. Your endorsement means the world to us and helps us grow!
We are excited to announce that the DevPool and UbiquiBot are now available to partners! Our ideal collaborators are globally distributed crypto-native organizations, who actively work on open source on GitHub, and excel in research & development. If you can introduce us to the repository maintainers in these types of companies, we have a special bonus in store for you!

ubiquibot[bot] commented 7 months ago

Task Creator Reward

rndquu: [ CLAIM 25.8 WXDAI ]

Keyrxng commented 7 months ago

Permit generation skipped because bounty amount after penalty is 0

If you've enjoyed your experience in the DevPool, we'd appreciate your support. Follow Ubiquity on GitHub and star this repo. Your endorsement means the world to us and helps us grow!
We are excited to announce that the DevPool and UbiquiBot are now available to partners! Our ideal collaborators are globally distributed crypto-native organizations, who actively work on open source on GitHub, and excel in research & development. If you can introduce us to the repository maintainers in these types of companies, we have a special bonus in store for you!

wtf? :))

rndquu commented 7 months ago

Permit generation skipped because bounty amount after penalty is 0

If you've enjoyed your experience in the DevPool, we'd appreciate your support. Follow Ubiquity on GitHub and star this repo. Your endorsement means the world to us and helps us grow!
We are excited to announce that the DevPool and UbiquiBot are now available to partners! Our ideal collaborators are globally distributed crypto-native organizations, who actively work on open source on GitHub, and excel in research & development. If you can introduce us to the repository maintainers in these types of companies, we have a special bonus in store for you!

@pavlovcik I guess we should send the reward manually

0x4007 commented 7 months ago

Permit generation skipped because bounty amount after penalty is 0

If you've enjoyed your experience in the DevPool, we'd appreciate your support. Follow Ubiquity on GitHub and star this repo. Your endorsement means the world to us and helps us grow!
We are excited to announce that the DevPool and UbiquiBot are now available to partners! Our ideal collaborators are globally distributed crypto-native organizations, who actively work on open source on GitHub, and excel in research & development. If you can introduce us to the repository maintainers in these types of companies, we have a special bonus in store for you!

Penalty system is not good in this implementation. Curious what issue was reopened that you were paid for.

0x4007 commented 7 months ago

/query @Keyrxng

ubiquibot[bot] commented 7 months ago

Skipping /query because it is disabled on this repo

0x4007 commented 7 months ago

I'll need to address this later then. Let me know what address you want me to send to @Keyrxng

Keyrxng commented 7 months ago

Curious what issue was reopened that you were paid for.

It can only have been when you reopened /ask I think

I claimed the faucet permit earlier today without issue in generating it so this is very strange