search

djmaze / resticker

Run automatic restic backups via a Docker container.
https://hub.docker.com/r/mazzolino/restic/
Apache License 2.0
511 stars 70 forks source link

First time compose initiation #126

Open PovilasID opened 2 years ago

PovilasID commented 2 years ago

Hey,

after running https://github.com/djmaze/resticker/blob/master/docker-compose.example.yml as not prune container launches before others and locks the connection stopping from making immediate connection. I suggest adding 

    depends_on:
      - backup

for prune and check containers Also each container creates their keys and config in the same folder will that not cause conflicts?

djmaze commented 2 years ago

@PovilasID I guess the current compose example makes no sense this way for real-world usage. I guess most people will run backup and prune on the same machine, but they don't have a need of running the prune immediately.

So I think we should rather change RUN_ON_STARTUP to false for the prune container in order to prevent this situation. What do you think?

And.. Sorry for the late response!

PovilasID commented 2 years ago

People do not need to run prune immediately, however if they don't they will never do 😆 I am talking here only about myself but I wanted to run prune immediately, so I have the backup system running and I would not have to return to it (hopefully never). I do not know what the ideal solution would be but depends waits for other to be initiated and that makes it the ideal for fire and forget situation even though it may not be as resource efficient.

djmaze commented 2 years ago

I don't think depends will work well in this case because the prune will be run when most probably the backup is still running. In order to test the prune correctly you should probably rather wait for the backup to finish and then run docker compose exec prune prune manually I guess.

PovilasID commented 2 years ago

Question: Can we merge pruning into the same container? Instead of having a separate container could be a command or an ENV var that would set pruning? My point is again to be able to 'set it and forget it'

djmaze commented 2 years ago

@PovilasID How would you imagine this to work in that case? Should the prune be run automatically after each backup?

PovilasID commented 2 years ago

You already have PRUNE_CRON maybe it could be dumped into the backup container environmentals and the same container would do two bot backups and pruning. Have not tested it. Would it work? Would it not be redundant?

djmaze commented 2 years ago

We could do that, but I do not see a real advantage. Then you would still need to set a suitable schedule for PRUNE_CRON. So apart from saving one container / go-cron process, there would be no win.

PovilasID commented 2 years ago

Other than naming that advantage there would also be the benefit of a little bit of 'idiotproofing' not that any user of this repo is one :D but reducing the complexity and making compose file runnable out of the box without any edits would be just nice

djmaze commented 2 years ago

Well, in order to do that we'd just need to set RUN_ON_STARTUP to false for the prune container :)

derickson2402 commented 2 years ago

Hey y'all just found this project and am trying it out! I'm using Storj.io S3 for the backend storage. I'm running into some wonky stuff and I'm hoping it can help you moving forward.

When starting the stack, all 3 containers try to init the bucket at the same time which fails. I have to manually go into the bucket and delete the config file and then start the stack again. I set the start order to backup -> prune -> check, which allowed the backup container to work but then the prune and check containers stopped working. I'll report back if I can find a fix.

ericvoshall commented 1 year ago

+1 to this, I am having issues with all three services (backup, prune, check) trying to simultaneously initialize a new repository, even with only one of the services having RUN_ON_STARTUP set to true.

Would it be possible to wait to check if a repository is initialized until the command actually runs (triggered either by the cron schedule or the RUN_ON_STARTUP flag)?

djmaze commented 1 year ago

Mhh.. That would be a solution, but not a good one IMO. I think it's quite helpful that the check is done on container start already (so you can see if there is a problem early on).

@ericvoshall As a quick solution, you could comment out two of the three services (or set replicas: 0 for them) for the first run. After the successful initialization of the repository, there should be no further problems I guess.

I know this is a bit ugly, but I am not quite sure what would be the best solution.

ericvoshall commented 1 year ago

Mhh.. That would be a solution, but not a good one IMO. I think it's quite helpful that the check is done on container start already (so you can see if there is a problem early on).

@ericvoshall As a quick solution, you could comment out two of the three services (or set replicas: 0 for them) for the first run. After the successful initialization of the repository, there should be no further problems I guess.

I know this is a bit ugly, but I am not quite sure what would be the best solution.

What about utilizing container health checks and reporting the container as healthy once it verifies the repository exists on startup? Then we could wait for the other containers to start until one reports it's healthy/the repository exists.