[Bug]: Unable to push to repo due to compressing error

[seacred]

What happened

After running ./script.sh the compressing procedure takes a while and the upload fails.

What did you expect to happen

Pushing configfiles to repository

How to reproduce

unable to tell

Additional information

No response

[Staubgeborener]

What exactly are you trying to upload? Is the amount of data perhaps a bit huge? Sounds like it's related to something like this on your system.

Could you please check the postBuffer with

cd ~/config_backup
git config --get http.postBuffer

You could try to change the value with git config http.postBuffer 524288000 and try to run ./script.sh again. If it still fails, please show your .env file (but hide your token). Please check if the repository, the username and also the branch name is correct.

[seacred]

None of my Files is bigger then 400kb. Running the command does not generate any output. My .ev is nothing special and mostly default and i've checked the username and repo.

It works until like 87% during the compression and tries to push after reaching to 100% but fails with error 400.

[Staubgeborener]

Odd. Only thing I can tell you: this is not a Klipper-Backup error but an issue with your system.

I would do the following:

# delete and re create your GitHub repository form scratch
sudo apt install --only-upgrade git
# or maybe updating the whole system with sudo apt update && sudo apt upgrade
sudo rm -rf ~/config_backup

After this run ./script.sh.

[CallumRD1]

I had klipper backup working fine for a while and then it stopped pushing backups altogether. After I realized this I manually ran the script so I'd get the verbose output and found similar set of errors. The compression step takes about 30 minutes and then it fails to push. It's just trying to backup my config folder, but when I comment out that folder in the .env and just try to have it backup the printer.cfg single file I get the same error.

[Tylerjet]

I had klipper backup working fine for a while and then it stopped pushing backups altogether. After I realized this I manually ran the script so I'd get the verbose output and found similar set of errors. The compression step takes about 30 minutes and then it fails to push. It's just trying to backup my config folder, but when I comment out that folder in the .env and just try to have it backup the printer.cfg single file I get the same error.

If you go to your $HOME/printer_data folder and run du -h how large does it say your config folder is?

[CallumRD1]

I had klipper backup working fine for a while and then it stopped pushing backups altogether. After I realized this I manually ran the script so I'd get the verbose output and found similar set of errors. The compression step takes about 30 minutes and then it fails to push. It's just trying to backup my config folder, but when I comment out that folder in the .env and just try to have it backup the printer.cfg single file I get the same error.

If you go to your $HOME/printer_data folder and run du -h how large does it say your config folder is?

378 MB, but 222 of that is input shaper results that I don't really want backed up. I tried changing my .env to only back up the cfg files I care about but I still get the same error, see the big screenshot in my previous post. Here is the view from Mainsail showing what I actually am trying to back up:

[Tylerjet]

If you increase the postBuffer as Staubgeborener mentioned above do you still get the error. it may be that because its still trying to push the deletion of those files to the remote its not able to complete.

Another option to try would be to go into config_backup and reset the repo to the latest commit git reset --hard then on github delete all the files there and go back to config_back and do a git pull and then try the script again.

[CallumRD1]

If you increase the postBuffer as Staubgeborener mentioned above do you still get the error. it may be that because its still trying to push the deletion of those files to the remote its not able to complete.

Another option to try would be to go into config_backup and reset the repo to the latest commit git reset --hard then on github delete all the files there and go back to config_back and do a git pull and then try the script again.

Those large files don't even exist on the repo. It was never able to get them to upload properly in the first place. The last commit that was successful was before the shake tune folders were added.

I have this printer set to backup a couple minutes after boot up and I've had it on nearly every day, so it's been trying and failing to backup for two weeks now.

[Tylerjet]

so lets try this in the config_backup folder, do git reset --hard origin/main this should reset your local repo to match what is currently on the remote as screenshotted and then try the running script.

[CallumRD1]

That appears to have worked.

I had modified the .env so that it's now only backing up the specific files I care most about, and that change is now mirrored on git. It didn't try to backup the shake tune files.

[Tylerjet]

NICE! yeah sounds like it was just the cached commits wanting to push all that data. Shouldn't be terribly hard to introduce excluding certain folders or just also skipping jpg's and csv's and .zips like we do for sym-linked files.

[CallumRD1]

It was almost certainly the pile of ~30 MB .csv files. If there was a way to easily exclude those, I'd be very happy. Automatically backing up the ~1 MB .png input shaper results would be useful, but I definitely don't need the raw data.

[Staubgeborener]

Read this to exclude specific files.

[Tylerjet]

actually that wouldn't work in this scenario, as that would be modifying .gitignore in the klipper_backup folder making it show as dirty in moonraker. Since we always clear config_backup after its complete and copy the fresh .gitignore on a new script run

[Staubgeborener]

Damn, never thought about this issue. We should fix this as .gitignore and the corresponding docs are somehow pointless in this case.

[CallumRD1]

Is it possible to put exclusions in the .env file similarly to how we choose what to include?

[Tylerjet]

I am working on that right now actually should be an easy change as we are already checking against a regex for the extra printer-[0-9]+_[0-9]+\.cfg$ files. so just making that a variable that can instead be set in .env so you can then just add all the file types and file names you want ignored.

[Staubgeborener]

@CallumRD1 Please update to the newest version, you can adjust the .env file to update the .gitignore in order to exclude more files flexible.

[github-actions[bot]]

This issue is stale because it has been open for 14 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.