borg create: Everything 0 archive_progress at end of backup

sophie-h commented 2 years ago

Have you checked borgbackup docs, FAQ, and open Github issues?

Yes

Is this a BUG / ISSUE report or a QUESTION?

BUG

System information. For client/server mode post info for both machines.

Your borg version (borg -V).

1.2.0

Operating system (distribution) and version.

Hardware / network configuration, and filesystems used.

How much data is handled by borg?

Tested with very small test repos

Full borg commandline that lead to the problem (leave away excludes and passwords)

Describe the problem you're observing.

During borg create I get the usual json output

{"original_size": 107384183, "compressed_size": 103279086, "deduplicated_size": 759473, "nfiles": 15, "time": 1649719099.1076434, "type": "archive_progress", "path": "somepath"}

But at the end I get

{"original_size": 0, "compressed_size": 0, "deduplicated_size": 0, "nfiles": 0, "time": 1649719099.2390628, "type": "archive_progress", "path": ""}

and the next message is

{"message": "Saving files cache", "operation": 2, "msgid": "cache.commit", "type": "progress_message", "finished": false, "time": 1649719106.242265}

I'm not sure how I should handle the archive_progress output with everything zero. Does it have a special meaning or is it a bug?

Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.

Include any warning/errors/backtraces from the system logs

ThomasWaldmann commented 2 years ago

That comes from Statistics.show_progress:

every dt this emits some progress info
at the end, it gets called with final=True and
- for the console this clears the progress display (and does not show any data)
- for json there is no special treatment of the final flag (which might be a bug)

TODO:

check the docs, if there is documented related behaviour
decide whether to introduce special treatment of final json

ThomasWaldmann commented 2 years ago

The docs say nothing about a "final" json for the archive_progress, they only describe the normal output.

The (somehow related) progress_percent / progress_message json includes finished=true/false for the same purpose and that is documented.

So I think finished=true/false should be added for all archive_progress messages and the empty stuff should be removed from the final finished=True output. Also the docs (frontend.rst) should be updated accordingly.

The final message can be used by GUI for similar purposes as for the text-based UI: e.g. clear the progress display.

@sophie-h @m3nu do you agree?

Bounty: https://app.bountysource.com/issues/106988150-borg-create-everything-0-archive_progress-at-end-of-backup

sophie-h commented 2 years ago

do you agree?

Thanks! Sounds good to me. I think it's much cleaner than adding a special case for 'everything 0' in the frontend code.

wisp3rwind commented 2 years ago

Also the docs (frontend.rst) should be updated accordingly.

It seems that this slipped through? The docs at https://borgbackup.readthedocs.io/en/stable/internals/frontends.html are now inaccurate, it would be great if they could be adapted. Thanks!.

ThomasWaldmann commented 2 years ago

@wisp3rwind thanks for the hint, fixed by #6889 #6890 #6891.

borgbackup / borg