as borg now uses repository.store_load and .store_save to load and save the chunks index, we need a rather high limit here.
a quick fix is to set this to ~4GiB (the maximum).
the real fix might be using chunks of the data (preferably <= MAX_OBJECT_SIZE), so there is less to unpack at once.
note that this only the case if the rpc code is used, for ssh: repositories.
Traceback (most recent call last):
File "borg/archiver/_common.py", line 176, in wrapper
File "borg/archiver/create_cmd.py", line 268, in do_create
File "borg/archiver/create_cmd.py", line 169, in create_inner
File "borg/archiver/create_cmd.py", line 527, in _rec_walk
File "borg/archiver/create_cmd.py", line 527, in _rec_walk
File "borg/archiver/create_cmd.py", line 467, in _rec_walk
File "borg/archiver/create_cmd.py", line 284, in _process_any
File "borg/archive.py", line 1381, in process_file
File "borg/archive.py", line 1142, in process_file_chunks
File "borg/archive.py", line 1132, in chunk_processor
File "borg/cache.py", line 754, in add_chunk
File "borg/cache.py", line 720, in seen_chunk
File "borg/cache.py", line 716, in chunks
File "borg/cache.py", line 667, in build_chunkindex_from_repo
File "borg/remote.py", line 539, in do_rpc
File "borg/remote.py", line 779, in call
File "borg/remote.py", line 898, in call_many
File "msgpack/_unpacker.pyx", line 399, in msgpack._cmsgpack.Unpacker.feed
File "msgpack/_unpacker.pyx", line 422, in msgpack._cmsgpack.Unpacker.append_buffer
msgpack.exceptions.BufferFull
ThomasWaldmann
commented
1 month ago
as borg now uses
repository.store_loadand.store_saveto load and save the chunks index, we need a rather high limit here.a quick fix is to set this to ~4GiB (the maximum).
the real fix might be using chunks of the data (preferably <= MAX_OBJECT_SIZE), so there is less to unpack at once.
note that this only the case if the rpc code is used, for
ssh:repositories.