Open thangckt opened 2 months ago
for temp use just modify dpdispatcher/contexts/ssh_context.py _get_file method? (tar_command and split command).
For example
# Step 1: Create a tar archive
tar -cvf archive.tar largefile.txt
# Step 2: Split the tar archive into 100 MB chunks
split -b 100M archive.tar archive_part_
# Step 3: Combine the split files back into a single tar file
cat archive_part_* > archive.tar
# Step 4: Extract the contents of the tar file
tar -xvf archive.tar
Dear developers,
Can you give an option to split a large .tar file into smaller parts before downloading?
It will more efficient for transferring.
I face a problem that the backward files have large size, then it can not be downloaded, even dpdispatcher finishs without error.
Thank