Problems with archive made on S3 file system

[valdisvi]

If DAR archive is created on SSHFS or S3 mounted file systems, archive file is created (processing takes time), but at the end file is zero size.

To Reproduce

1. Mount SSHFS or S3 file system on some folder
2. Create DAR archive in that folder (tested with large arrays with volumes of 2GB in size)
3. Wait for DAR to finish
4. Check size of created archive

Expected behavior Proper archive file/files is/are created on mounted remote file system.

What happens After waiting archive is created, but file size is zero.

Software used:

• dar binary version 2.6.8 from Ubuntu package
• dar-2.6.13 tarball as source code package

Operating system used: Ubuntu 20.04 server (x86_64 and aarch64)

Additional context I suspect, stock Ubuntu package is just wrongly compiled and it should be compiled with:

Remote repository support  : YES

I tried to compile source code myself and I assume for that libcurl is needed. I have libcurl4-openssl-dev installed and (documentation says, how to disable libcurl, but even with that configure always says:

Remote repository support  : NO

Is my suspicion right, and how to recompile DAR with support for remote file system support?

[Edrusb]

Hi,

if you create a backup using tar instead of dar, does the resulting archive is also an empty file (no S3 or SSHFS available right now to test it).

the remote repository support let dar directly read or write a backup using FTP or SFTP to a remote host, this does not change the way the data is backed up

For more about the building process of dar see the documentation: http://dar.linux.free.fr/doc/from_sources.html

Regards, Denis

[valdisvi]

It actually works now with both tar and dar :

# tar czf /home/s3/tools/valdis.tgz home/valdis/
# ls -lh /home/s3/tools/valdis.tgz 
-rw-r--r-- 1 root root 624M feb 17 16:47 /home/s3/tools/valdis.tgz
# dar -Q -c /home/s3/tools/valdis.dar -w -asecu -aSI -s 2G -D -R /home/valdis
 --------------------------------------------
 20099 inode(s) saved
   including 23 hard link(s) treated
 0 inode(s) changed at the moment of the backup and could not be saved properly
 0 byte(s) have been wasted in the archive to resave changing files
 0 inode(s) not saved (no inode/file change)
 0 inode(s) failed to be saved (filesystem error)
 0 inode(s) ignored (excluded by filters)
 0 inode(s) recorded as deleted from reference backup
 --------------------------------------------
 Total number of inode(s) considered: 20099
 --------------------------------------------
 EA saved for 25 inode(s)
 FSA saved for 20071 inode(s)
 --------------------------------------------
# ls -lh /home/s3/tools/valdis.dar.1.dar 
-rw-r--r-- 1 root root 892M feb 17 16:49 /home/s3/tools/valdis.dar.1.dar
# dar -Q -c /home/s3/tools/valdis_`date -I` -w -asecu -aSI -s 2G -D -R /home/valdis/ \
-A /home/s3/tools/valdis.dar
 --------------------------------------------
 23 inode(s) saved
   including 23 hard link(s) treated
 0 inode(s) changed at the moment of the backup and could not be saved properly
 0 byte(s) have been wasted in the archive to resave changing files
 20076 inode(s) not saved (no inode/file change)
 0 inode(s) failed to be saved (filesystem error)
 0 inode(s) ignored (excluded by filters)
 0 inode(s) recorded as deleted from reference backup
 --------------------------------------------
 Total number of inode(s) considered: 20099
 --------------------------------------------
 EA saved for 0 inode(s)
 FSA saved for 0 inode(s)
 --------------------------------------------
# ls -lh /home/s3/tools/valdis
valdis_2021-02-17.1.dar  valdis.dar.1.dar         valdis.tgz               
# ls -lh /home/s3/tools/valdis_2021-02-17.1.dar 
-rw-r--r-- 1 root root 3,5M feb 17 16:53 /home/s3/tools/valdis_2021-02-17.1.dar

So problem should be somewhere else.

[valdisvi]

Maybe it is S3 problem. I got it fail with -z switch, but not all times:

https://user-images.githubusercontent.com/1501944/108249428-bbb48800-715d-11eb-83e0-3a60bf598e1a.mov

[Edrusb]

OK, thanks for feedback!

[valdisvi]

I think this is actually S3FS problem.

[valdisvi]

That was S3FS problem, which was solved by using newer version.