archiecobbs / s3backer

FUSE/NBD single file backing store via Amazon S3
Other
538 stars 77 forks source link

No "x-amz-meta-s3backer-filesize" meta-tag #141

Closed warren-ru closed 3 years ago

warren-ru commented 3 years ago

No meta-tag is provided for S3 objects for further block size auto-detection.

archiecobbs commented 3 years ago

I'm going to need more detail.

http://testthewebforward.org/docs/bugs.html

warren-ru commented 3 years ago

OS: Ubuntu 18.04.5 LTS (GNU/Linux 4.15.0-128-generic x86_64) s3backer version: 1.6.0

mount bucket by following command:

s3backer \
    --listBlocks    \
    \
    --blockSize=512k    \
    --size=500G     \
    \
    --blockCacheFile=/mnt/nextcloud-bucket/blockCacheFile   \
    --blockCacheMaxDirty=64     \
    --blockCacheSize=256        \
    --blockCacheThreads=1       \
    --blockCacheWriteDelay=1000 \
    \
    --encrypt   \
    --passwordFile=/etc/s3backer/passwordFile   \
    \
    --accessFile=/etc/s3backer/accessFile               \
    --baseURL=https://storage.yandexcloud.net/  \
    --storageClass=STANDARD_IA  \
    \
nextcloud-example   \
/mnt/nextcloud-bucket/s3backer

then create a filesystem: mkreiserfs -f -b 4096 -s 513 /mnt/nextcloud-bucket/s3backer/file

mount as a storage: mount -o loop /mnt/nextcloud-bucket/{s3backer/file,filesystem}

archiecobbs commented 3 years ago

Thanks. What exactly is the problem? What were you expecting to occur that didn't?

warren-ru commented 3 years ago

I need to set block size and drive size every time I mount S3 drive

aaruni96 commented 3 years ago

s3backer will autodetect block size and drive size on its own.

in fact, if you provide a block size or drive size which is different from the auto detected value, s3backer will refuse to continue except if you also provide the --force tag.

warren-ru commented 3 years ago

It can't automatically detect block size and drive size. When I try to mount the bucket not providing these options It gives no results

I have to set it by myself.

And I read in the manual that it safer no to set block size, but use built-in detection

archiecobbs commented 3 years ago

OK now I've reproduced the problem.

In the meantime, just continue to include --size and --blockSize as a workaround.

And I read in the manual that it safer no to set block size, but use built-in detection

This is misleading... as long as you don't use --force then it won't try to use the wrong size(s).

archiecobbs commented 3 years ago

Should be fixed in e6583a6.

warren-ru commented 3 years ago

A little bit off topic, but... The way you publish s3backer releases doesn't give opportunity for users to get notifications about them through GitHub subscriptions.

More here: https://github.community/t/not-getting-release-notification-despite-watching-releases/1792/4

archiecobbs commented 3 years ago

I don't "publish releases" on github. All I do is create git tags. All the other stuff that happens in github's UI (e.g., listing tags under "Releases", creating downloadable tar files that people don't know what to do with because there's no ./configure script) is stuff that github does automatically and which I have no control over.

So I'd say this is an issue for them to deal with... it's their mess!