how to use with Google Cloud Storage?

[stefangweichinger]

Is there any documentation or howto for using a Google Cloud Storage bucket? I was once told that this is possible but never figured out the actual config.

Does anyone use that?

[prajwaltr93]

Access ID: GOOG1...

even the access_key i have access to has this prefix GOOG1, meaning they are the same, interchangeable terms likely

[stefangweichinger]

Why "AWS4", when it's Google Cloud Storage in my case?

AWS4 here stands for AWS Signature Version 4 is a standard for authentication, its common across major cloud storage providers.

Can't read label: While trying to read tapestart header:

could this be due to use of latest curl library ? reverting back might help

downgrading curl now for a test. 7.88.1-r2 ... I'll see in a minute.

[stefangweichinger]

Access ID: GOOG1...

even the access_key i have access to has this prefix GOOG1, meaning they are the same, interchangeable terms likely

I was looking for how to generate these keys with the CLI-tools for Google Cloud. I am not sure yet.

[stefangweichinger]

curl-7.88.1-r2 doesn't make a difference :-(

The request signature we calculated does not match the signature you provided. Check your Google secret key and signing method.

[stefangweichinger]

Same with curl-7.87.0-r2

[stefangweichinger]

Tried my second HMAC-Keypair. Same errors.

[stefangweichinger]

More context: this is when I run amcheck.

slot 1: Can't read label: While trying to read tapestart header: The request signature we calculated does not match the signature you provided. Check your Google secret key and signing method. (SignatureDoesNotMatch) (HTTP 403)

This is only with "STORAGE_API" "AWS4".

With "STORAGE_API" "OAUTH2" I get "Missing client_id properties", when I comment out "STORAGE_API", I see:

$ amcheck abt -o storage=cloud
Amanda Tape Server Host Check
-----------------------------
NOTE: Holding disk '/mnt/amhold/abt': 363 GB disk space available, using 363 GB
slot 7: Can't read label: Amanda header not found -- unlabeled volume?
slot 8: Can't read label: Amanda header not found -- unlabeled volume?

$ amlabel abt -o storage=cloud cloud_01 slot 1
Reading label...
Found an empty tape.
Writing label 'cloud_01'...
Error writing label: While writing amanda header: Too many retries; last message was 'S3 Error: Unknown (empty response body)' (None) (CURLcode 92) (HTTP 400) (after 14 retries).
Error writing label: While writing amanda header: Too many retries; last message was 'S3 Error: Unknown (empty response body)' (None) (CURLcode 92) (HTTP 400) (after 14 retries).

[stefangweichinger]

I added

device_property visible "S3_BUCKET_LOCATION"    "europe-west3" # defaults to us-east-1

because the bucket is located there. I assume that's important, but it didn't yet fix things.

[prajwaltr93]

i have curl 7.29, which is very old, i will check and see if upgrading to any other version breaks mine.

also found this : https://github.com/zmanda/amanda/pull/137#issuecomment-1562086498

content-length header was not accurate

it is similar to findings i had when using latest CURL library,

[prajwaltr93]

i faced similar issues on my Debian 11 WSL

amandabackup@workstation:/etc/amanda$ amlabel MyConfig MyConfig-1 slot 1
'/etc/amanda/MyConfig/amanda.conf', line 8: warning: Global changerfile is deprecated, it must be set in the changer section
Reading label...
Error reading volume label: While creating new S3 bucket: The request signature we calculated does not match the signature you provided. Check your Google secret key and signing method. (SignatureDoesNotMatch) (HTTP 403).
Not writing label.
Not writing label.
amandabackup@BSL-BNG-L591:/etc/amanda$ apt list --installed | grep libcurl4

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

libcurl4-nss-dev/stable,stable-security,now 7.74.0-1.3+deb11u7 amd64 [installed,automatic]
libcurl4/now 7.74.0-1.3+deb11u3 amd64 [installed,upgradable to: 7.74.0-1.3+deb11u7]
amandabackup@BSL-BNG-L591:/etc/amanda$

so its narrowing down to libcurl library

[stefangweichinger]

Oh, interesting! I thought I was crazy ;-) I wonder how I can downgrade even more on that gentoo server. Additionally: do I have to recompile amanda (does it include these libraries) or is it enough to have the binary or/and libs in place?

[stefangweichinger]

trying to compile older versions now. 7.81 didn't help. 7.79 neither. But I have to recompile amanda, that's what I missed here. STILL the same errors after compiling amanda against libcurl-7.79 (at least I assume so, maybe I have to recheck things and cleanup some things. This was just a quick first shot)

[stefangweichinger]

No success so far. Went back to curl-7.79 (installed manually), re-compiled amanda etc I think I will try it on a debian server asap. The gentoo-environment will be removed anyway sooner or later.

[stefangweichinger]

It might be easier for me to compile amanda-3.6. In https://github.com/zmanda/amanda/issues/213#issuecomment-1475544236 it was mentioned that 3.6 doesn't have that issue. Pls tell me which branch to use.

[prajwaltr93]

since you have access_key and secret_key i would suggest 3_6, since this PR branch is based off of 3_5.

[prajwaltr93]

do I have to recompile amanda (does it include these libraries) or is it enough to have the binary or/and libs in place?

amanda is likely built based on curl library available in /usr/lib/curl/ so i would suggest recompiling.

[stefangweichinger]

since you have access_key and secret_key i would suggest 3_6, since this PR branch is based off of 3_5.

Thanks. This leads to new problems, autogen fails determining the platform ... all this leads way too far already.

[stefangweichinger]

went back to curl-7.63 without success

[stefangweichinger]

I'd love to see a patched version with curl enabled, that brings a working libcurl with it ... Gentoo in theory gives me plenty of options to fine tune the compilation but on the other hand I don't even know which curl really works and I am rather alone as amanda user on gentoo (plus using S3).

I currently have to fix the old non-S3 backup installation and feel a bit frustrated here. Basically I have a backup of a 3.5.1-release with buggy libcurl I should be able to roll back to (and that currently also doesn't work fully).

Any ideas? Shouldn't there be a patch for curl maybe? I think of patching a current curl-release or so.

[stefangweichinger]

I will set up a new amanda installation on a brand new Debian11 machine and retry things there.

curl-7.74 there ...

[stefangweichinger]

opened bug at gentoo as well: https://bugs.gentoo.org/907685

[prajwaltr93]

Any ideas? Shouldn't there be a patch for curl maybe? I think of patching a current curl-release or so.

we have ticket tracking this exact issue since it was reported. fixing amanda to work with latest curl library would be great, haven't gotten into it yet. will update if any progress is made,

[stefangweichinger]

@prajwaltr93 Why does 3_6 work then? I might try to use that if possible. I could try to come up with a working ebuild for 3_6 and/or build packages for Debian 11. For sure I'd appreciate if you could provide these debian packages also. I contacted the Debian maintainer for amanda also and asked if he already has packages for 3_6.

[prajwaltr93]

but for debain there should be a folder called debian under packaging, you should be able to install dependencies using the control file, and get a .deb installer by running ./packaging/deb/buildpkg, ./packaging/deb/buildpkg server if only server binaries are required.

you can build deb packages yourself from instructions above, this is how i usually build and install amanda for debian machines.

Why does 3_6 work then?

i was referring to the issue tracker myself, but have to investigate.

[stefangweichinger]

but for debain there should be a folder called debian under packaging, you should be able to install dependencies using the control file, and get a .deb installer by running ./packaging/deb/buildpkg, ./packaging/deb/buildpkg server if only server binaries are required.

you can build deb packages yourself from instructions above, this is how i usually build and install amanda for debian machines.

Sure. It would also be great if the amanda project would provide packages for the stable releases at least (3.5.3 anyone?). I think of build pipelines, github actions etc ... just mentioning ... why aren't these things used?

-> A set of packages which are known to work and can be used/tested by multiple people.

Building for gentoo is a different issue, I understand ... the current packaging subdir doesn't cover that as far as I see.

Why does 3_6 work then?

i was referring to the issue tracker myself, but have to investigate.

Thank you.

[prajwaltr93]

Sure. It would also be great if the amanda project would provide packages for the stable releases at least (3.5.3 anyone?).

you can get 3.5.3 packages here

I think of build pipelines, github actions etc ... just mentioning ... why aren't these things used?

that would be a great add, let me check with my team if we can do that.

[stefangweichinger]

Sure. It would also be great if the amanda project would provide packages for the stable releases at least (3.5.3 anyone?).

you can get 3.5.3 packages here

Really? Where?

I think of build pipelines, github actions etc ... just mentioning ... why aren't these things used?

that would be a great add, let me check with my team if we can do that.

Sure, go for it. I think it would be great to have a pipeline that builds amanda with some sane defaults automatically. Even better: let it build the packages for debian etc ... the packaging scripts are there, I assume it shouldn't be that hard . Looking forward to this progress.

[prajwaltr93]

you can get 3.5.3 packages here

https://www.zmanda.com/downloads/

sorry, missed the link there

[stefangweichinger]

@prajwaltr93 Last time I was at that link, there were only downloads for Debian 8 or so. Thanks, will check these out.