search

Orange-OpenSource / casskop

This Kubernetes operator automates the Cassandra operations such as deploying a new rack aware cluster, adding/removing nodes, configuring the C* and JVM parameters, upgrading JVM and C* versions, and many more...
https://orange-opensource.github.io/casskop/
Apache License 2.0
183 stars 54 forks source link

Enable cassandraBackups to create s3 buckets if needed #299

Closed zachlipp closed 3 years ago

zachlipp commented 3 years ago

Feature Request

Is your feature request related to a problem? Please describe.

I'm using a CassandraBackup that backs up to a minio s3 server within a cluster. If it's the first time this backup operation is performed, the bucket does not exist and the backup fails.

Describe the solution you'd like to see I'd like CassandraBackupSpec to add an optional field (e.g. createMissingBucket) that will create an s3 bucket if one does not exist.

Icarus accepts an argument for createMissingBucket; I think this feature could be implemented by populating this argument.

Describe alternatives you've considered It's possible to create this bucket with an initContainer when the minio pod starts.

cscetbon commented 3 years ago

Hey @zachlipp, why not, do you feel you could provide a small PR with that change ? It shouldn't be too hard.

zachlipp commented 3 years ago

Awesome. Yeah, I'll take care of it.

cscetbon commented 3 years ago

@zachlipp do you need any guidance ?

zachlipp commented 3 years ago

@cscetbon I haven't quite started; I'll get to it in the next few days

SuhailRahman commented 3 years ago

Hi @cscetbon, I'm interested in making the changes, can you please guide me in doing so.

cscetbon commented 3 years ago

@SuhailRahman It should be as simple as adding the parameter mentioned to https://github.com/Orange-OpenSource/casskop/blob/e5f169582b5794493c1d431658236f0cc6c1ae9b/pkg/backrest/backrest.go#L105-L116 as you can see at https://github.com/instaclustr/instaclustr-icarus-go-client/blob/1f9f5e201fe23f965a92a2b3215f112c1edff16e/pkg/instaclustr_icarus/model_backup_operation_request.go#L43

SuhailRahman commented 3 years ago

I tried it out it works, but I'm observing that first-time backup fails and creates the bucket, the second time onwards backup works completely fine.

smiklosovic commented 3 years ago

Can you please tell me what exactly fails? I am little bit lost here.

What I can do is to move this check / creation to coordinator (an Icarus instance contacted by operator as first when it is a global request) instead of having each individual Icarus doing the check but I am not sure if I am fixing the right thing here.

smiklosovic commented 3 years ago

Please check this one and let me know, it seems good here

gcr.io/cassandra-operator/instaclustr-icarus:1.0.10-SNAPSHOT-2

smiklosovic commented 3 years ago

@SuhailRahman does my latest comment fix your issue?

SuhailRahman commented 3 years ago

@smiklosovic I haven't tried it out, I'm little busy in another work. I'll try this tomorrow and let you know.

SuhailRahman commented 3 years ago

I tried it out and it works perfectly, thanks for the help!!

smiklosovic commented 3 years ago

I have pushed that change I did to Icarus' main brain but it is not released yet. You are free to use that image I pushed until then.

cscetbon commented 3 years ago

@SuhailRahman any news on that ? I suppose you need to upgrade the backrest image ?

SuhailRahman commented 3 years ago

@cscetbon are you referring to restoring tables?

cscetbon commented 3 years ago

@SuhailRahman no, this ticket only refers to the backup and the option to add. I'm asking what is the current status of that ticket

SuhailRahman commented 3 years ago

You can close this ticket.