search

NOAA-GSL / VxIngest

Other
2 stars 0 forks source link

change storage on on-prem couchbase systems to MAGMA from COUCHSTORE. #359

Open randytpierce opened 3 months ago

randytpierce commented 3 months ago

We learned conclusively in the capella evaluation that our dataset is far better suited to Magma storage than it is to Couchstore storage. We need to convert the storage over. For the adb-cb[234] cluster we simply need to drop and recreate the bucket, but for the adb-cb1 standalone cluster we need to either create a new bucket or backeup, drop, recreate, and restore the bucket that exists. We can rely on xdcr to then replicate the adb-cb1 data to the production cluster.

github-actions[bot] commented 2 weeks ago

This issue is stale because it has been open 90 days with no activity.

randytpierce commented 5 days ago

This is done on adb-cb1 but still needs to be done on the three node cluster

gopa-noaa commented 23 hours ago

Screenshot 2024-07-22 at 11 27 59 AM

gopa-noaa commented 23 hours ago

@randytpierce adb-cb1 says storage is still Couchstore. Looks like I may have to stop the server, and see if I would get an option to convert on the fly to Magma. If not we would have to drop and recreate the bucket. One option is to leave adb-cb1 alone and drop and recreate cluster buckets, and let XDCR take care of re-populating. Let me know what you all think.

randytpierce commented 23 hours ago

It does. I really thought I had already changed it, but maybe it was on the VM couchbase instance. Sorry about that. The XDCR is going from adb-cb1 to the cluster. It seems to me like we need to turn xdcr off, change the bucket on adb-cb1, then on the cluster, then turn xdcr back on. randy

On Mon, Jul 22, 2024 at 11:33 AM Gopa @.***> wrote:

@randytpierce https://github.com/randytpierce adb-cb1 says storage is still Couchstore. Looks like I may have to stop the server, and see if I would get an option to convert on the fly to Magma. If not we would have to drop and recreate the bucket. One option is to leave adb-cb1 alone and drop and recreate cluster buckets, and let XDCR take care of re-populating. Let me know what you all think.

— Reply to this email directly, view it on GitHub https://github.com/NOAA-GSL/VxIngest/issues/359#issuecomment-2243470878, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGDVQPQBCPPFHB57LY4TIBTZNU66TAVCNFSM6AAAAABKIUXMHWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBTGQ3TAOBXHA . You are receiving this because you were mentioned.Message ID: @.***>

-- Randy Pierce

gopa-noaa commented 13 minutes ago

I tried pausing XDCR on adb-cb1, but it still would not allow me to change the storage to Magma. Lets discuss this at the dev meeting and see what would be a best way to move forward. If switching to Magma on adb-cb1 means dropping and repopulating the vxdata bucket on adb-cb1, my take is leave adb-cb1 alone and drop and re-create the buckets with Magma on the on-prem cluster.