openstack cinder-backup integration

jsm222 commented 2 years ago

Hi

FYI I did some work here to wrap benji into the openstack cinder-backup service with a custom backup driver. https://github.com/jsm222/cinder-backup-benji-driver. To test on devstack use cinder.conf:

backup_driver =  cinder.backup.drivers.benjidriver.BenjiBackupDriver

the rest of its options can be read in the source for now. benji in an venv created with --system-site-packages should work at least on devstack. Place it in /opt/stack/cinder/cinder/backup/drivers for default devstack...

elemental-lf commented 2 years ago

Thank you for letting us know! Do you have any plans to flesh this out some more and maintain it in the future? If that is the case, I'd add a link to the documentation when you think it's ready for that.

It might be useful to add some custom labels with OpenStack specific metadata to the backups if that information is available. That way users could filter on this metadata when using benji ls or they could use it to implement different retention policies based on it for example.

jsm222 commented 2 years ago

Thanks for your reply.

Using mysql I found

sqlalchemy.exc.InternalError: (pymysql.err.InternalError) (1170, "BLOB/TEXT column 'checksum' used in key specification without a key length")
[SQL: CREATE INDEX ix_blocks_checksum ON blocks (checksum)]

solved by

    --- /usr/local/lib/python3.8/dist-packages/benji/database.py.orig   2022-06-20 19:39:16.450627651 +0000
+++ /usr/local/lib/python3.8/dist-packages/benji/database.py    2022-06-20 19:39:28.657846049 +0000
@@ -786,7 +786,7 @@
         sqlalchemy.Index(None, 'uid_left', 'uid_right'),
         # Maybe using an hash index on PostgeSQL might be beneficial in the future
         # Index(None, 'checksum', postgresql_using='hash'),
-        sqlalchemy.Index(None, 'checksum'),
+        sqlalchemy.Index(None, 'checksum',mysql_length=64),
     )

     def deref(self) -> DereferencedBlock:

I do not know if mysql_length is ignored on postgres but i assume it is...

Also are benji using any specific sqlalchemy features not present in 1.3.19 or 1.3.22? I could not find any issue with light testing on 1.3.19 so why is the version bumped in v.0.16.1 this high in https://github.com/elemental-lf/benji/commit/69bb667e493f2542887038800a7955b6f966ac44 Thanks

elemental-lf / benji

openstack cinder-backup integration #143