Migration of storage layer to delta-rs

This is the initial effort of switching our custom MVCC/Lakehouse protocol to use the Delta lake protocol instead.

A couple of general points:

Full range of DDL & DML statements are supported, including DELETE & UPDATE
VACUUM still not implemented for new tables
Likewise, GC of dropped new tables is not implemented yet, though the needed foundation is there (dropped_table table and related methods)
No perf testing has been done yet (the object upload code is likely sub-optimal as it doesn't do multipart upload)
cleanup of legacy code hasn't been completely done yet: I only replaced the stuff that was in the way of getting things working with delta-rs, in theory the entire writing logic for Seafowl tables can be erased now. The reading logic for Seafowl tables should still remain (there are tests for that too), as the plan is to have this available throughout the transient Seafowl versions for easier data migration.
I've also bumped to Datafusion 19.0.0/arrow post-33.0.0 in order to pick up https://github.com/apache/arrow-rs/pull/3752 and https://github.com/delta-io/delta-rs/pull/1212.

As for the implementation details:

I've opted for keeping a single root object store around (the inner in internal_object_store), and then "scoping" it for each DeltaTable accordingly. This means wrapping it up with some other object stores so that / points to the table root uri (see below). The other alternative that delta-rs offers is providing the store_options, out of which it will then build the store for a given uri, but I didn't like this mostly because the memory store implementation then doesn't work (it get's re-created from scratch for each create/write/table builder instantiation).
The table root is set to <inner_store_root>/<table_uuid>/ (that is the place where delta-rs puts all the table-related stuff as per the protocol). For instance s3://my_bucket/e62c26e2-b6e8-414a-a37a-66dd77430b5d/ or file:///data/dir/e62c26e2-b6e8-414a-a37a-66dd77430b5d/. The table_uuid is kept in the table catalog table and created randomly for each new table.
This approach means that dropping a schema/table can be done lazily (e.g. during subsequent VACUUMs in chunked batches) because table uri is unique to a given table and independent of db/schema/table name. Likewise, renaming a table is only a catalog operation.

splitgraph / seafowl