linkedin / openhouse

Open Control Plane for Tables in Data Lakehouse
https://www.openhousedb.org/
BSD 2-Clause "Simplified" License
273 stars 43 forks source link

Add interfaces for StorageClient, Storage and its Implementation for Local #82

Closed HotSushi closed 2 months ago

HotSushi commented 2 months ago

Summary

This PR sets foundation for Storage interfaces and provides a sample implementation called LocalStorage

two primary interfaces are:

Storage {
.. high level functionality expected from the storage
 }

StorageClient<T> {
 .. lower level client functionality such as token refresh.
}

This PR also adds LocalStorage,LocalStorageClient to show how new implementations should be added.

Methods in Storage,StorageClient are minimal right now, but they'll be extended as we do refactoring.

For more details please see the document here.

Whats with @Lazy?

Yaml properties are not available when the @Components are initialized, they get configured after the components are initialized, therefore we cannot use handy tools like @ConditionalOnProperty/ @ConditionalOnBean. @Lazy is next best thing, it makes the bean initialize lazily and this is fantastic for heavy-lifting fsclients. This is also similar with our previous behavior: HdfsStorageProvider

Testing Done