linkedin / openhouse

Open Control Plane for Tables in Data Lakehouse
https://www.openhousedb.org/
BSD 2-Clause "Simplified" License
294 stars 50 forks source link

Introduce StorageManager and an Hdfs Implementation for Storage #90

Closed HotSushi closed 5 months ago

HotSushi commented 5 months ago

Summary

Laying foundations for storage part 3: StorageManager and an implementation for Hdfs: HdfsStorage.

StorageManager interface looks like

StorageManager {
  Storage getDefaultStorage()
  Storage getStorage(Type)
}

It runs through all configured Storages and returns the appropriate storage for default or for particular type.

HdfsStorage is yet another implementation for Storage similar to LocalStorage. We've abstracted the common code for all storage-impls into BaseStorage

Changes

Testing Done