search

apache / iceberg-python

Apache PyIceberg
https://py.iceberg.apache.org/
Apache License 2.0
363 stars 136 forks source link

Consolidate catalog behavior #813

Open kevinjqliu opened 2 months ago

kevinjqliu commented 2 months ago

Feature Request / Improvement

We should ensure consistency in catalog behaviors for all implementations.

Currently, we have a few different catalog implementations (dynamodb, glue, hive, rest, and sql). Each catalog implementation supports a set of functions described in the Catalog ABC class (https://github.com/apache/iceberg-python/blob/main/pyiceberg/catalog/__init__.py#L276)

The behavior for each catalog implementation should be standardized. And we should test that a set of behaviors is consistent across all catalog implementations.

Here are a few related issues

Add SqlCatalog to test console #488 is an example PR that utilizes tests/cli/test_console.py to ensure the same behaviors for different catalogs.

rohithkanchukatla commented 1 month ago

Hii Kevin, Can I take up this issue?

kevinjqliu commented 1 month ago

@rohithkanchukatla sure thing

soumya-ghosh commented 2 weeks ago

@kevinjqliu this can be closed as well as all tasks in this issue are completed.

kevinjqliu commented 2 weeks ago

The behavior for each catalog implementation should be standardized. And we should test that a set of behaviors is consistent across all catalog implementations.

@soumya-ghosh I want to keep this issue open until we have a solution to address future updates. Ideally, there's a set of tests that will check the behavior of all the catalog implementations.

Add SqlCatalog to test console https://github.com/apache/iceberg-python/pull/488 is an example PR that utilizes tests/cli/test_console.py to ensure the same behaviors for different catalogs.

A possible solution is to test the input and expected output via the CLI