search

apache / polaris

Apache Polaris, the interoperable, open source catalog for Apache Iceberg
https://polaris.apache.org/
Apache License 2.0
1.13k stars 123 forks source link

[BUG] Inability to Create Catalog via REST API Without Using Polaris CLI First #358

Open mouadk opened 1 month ago

mouadk commented 1 month ago

Is this a possible security vulnerability?

Describe the bug

When creating a catalog using the CLI, a CreateCatalogRequest is created, and a CatalogEntity gets persisted in the underlying storage. During this process, the IcebergCatalogAdapter#getConfig (corresponding to the api/v1/config) is invoked, which fetches the catalog name surfaced in the warehouse field and returns properties (https://iceberg.apache.org/docs/latest/configuration/#catalog-properties) for overrides if applicable.

However, when creating a catalog for the first time (assuming it was not done using the Polaris CLI), there is no existing catalog unless you bypass the api/v1/config call and directly invoke catalog creation, similar to the CLI (as seen in regtests). For example, when using the Java core API, the process first fetches the config during catalog build (org.apache.iceberg.rest.RESTSessionCatalog.initialize) to determine if any overrides are required. This results in the error:

Exception in thread "main" org.apache.iceberg.exceptions.RESTException: Unable to process: Unable to find warehouse XXXX

I am forced to first create the catalog using the Polaris CLI and cannot achieve the same via the REST endpoint by specifying the warehouse location in the configuration.

Would returning something like {"defaults":{},"overrides":{}} be a solution in this case? Also, why is it that the warehouse field requires the catalog name instead of allowing a direct reference to locations like S3 or Blob storage?

Is this expected behavior; where we must first create the catalog using the Polaris CLI? Or did I overlook something?

Thanks in advance.

To Reproduce

Use Java Core API to create a Catalog.

Actual Behavior

If Catalog unknown to Polaris, Error is raised when creating a new catalog (NOT FOUND)

Expected Behavior

Empty Result ?

Additional context

No response

System information

MacBook Pro: 2.9 GHz Quad-Core Intel Core i7

mouadk commented 4 weeks ago

Also it looks like the implementation is not matching the API Spec https://github.com/apache/iceberg/blob/208ab20dc9ab8bcab3ee525d0ddaba80eeae7609/open-api/rest-catalog-open-api.yaml#L75