apache / polaris

Apache Polaris, the interoperable, open source catalog for Apache Iceberg
https://polaris.apache.org/
Apache License 2.0
1.17k stars 130 forks source link

Log the full TableMetadata #458

Open andrew4699 opened 2 days ago

andrew4699 commented 2 days ago

Description

Adds more logs. This is useful for determining the types of workloads happening on Polaris.

Type of change

How Has This Been Tested?

Checklist:

andrew4699 commented 1 day ago

I removed the ReportMetricsRequest logging as it seems like we already log it as a tag with Invoking CatalogApi with params.

RussellSpitzer commented 23 hours ago

I would consider this probably a "trace" level event. I agree with @snazy's comments here. I would hesitate to log these whole entities as well for security reasons.

andrew4699 commented 21 hours ago

@RussellSpitzer @snazy Thank you for the feedback. I'm new to the Iceberg space and appreciate the context you have on these objects. My hope is to make it easier to see the "unstructured" data that gets self-reported by the query engines. Would you feel more comfortable if this was scoped down to printing the last 5 Snapshot ID & summaries?

snazy commented 3 hours ago

@RussellSpitzer @snazy Thank you for the feedback. I'm new to the Iceberg space and appreciate the context you have on these objects. My hope is to make it easier to see the "unstructured" data that gets self-reported by the query engines. Would you feel more comfortable if this was scoped down to printing the last 5 Snapshot ID & summaries?

What's the use case for logging it at all, that cannot be done from an Iceberg client or curl?