airbnb / chronon

Chronon is a data platform for serving for AI/ML applications.
Apache License 2.0
746 stars 53 forks source link

Use builder pattern to create java fetcher and add a default log publish error rate of 0.1% #848

Closed pengyu-hou closed 1 month ago

pengyu-hou commented 1 month ago

Summary

We noticed log publish failures but the exception message is not clear. This PR will sample the error logs at 0.1% by default is no sample percentage is set in the metadata. Otherwise, it will take the number from the metadata.samplePercent.

To simplify the java fetcher debug process, refactored the code to use builder pattern so that different params can be set separately.

Why / Goal

Improve debuggability

Test Plan

Checklist

Reviewers

@airbnb/zipline-maintainers