airbnb / chronon

Chronon is a data platform for serving for AI/ML applications.
Apache License 2.0
673 stars 36 forks source link

Don't request null keys from KVStore #774

Open jbrooks-stripe opened 1 month ago

jbrooks-stripe commented 1 month ago

Summary

Updates the Fetcher to not include or attempt any requests when the keys are null.

Why / Goal

KVStore implementations are left up to consumers. In the case where null entries are being written out, this can lead to online/offline consistency issues. The GroupByUpload jobs are all setup to filter out any null keys, but the KVStore will still attempt to pull values if they exist. This adds parity with the null checks done in the GroupByUpload jobs to the online path and will exclude any attempt to pull values for null keys.

Test Plan

Checklist

Reviewers

piyushn-stripe commented 3 weeks ago

@jbrooks-stripe - might need a rebase of this branch