Nike-Inc / spark-expectations

A Python Library to support running data quality rules while the spark job is running⚡
https://engineering.nike.com/spark-expectations
Apache License 2.0
148 stars 32 forks source link

[FEATURE] Enable concurrent writes to stats table #60

Closed asingamaneni closed 7 months ago

asingamaneni commented 7 months ago

Is your feature request related to a problem? Please describe. Different processes writing to same stats table simultaneously is failing. Need to handle concurrent writes to stats table

Describe the solution you'd like Provide a retry logic for the writes to stats table here in this section

Describe alternatives you've considered NA

Additional context NA