[Storage] Remove file path cache tech debt from S3SingleDriverLogStore

delta-io / delta

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

https://delta.io

Apache License 2.0

7.64k stars 1.71k forks source link

[Storage] Remove file path cache tech debt from S3SingleDriverLogStore #3685

Closed scottsand-db closed 2 months ago

scottsand-db commented 2 months ago

Which Delta project/connector is this regarding?

[ ] Spark
[ ] Standalone
[ ] Flink
[ ] Kernel
[X] Other (Storage)

Description

S3 has been strongly consistent (for GET, PUT, LIST) for years (announcement), yet the S3SingleDriverLogStore has had code that assumes S3 is read-after-write inconsistent.

This PR removes such redundant code.

How was this patch tested?

Existing unit tests.

Does this PR introduce any user-facing changes?

No.