search

qubole / rubix

Cache File System optimized for columnar formats and object stores
Apache License 2.0
182 stars 74 forks source link

Avoid caching presto worker nodes #462

Open JamesRTaylor opened 3 years ago

JamesRTaylor commented 3 years ago

After a presto worker nodes gets deregistered from the coordinator. because it is no longer responsive, we've noticed that it still receives cache requests to be read from for a period of time. This can lead to internal errors for the query and/or slower response times. Instead, since the rubix presto cluster manager has a handle to the presto node manager, it can get the worker nodes directly from it, sorting and adjusting the node index only when the nodes have changed.

JamesRTaylor commented 3 years ago

@stagraqubole - I can take this one if you'd like.

shubhamtagra commented 3 years ago

Sure @JamesRTaylor