qubole / rubix

Cache File System optimized for columnar formats and object stores
Apache License 2.0
183 stars 74 forks source link

Reduce the impact of cache pre-warm on the jobs - Part 1 #64

Closed abhishekdas99 closed 6 years ago

abhishekdas99 commented 7 years ago

Parallelize the cache pre warm process by directly reading the data from remote filesystem if block not found in local cache. As part 1, we will be implementing for data not present in local node. Later we will implement for remote node