qubole / rubix

Cache File System optimized for columnar formats and object stores
Apache License 2.0
182 stars 74 forks source link

Lower the default max wait timeout #470

Open JamesRTaylor opened 3 years ago

JamesRTaylor commented 3 years ago

The default max wait timeout (DEFAULT_POOL_MAX_WAIT_TIMEOUT) should be lowered from 5000 milliseconds to 500 milliseconds. Having a higher wait time leads to more query timeouts and failures. The only downside of lowering it is that you'll fallback to S3 reads more frequently, but this is probably the correct behavior when the system is under heavy load and not responding within the default allowed time.