Summary: Integrated Boot to accelerate the HPO and tuning loops of Proto-X
Demo:
On TPC-H SF10, using Boot (image 1), Proto-X finds a configuration with a "Best Seen Metric" of 69.67s after ~1hr of tuning. Without Boot (image 2), Proto-X only finds a configuration with a "Best Seen Metric" of 103.10s after ~1hr of tuning. Note that "Best Seen Metric" refers to the total runtime of the last run of all 22 queries TPC-H. "Best Metric" is set to "Best Seen Metric" whenever a run has no timed-out queries. Both runs used the same hyperparameters and the same per-query timeout.
Details:
Boot is automatically installed when running postgres build.
Redis (needed by Boot) is automatically started when Boot is enabled. This Redis does not conflict with the Redis instance used by Ray.
Boot can be toggled on and off with the CLI arg --enable-boot-during-[hpo|tune]. Boot is enabled separately for HPO and for tuning.
Boot can be configured with a .yaml file passed in through the CLI.
Currently, the results from Boot being enabled show the estimated runtime of the TPC-H workload, not the actual runtime. Thus, the results should be taken with a grain of salt. A future PR will resolve this.
Boot could potentially be even more effective if the per-query timeout was also decreased to reflect how Boot accelerates queries. This may be investigated in the future.
Currently, Boot's entire intelligent cache is cleared every time the configuration changes. A future PR will implement logic to selectively clear parts of the cache when the configuration changes.
Summary: Integrated Boot to accelerate the HPO and tuning loops of Proto-X
Demo: On TPC-H SF10, using Boot (image 1), Proto-X finds a configuration with a "Best Seen Metric" of 69.67s after ~1hr of tuning. Without Boot (image 2), Proto-X only finds a configuration with a "Best Seen Metric" of 103.10s after ~1hr of tuning. Note that "Best Seen Metric" refers to the total runtime of the last run of all 22 queries TPC-H. "Best Metric" is set to "Best Seen Metric" whenever a run has no timed-out queries. Both runs used the same hyperparameters and the same per-query timeout.
Details:
postgres build.--enable-boot-during-[hpo|tune]. Boot is enabled separately for HPO and for tuning.