pjspereira
commented
2 weeks ago I am attempting to use the SQLAlchemy backend right now, migrating from custom botorch to your service api. This was primarily based on what I estimated would be most efficient for many concurrent experiments with 0.1-3600 seconds sample latency (limited by black box function eval) and 10-100k samples, and some level of persistence as experiments may run for many days. I don’t have any metrics or experience to back the selection though. We use json file storage today, though it is not optimal. Can you compare your backend options for typical experiments? Perhaps just periodically saving json is fine?
In https://github.com/facebook/Ax/pull/2974 we added a message that reads "ATTENTION: The Ax team is considering deprecating SQLAlchemy storage. If you are currently using SQLAlchemy storage, please reach out to us via GitHub issues."
The Ax team spends a non-trivial amount of time maintaining support for saving and loading Ax Experiments and their associated structures to SQL databases via SQLAlchemy. However, its not easy for us to tell if this feature is being used by our open source community. In order to help us make the right decisions for Ax please let us know if you use this feature by commenting on this post.
Thank you to everyone in the Ax community for helping to make Ax a successful open source platform.