Closed Andrew-S-Rosen closed 2 years ago
Yeah, unfortunately, I don't think it is possible to use the simplified format to do this, as the shorthand only supports Store keyword arguments being strings, dicts, floats, or lists and not arbitrary python objects like other stores. We could add in shorthand specifically for S3Store
. In the time being, the easiest would be to use the long form specification. I.e., the full yaml output from JobStore.as_dict
.
The easiest way to do this would be something like:
from monty.serialization import dumpfn
store = JobStore(...) # construct job store object
jobflow_config = {"JOB_STORE": store}
dumpfn(jobflow_config, "jobflow.yaml")
That's what I figured. Thanks for the workaround! That should work!
I'm going to close this for now as there is a workaround. But if it bothers you enough to want to submit a PR for this custom case then I'll happily accept it.
I am trying to set up a MinIO bucket as an additional store in my
jobflow.yaml
. The format I have so far is below. However, this isn't sufficient because theS3Store
takes anindex
as the first argument, which itself is aMongoStore
pointing to the collection to use as an index for the data in S3/MinIO. Perhaps it's just not clear to me how to do this (in which case a docs update could help), but I don't think there is an easy way to have theindex
point to thedocs_store
.