One line MongoDB setup + usage example

[ahundt]

Once #205 is resolved, can we add a section to the sacred docs with:

1. One line MongoDB setup based on https://github.com/bitnami/bitnami-docker-mongodb
2. Add a small example that shows how to:
   1. uses the one line setup
   2. writes to the database
   3. view the stored configuration
   4. load the stored configuration

(I don't quite know how to do all of this yet)

[Qwlouse]

Good idea. Let me know if you need help with any of these.

[uatach]

Any news? Can I help with this?

[ahundt]

Not much progress unfortunately. Yeah that would be great, unfortunately I've ended up just doing regular json dumps to a folder for my own work because I ran out of time to spare on this, and simplified setup + creating ways of calling sacred from existing apis without using the experiment @ tag setup still seem to be in progress.

[Qwlouse]

@ahundt and @uatach : You are interested in a full example on how to use MongoDB in the docs, right? This should not be hard to do.

1. On Ubuntu sudo apt install mongo installs a local MongoDB (though official docs recommend using their mongodb-org package). You might need to manually start the demon with sudo service mongodb start (not sure). Of course the docker based setup should also work, but might require some network configuration and later specifying the url in the mongoobserver. You can verify if it is running correctly by starting the mongo client with mongo.

2. With that in place you can just use -m my_sacred_db_name on your experiment to add a MongoObserver that writes to that database. Make sure to have the pymongo package installed (pip install pymongo)

3. You can use the mongo client to look at stored results.

   $ mongo
   MongoDB shell version v3.4.7
   connecting to: mongodb://127.0.0.1:27017
   MongoDB server version: 3.4.7
   > show dbs
   admin   0.000GB
   local   0.000GB
   my_sacred_db_name
   > use my_sacred_db_name
   > show collections
   fs.chunks
   fs.files
   runs
   > db.runs.findOne()
   {
   "_id" : 1,
   "experiment" : {
       "name" : "hello_cs",
       "base_dir" : "/home/greff/programming/sacred/examples",
   ....

   But maybe a nicer way is to query from python using pymongo:

   import pymongo
   client = pymongo.MongoClient()
   runs = client.my_sacred_db_name.runs
   entry = runs.find_one()

   Which returns a dictionary with the stored results for a run. The mongo interface has rich capacities for querying, to find the right experiment entry.

4. Calling the experiment with the retrieved config:

   from my_experiment import ex
   ex.run(config_updates=entry['config'])

   Is that the information you were looking for? @ahundt : What do you mean by "creating ways of calling sacred from existing apis without using the experiment @ tag setup still seem to be in progress."? @uatach : If you clean up these instructions I'd be happy to include them in the docs.

[ahundt]

I mean something very similar to the stuff that was discussed in https://github.com/IDSIA/sacred/issues/193, I have a bunch of code where I could easily pass something object oriented in and add to it, but that code doesn't mix well with how experiments are defined in a global namespace, and I didn't find examples using sacred configs and variables within the context of a class or another function that fit well with my existing code base. It is certainly possible I missed something obvious, I just ran out of time to figure it all out.

For example, I'm using gpyopt for hyper parameter optimization, so if that chooses my config ~2000 times in a single hyperopt run which is passed in a callback, so I wasn't sure about correctly storing it in sacred without breaking anything.

Ultimately I started dumping a dictionary to json was a one liner json.dump(), I threw a time stamp into the filename and that covered what was truly essential to my project with lower time investment. I know that obviously lacks many of the nice sacred features, but I've been able to reproduce my results reasonably well with that approach. Heh, sorry if that sounds less than ideal, I just had to move along quickly.

[Qwlouse]

@ahundt No worries! Feel free to use Sacred any way you like. ;-) I was just trying to figure out if there was anything important left for me to do.

[zacwellmer]

@Qwlouse That mongo example would be great in the documentation. It would be even better if there was a good example of how to link this was sacredboard(though I know this is outside the scope of this repo)

[JarnoRFB]

The desired setup instructions should be documented at https://sacred.readthedocs.io/en/stable/examples.html#docker-setup. For a complete walk trough see also https://github.com/JarnoRFB/pydata_berlin_sacred. Closing this for now. Feel free to reopen if you feel that something is missing.