Closed jonastim closed 1 year ago
Yeah totally. So there are a few things that should help with this...
First, you're right the result only has general metadata bout the experiment though you can augment the data it does collect. But it still would fall short of actually having the trained learner in hand. There is no functionality presently to easily store that inside an experiment's results.
Now regarding getting the trained learner there are two reasons why modifying the learner "in-place" is a little nuanced:
At present, there are a few features/tricks you can use to get around this:
If you let me know more about what you're trying to do and what you want to know I could send you a small code example.
Also, in the next two weeks I plan on upgrading Coba so that you can run experiments on multiple environments and get the learners back. It's funny that you ask about this now because it was next on my to do list. You'd still be limited to only using a single core but it'd be better than what we've got now.
Thanks a lot for all the great information, Mark!
For my current explorations, I've run only a single process with a couple of shuffled environments. With the shuffling removed, I see that the learners are updated in place which is great.
The way I've accessed the learner so far is by adding them to the interactions table (alongside some other information) which is a bit wasteful but not a major concern for my experiments.
I hadn't thought about the multi-process aspect of it. Having a way to serialize the models and experiment results at the end of the run would be helpful for that.
Looking forward to trying out the new changes that are coming down the line 👍
Cool, yeah you've got the right idea... Yes, you can stick anything into what is returned and it ends up in the interactions table.
Usually what I do is something like this:
from coba.experiments import SimpleEvaluation
my_trained_learners = []
class MyEvaluation:
def process(learner,interactions):
yield from SimpleEvaluation().process(learner,interactions)
#Here you can put any custom code you want for learner... for example you could serialize it to file....
my_trained_learners.append(learner)
learners = ...
environments = ...
cb.Experiment(env,lrn,evaluation_task=MyEvaluation()).run()
#my_trained_learners should have all your learners now...
That would mean you don't have to get into the guts and mess with modifying coba code and installing locally.
A second option still in line with what you're doing is if all you want is the learner would be something like this:
class MyEvaluation:
def process(learner,interactions):
#we list to make it evaluate but we don't actually care about its return
list(SimpleEvaluation().process(learner,interactions))
#now our learner is trained because we evaluated above
#the interactions table will now have a single interaction with a learner column
yield {"learner":learner}
Does that make sense? The magic here is that you can define your own evaluator and really do anything you want...
Good points! I forked the repo and will try to contribute changes that are generally useful upstream (e.g. the option to log context with the interactions) and hope to address the remaining customizations through subclassing.
That'd be awesome. Looks like maybe you saw I changed SimpleEvaluation a tiny bit when making your pull request. I've been working on a research paper regarding off-policy learning so I've been poking around in there lately. I should be done now, and It'd still be great to have context as a recording option.
Is there are straight-forward way to access the trained model artifacts at the end of an experiment run?
I would like to perform further analyses like looking at the feature importance as well as eventually serialize the models. The result data-structure seems to only contain model meta-data but not the actual artifacts. The learners that are passed into the experiment also don't seem to be updated in place.
I hacked around it by retrieving the model during the training step but hope there's a better way.