VowpalWabbit / coba

Contextual bandit benchmarking
https://coba-docs.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
48 stars 19 forks source link

VowpalWabbit flags on the labels #2

Closed ataymano closed 3 years ago

ataymano commented 3 years ago

Hi,

thanks for great project!

Is it possible to add vw flags to plot labels?

Right now I am seeing that kind of plot: image

when I am doing something like this:

    learner_factories = [
        VowpalLearner(epsilon=0.2, seed=10, flags=''),
        VowpalLearner(epsilon=0.2, seed=10, flags='--coin')
        ]
    benchmark.evaluate(learner_factories).standard_plot()

and it is hard to correlate plots with configurations.

mrucker commented 3 years ago

Glad you like it. I can totally do that. I'll commit and release an update for it today.

Also, just so you know, since we last talked I released an update for COBA that's got a lot of new features which make it easier to add custom extensions to benchmarks. I'll be adding more complete documentation for the new stuff in the next few weeks.

mrucker commented 3 years ago

Commit aa2c2a8f0fcd3c515de17c62e14403c565dec570 contains this improvement. I've also released this to PyPi in version 2.2.1.

mrucker commented 3 years ago

(or if you want to do it yourself in your own fork you can change the following method in VowpalLearner.)

class VowpalLearner:
    ...
    @property
    def params(self) -> Dict[str, Any]:

        params = {**self._learning.params(), **self._exploration.params()}

        if self._flags != '':
            params['flags'] = self._flags

        return params
ataymano commented 3 years ago

thank you for quick response!