search

VowpalWabbit / vowpal_wabbit

Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning with techniques such as online, hashing, allreduce, reductions, learning2search, active, and interactive learning.
https://vowpalwabbit.org
Other
8.49k stars 1.93k forks source link

Where has mtr gone? #1541

Closed matanox closed 2 years ago

matanox commented 6 years ago

Can we still use Importance Weighted Multitask Regression as in this tutorial?

image

Right now, mtr is not allowed as a cb_type (version 8.5.0 at least).

warning: cb_type must be in {'ips','dm','dr'}; resetting to dr.

Or is it restricted to a subset of CB scenarios?

JohnLangford commented 6 years ago

mtr works for --cb_adf. We don't have an implementation for --cb at the moment. This is a 'patches welcome' thing.

-John

On Sun, Jul 22, 2018 at 11:43 AM, matanster notifications@github.com wrote:

Can we still use Importance Weighted Multitask Regression as in this tutorial?

[image: image] https://user-images.githubusercontent.com/4996067/43047375-15a6533a-8ddf-11e8-9105-f28a98f8004f.png

Right now, mtr is not allowed as a cb_type (version 8.5.0 at least).

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/JohnLangford/vowpal_wabbit/issues/1541, or mute the thread https://github.com/notifications/unsubscribe-auth/AAE25ucHTU3JCDTfvZgyUE_58uWFXaetks5uJJ29gaJpZM4VaCpD .

matanox commented 6 years ago

Thanks a lot!

@JohnLangford would you generally recommend switching to ADF for this, if we don't have features per (context, action) pair in our domain, only features shared across all actions? would the models train just as well switching to ADF, if we've not identified features to be associated at a (context, action) level, or will it potentially undermine the learning in that case?

Sorry for being a little vocal on the mailing list, but we're making really good progress in our usage reducing from novel domains/problems into the framework of Contextual Bandits using VW. I've watched the latest tutorial of course, and have plans to go deeper into the theory following Alekh's course materials and other curated resources.

Matan

Changing action set or featurized actions

In many applications, the set of actions is richer in that it changes over time or we have rich information regarding each action. In either of these cases, it is a good idea to create features for every (context, action) pair rather than features associated only with context and shared across all actions. We call this setting ADF, and it builds on the csoaa_ldf learner that supports cost-sensitive classification with label-dependent features. An example dataset for this setting might look like:

JohnLangford commented 5 years ago

mtr is pretty good in my experience. Yes, I'd suggest switching to ADF for this.

In general, we probably want to modify the non-adf implementation to use the adf implementation which is more general.

bassmang commented 2 years ago

This is resolved by using cb_to_cb_adf by default and giving a warning to switch to dr when --cb_force_legacy is used.