search

VowpalWabbit / vowpal_wabbit

Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning with techniques such as online, hashing, allreduce, reductions, learning2search, active, and interactive learning.
https://vowpalwabbit.org
Other
8.49k stars 1.92k forks source link

Oracle generation is broken for LDF-mode in search_covington.ipynb #2175

Open lokitoth opened 4 years ago

lokitoth commented 4 years ago

Some time ago we changed the logic for oracle validation in Python to prevent zeroes from being passed in (see #1296 and #1042). This leads to broken oracle generation. For complex LDF cases, there seems to be additional issues once the oracle generation is fixed that prevents the system from learning correctly.

This issue is a tracking issue for the work to get search to a better place and update the examples to produce the expected output.

lokitoth commented 4 years ago

Looks like at least part of the issue may be due to bad ternary operator as per #2374

jackgerrits commented 2 years ago

@lokitoth Is there a way to reproduce this isssue?

lokitoth commented 2 years ago

This gets reproduced in the search_covington.ipynb notebook: https://github.com/VowpalWabbit/vowpal_wabbit/blob/master/python/docs/source/examples/search_covington.ipynb

In particular, see the end printout:

testing LDF
[1, 2, -1, 1, 2]
should have printed [ 1 2 -1 4 2 ]

There is strong reason to believe that the core of the search algorithm works properly - I spent time inside of the LDF-specific code-paths, and it did not appear that the issue was there. My current suspicion is that it is a problem either in the python binding layer, or the example is improperly put-together.

Updating title to make it clear what is going on.