When trying to do multiline structural learning (search) for NER the C# interface fail on predictions!

andy-soft commented 4 years ago

Describe the bug

I am using C# interface to WV version 8/4 as well as 8.7 under windows

To Reproduce

Steps to reproduce the behavior: Here is the training var settings = new VowpalWabbitSettings(); int bits = 24; var argx = $"-k -c -b {bits} --invariant --passes {passes} --search_rollout none --search_task sequencespan --search 9 --holdout_off"; vn = new VowpalWabbit(argx); var sett = vn.Settings; var args = vn.Arguments; // -d { file} var lines = file.ReadLines(); var sents = new List<string[]>(); List sent = new List(); foreach (string s in lines) { if (s.isEmpty()) { sents.Add(sent.ToArray()); sent = new List(); } sent.Add(s); } foreach (var sen in sents) vn.Learn(sen); vn.RunMultiPass(); var stat = vn.PerformanceStatistics; vn.SaveModel($"{path}\esp_ddl.model");

Expected behavior

A clear and concise description of what you expected to happen. I cannot get the WV to get structured predictions from C# programmatically (only command line wise)

Observed Behavior

How did VW behave? Please include any stack trace, log messages or crash logs.

the Exception error thrown is: "Tried to use a multiline reduction as a singleline reduction"

Environment

VW 8.6.1 on Windows with the C# bindings`

Additional context

Add any other context about the problem here.

C# CODE: var settings = new VowpalWabbitSettings(); var nam = "test.txt"; var infile = $"{nam}"; var outfile = $"{nam}.pred"; var argx = $" -t -i {path}\esp_ddl.model -d {infile} -p {outfile}"; vn = new VowpalWabbit(argx); var sett = vn.Settings; var args = vn.Arguments; // -d { file} var lines = file.ReadLines(); var sents = new List<string[]>(); var sent = new List(); foreach (string s in lines) { if (s.isEmpty()) { sents.Add(sent.ToArray());
IT FAILS HERE ----> var pred = vn.Predict(arr, VowpalWabbitPredictionType.xxxxx); (on any option it fails!) the Exception error thrown is: "Tried to use a multiline reduction as a singleline reduction" sent = new List();

                }
                sent.Add(s);
            }

` And on the command line it does work m and generates the correct Predictions!!!

How could I get the predictions from within C# on this "multiline reduction problem"

please help!

lokitoth commented 4 years ago

Hi @andy-soft. Sorry for the delay getting to this. I am currently looking at this issue.

lokitoth commented 4 years ago

Hi @andy-soft: The issue here stems from there not being a set of helpers for Search (like there are for CB/ADF), which means you have to use the lower-level API to extract predictions.

Here is an example (built on Tests 48/49 from RunTests, which are similar to what you are trying to achieve): https://gist.github.com/lokitoth/bd8e9e6f1c1be57c0cc320e638ce6b29

For an input file that looks like:

2 | I
1 | really
1 | like
2 | my
2 | mother
1 | ,
6 | the
7 | United
7 | States
7 | of
7 | America
1 | and
6 | it's
4 | Congress
1 | .

Starting with passes = 2, it will have the expected result 2 1 1 2 2 1 6 7 7 7 7 1 6 4 1

Does this help resolve your issue?

VowpalWabbit / vowpal_wabbit