Multiclass Classification not working through C# wrapper?

[HappyCoderMan]

I'm using the C# wrapper, and multiclass classification seems to fail (or I'm using it wrong).

I've written my records to text, and I can get the vw.exe to work. The training and predictions appear valid. If I use a non-multiclass classification, VW instance then the C# wrapper works great and I get valid training and predictions.

I'm using VW C# very similar to the examples here: https://github.com/JohnLangford/vowpal_wabbit/blob/master/cs_test/Program.cs

I call this: VWNative.Initialize with a string like this: --oaa 50 -f model.vw --cache_file cache

Then, I build an example: IntPtr example = VWNative.ImportExample(vw, label, pinnedFeatureSpace.AddrOfPinnedObject(), featureSpaceCount);

Then, I call Learn: float score = VWNative.Learn(vw, example); But, score is a terrible value.

I've tried ect, oaa, and csoaa. They each behave this same way.

I've tried to train several thousand examples training and predicting, and they all behave the same.

Thank you for any help or advise you can provide.

[JohnLangford]

I don't think the Pinvoke wrapper is setup to do multiclass labels, so it ends up reading a uin32_t as a float.

@eisber, does the CLI interface support multiclass labels? In general, redoing the cs_test stuff with CLI seems like the thing to do if we are going to shift to the new interface.

-John

On 10/31/2015 01:55 AM, ErikOmberg wrote:

I'm using the C# wrapper, and multiclass classification seems to fail (or I'm using it wrong).

I've written my records to text, and I can get the vw.exe to work. The training and predictions appear valid. If I use a non-multiclass classification, VW instance then the C# wrapper works great and I get valid training and predictions.

I'm using VW C# very similar to the examples here: https://github.com/JohnLangford/vowpal_wabbit/blob/master/cs_test/Program.cs

I call this: VWNative.Initialize with a string like this: --oaa 50 -f model.vw --cache_file cache

Then, I build an example: IntPtr example = VWNative.ImportExample(vw, label, pinnedFeatureSpace.AddrOfPinnedObject(), featureSpaceCount);

Then, I call Learn: float score = VWNative.Learn(vw, example); But, score is a terrible value. image https://cloud.githubusercontent.com/assets/8632110/10862113/bb07c594-7f68-11e5-9162-490c401d7e23.png

I've tried ect, oaa, and csoaa. They each behave this same way.

I've tried to train several thousand examples training and predicting, and they all behave the same.

Thank you for any help or advise you can provide.

— Reply to this email directly or view it on GitHub https://github.com/JohnLangford/vowpal_wabbit/issues/850.

[eisber]

So the CLI wrapper supports arrays of int through VowpalWabbitMultilabelPredictionFactory. Not sure how much it’s tested though, but the actual code is minimal. You might want to take a look at https://www.nuget.org/packages/VowpalWabbit/ and https://github.com/JohnLangford/vowpal_wabbit/wiki/C%23-Binding (though the wiki is slightly outdated, but should give you an idea).

Markus

From: John Reply-To: JohnLangford/vowpal_wabbit Date: Saturday, October 31, 2015 at 15:24 To: JohnLangford/vowpal_wabbit Cc: Markus Cozowicz Subject: Re: [vowpal_wabbit] Multiclass Classification not working through C# wrapper? (#850)

I don't think the Pinvoke wrapper is setup to do multiclass labels, so it ends up reading a uin32_t as a float.

@eisber, does the CLI interface support multiclass labels? In general, redoing the cs_test stuff with CLI seems like the thing to do if we are going to shift to the new interface.

-John

On 10/31/2015 01:55 AM, ErikOmberg wrote:

I'm using the C# wrapper, and multiclass classification seems to fail (or I'm using it wrong).

I've written my records to text, and I can get the vw.exe to work. The training and predictions appear valid. If I use a non-multiclass classification, VW instance then the C# wrapper works great and I get valid training and predictions.

I'm using VW C# very similar to the examples here: https://github.com/JohnLangford/vowpal_wabbit/blob/master/cs_test/Program.cs

I call this: VWNative.Initialize with a string like this: --oaa 50 -f model.vw --cache_file cache

Then, I build an example: IntPtr example = VWNative.ImportExample(vw, label, pinnedFeatureSpace.AddrOfPinnedObject(), featureSpaceCount);

Then, I call Learn: float score = VWNative.Learn(vw, example); But, score is a terrible value. image https://cloud.githubusercontent.com/assets/8632110/10862113/bb07c594-7f68-11e5-9162-490c401d7e23.png

I've tried ect, oaa, and csoaa. They each behave this same way.

I've tried to train several thousand examples training and predicting, and they all behave the same.

Thank you for any help or advise you can provide.

— Reply to this email directly or view it on GitHub https://github.com/JohnLangford/vowpal_wabbit/issues/850.

— Reply to this email directly or view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fgithub.com%2fJohnLangford%2fvowpal_wabbit%2fissues%2f850%23issuecomment-152765287&data=01%7c01%7cmarcozo%40064d.mgd.microsoft.com%7c92e207b3a5b54eb5565f08d2e228db8f%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=nuOshvYhHkuPqxA%2f5GyuDC7lT%2fgxKh3RJjHyDb0lo7Q%3d.

[JohnLangford]

The type of labels for multiclass is uint32_t not array of.

-John

On 10/31/2015 03:52 PM, Markus Cozowicz wrote:

So the CLI wrapper supports arrays of int through VowpalWabbitMultilabelPredictionFactory. Not sure how much it’s tested though, but the actual code is minimal. You might want to take a look at https://www.nuget.org/packages/VowpalWabbit/ and https://github.com/JohnLangford/vowpal_wabbit/wiki/C%23-Binding (though the wiki is slightly outdated, but should give you an idea).

Markus

From: John Reply-To: JohnLangford/vowpal_wabbit Date: Saturday, October 31, 2015 at 15:24 To: JohnLangford/vowpal_wabbit Cc: Markus Cozowicz Subject: Re: [vowpal_wabbit] Multiclass Classification not working through C# wrapper? (#850)

I don't think the Pinvoke wrapper is setup to do multiclass labels, so it ends up reading a uin32_t as a float.

@eisber, does the CLI interface support multiclass labels? In general, redoing the cs_test stuff with CLI seems like the thing to do if we are going to shift to the new interface.

-John

On 10/31/2015 01:55 AM, ErikOmberg wrote:

I'm using the C# wrapper, and multiclass classification seems to fail (or I'm using it wrong).

I've written my records to text, and I can get the vw.exe to work. The training and predictions appear valid. If I use a non-multiclass classification, VW instance then the C# wrapper works great and I get valid training and predictions.

I'm using VW C# very similar to the examples here:

https://github.com/JohnLangford/vowpal_wabbit/blob/master/cs_test/Program.cs

I call this: VWNative.Initialize with a string like this: --oaa 50 -f model.vw --cache_file cache

Then, I build an example: IntPtr example = VWNative.ImportExample(vw, label, pinnedFeatureSpace.AddrOfPinnedObject(), featureSpaceCount);

Then, I call Learn: float score = VWNative.Learn(vw, example); But, score is a terrible value. image

https://cloud.githubusercontent.com/assets/8632110/10862113/bb07c594-7f68-11e5-9162-490c401d7e23.png

I've tried ect, oaa, and csoaa. They each behave this same way.

I've tried to train several thousand examples training and predicting, and they all behave the same.

Thank you for any help or advise you can provide.

— Reply to this email directly or view it on GitHub https://github.com/JohnLangford/vowpal_wabbit/issues/850.

— Reply to this email directly or view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fgithub.com%2fJohnLangford%2fvowpal_wabbit%2fissues%2f850%23issuecomment-152765287&data=01%7c01%7cmarcozo%40064d.mgd.microsoft.com%7c92e207b3a5b54eb5565f08d2e228db8f%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=nuOshvYhHkuPqxA%2f5GyuDC7lT%2fgxKh3RJjHyDb0lo7Q%3d.

— Reply to this email directly or view it on GitHub https://github.com/JohnLangford/vowpal_wabbit/issues/850#issuecomment-152766590.

[eisber]

I misread multi label vs multi class. Thus we don’t support multilclass yet, but it can easily be added by adding another prediction factory to vw_prediction.h and vw_prediction.cpp.

Markus

From: John Reply-To: JohnLangford/vowpal_wabbit Date: Saturday, October 31, 2015 at 16:51 To: JohnLangford/vowpal_wabbit Cc: Markus Cozowicz Subject: Re: [vowpal_wabbit] Multiclass Classification not working through C# wrapper? (#850)

The type of labels for multiclass is uint32_t not array of.

-John

On 10/31/2015 03:52 PM, Markus Cozowicz wrote:

So the CLI wrapper supports arrays of int through VowpalWabbitMultilabelPredictionFactory. Not sure how much it’s tested though, but the actual code is minimal. You might want to take a look at https://www.nuget.org/packages/VowpalWabbit/ and https://github.com/JohnLangford/vowpal_wabbit/wiki/C%23-Binding (though the wiki is slightly outdated, but should give you an idea).

Markus

From: John Reply-To: JohnLangford/vowpal_wabbit Date: Saturday, October 31, 2015 at 15:24 To: JohnLangford/vowpal_wabbit Cc: Markus Cozowicz Subject: Re: [vowpal_wabbit] Multiclass Classification not working through C# wrapper? (#850)

I don't think the Pinvoke wrapper is setup to do multiclass labels, so it ends up reading a uin32_t as a float.

@eisber, does the CLI interface support multiclass labels? In general, redoing the cs_test stuff with CLI seems like the thing to do if we are going to shift to the new interface.

-John

On 10/31/2015 01:55 AM, ErikOmberg wrote:

I'm using the C# wrapper, and multiclass classification seems to fail (or I'm using it wrong).

I've written my records to text, and I can get the vw.exe to work. The training and predictions appear valid. If I use a non-multiclass classification, VW instance then the C# wrapper works great and I get valid training and predictions.

I'm using VW C# very similar to the examples here:

https://github.com/JohnLangford/vowpal_wabbit/blob/master/cs_test/Program.cs

I call this: VWNative.Initialize with a string like this: --oaa 50 -f model.vw --cache_file cache

Then, I build an example: IntPtr example = VWNative.ImportExample(vw, label, pinnedFeatureSpace.AddrOfPinnedObject(), featureSpaceCount);

Then, I call Learn: float score = VWNative.Learn(vw, example); But, score is a terrible value. image

https://cloud.githubusercontent.com/assets/8632110/10862113/bb07c594-7f68-11e5-9162-490c401d7e23.png

I've tried ect, oaa, and csoaa. They each behave this same way.

I've tried to train several thousand examples training and predicting, and they all behave the same.

Thank you for any help or advise you can provide.

— Reply to this email directly or view it on GitHub https://github.com/JohnLangford/vowpal_wabbit/issues/850.

— Reply to this email directly or view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fgithub.com%2fJohnLangford%2fvowpal_wabbit%2fissues%2f850%23issuecomment-152765287&data=01%7c01%7cmarcozo%40064d.mgd.microsoft.com%7c92e207b3a5b54eb5565f08d2e228db8f%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=nuOshvYhHkuPqxA%2f5GyuDC7lT%2fgxKh3RJjHyDb0lo7Q%3d.

— Reply to this email directly or view it on GitHub https://github.com/JohnLangford/vowpal_wabbit/issues/850#issuecomment-152766590.

— Reply to this email directly or view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fgithub.com%2fJohnLangford%2fvowpal_wabbit%2fissues%2f850%23issuecomment-152771487&data=01%7c01%7cmarcozo%40064d.mgd.microsoft.com%7cdfe9b06fe46141cdb43f08d2e2351693%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=3%2bUDsdmBRJlpzAmcypn9MwINvnGOqMT7FrfahuFsSqM%3d.

[HappyCoderMan]

Thank you very much for the helpful response.

I added this code to vwdll.cpp and .h and got the behavior I wanted after a rebuild: VW_DLL_MEMBER float VW_CALLING_CONV VW_PredictCostSensitive(VW_HANDLE handle, VW_EXAMPLE e) { vw * pointer = static_cast<vw>(handle); example * ex = static_cast<example>(e); pointer->l->predict(*ex); return VW::get_cost_sensitive_prediction(ex); }

[eisber]

Just for reference the new API already supports this: VowpalWabbitPredictionType.CostSensitive

From: ErikOmberg [mailto:notifications@github.com] Sent: Monday, November 2, 2015 9:13 PM To: JohnLangford/vowpal_wabbit vowpal_wabbit@noreply.github.com Cc: Markus Cozowicz marcozo@microsoft.com Subject: Re: [vowpal_wabbit] Multiclass Classification not working through C# wrapper? (#850)

Thank you very much for the helpful response.

I added this code to vwdll.cpp and .h and got the behavior I wanted after a rebuild: VW_DLL_MEMBER float VW_CALLING_CONV VW_PredictCostSensitive(VW_HANDLE handle, VW_EXAMPLE e) { vw * pointer = static_cast(handle); example * ex = static_cast(e); pointer->l->predict(*ex); return VW::get_cost_sensitive_prediction(ex); }

— Reply to this email directly or view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fgithub.com%2fJohnLangford%2fvowpal_wabbit%2fissues%2f850%23issuecomment-153217948&data=01%7c01%7cmarcozo%40064d.mgd.microsoft.com%7c767df5bbdcdc4cb70ee708d2e3f45f2e%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=y6hWCYjiBPcfbhRm9JUwaw8O7bi%2b4%2fqEIKTI1SJHvOA%3d.

[JohnLangford]

Erik, can you send a pull request?

[HappyCoderMan]

Pull request sent.