search

VowpalWabbit / vowpal_wabbit

Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning with techniques such as online, hashing, allreduce, reductions, learning2search, active, and interactive learning.
https://vowpalwabbit.org
Other
8.48k stars 1.93k forks source link

C# (CSharp) interface usage for joint prediction (search) #2524

Open andy-soft opened 4 years ago

andy-soft commented 4 years ago

Description

A brief description of the error, missing documentation or what you would like added I want to use VW from C# and not use the command line, just use a service and feed data in (to train), and pump predictions out The problem is I did not find the C#API, no description of the usage, the Wiki info and the readme are not enough to understand the interfacing models.

I want to use "search engine", and do POS tagging, NER and Dependency Parsing, (all sequential) and don't know how to feed the appropriated formatted multiple strings (of a tagged / splitted sentence) and get the prediction(s) out, The internal API classes (C# project) are complicated to inference from!

I successfully got the multiclass classification to work !

But it still has a BUG => the linked class (the DLL) don't get released (the operating-system file-lock) even after disposing the VW object and removing the whole assembly! So there is a serious threat to put this into a windows service!

Please help!

Link to Documentation Page

Where is the documentation in question?

lokitoth commented 4 years ago

I added a link to an example gist in #2578. Will use this issue to track the work to provide better documentation, and focus on assisting @andy-soft with specific questions in the other one.

Edit: Regarding the issue you mentioned about the DLL not being released: In .NET Framework, assemblies do not get unloaded unless the AppDomain into which it was loaded is unloaded (This functionality, but not other AppDomain concepts, was pulled into AssemblyLoadContext in newer versions of .NET)

andy-soft commented 4 years ago

Thank you! I am testing it right now! (and it seems to work!) But.. I found no information on the documentation to allow me do this construct!!! Thanks a lot! Just I am testing the NER parser (in Spanish) and it overfits a lot, it's really fast but commits lots of errors! F1 < 60% when others reach 92% on same dataset. (Ancora) I also added lots of features on the peature space, not only word and POS

andy-soft commented 3 years ago

Hi there, it worked fine! There are some issues out there, ¿may I can get some additional help?

I Want to make use from ..NET in C# of the reinforcement and online learning features,

For this, to work fine I need the ability to create in C# the functions for the GOLD-Standard (Oracle) to be useful, and even to train online, but in Python there are plenty of samples of the function, but unfortunately under .NET (C# interface API)

I did not manage how to link this function, just because my joint-prediction will be context-aware I want to parse natural language using a dynamic semantic-context I do heavy NLP research but unfortunately did not find how to build this linking on .NET.

¿Can you provide me a sample like the other, which I found really enlightening?

I am managing to use the C# interface for training even (multiline) backed out of what you sent me, when ready I'll share it with you. What feels disappointing is that VW quality CLAIMS on POS-Tagging and NER / NEC extraction are somehow not entirely true.

Last but not least, the claimed on most papers (around 2016) F-Score of the NER and POS tagger are false.

I tested now, (in C# .NET), with and the SAME Ancora dataset, the resulting score does not hold the claims, or simply is not true, (F-Score it's far lower, around 60%) may I be doing something wrong?

The library seems NOT TO GENERALIZE WELL in SPANISH, and despite the apparent scores of the training are high, the use with other data fails, and the recognition of tags is poor, no to say bad! ¿Maybe the library is overfitting too much?

I guess I am training well, on a well-converted dataset towards the vw format, and the loss goes really down! Buth the predictions on data outside the dataset are pretty bad!

Tried lots of parameter and combinations, like cubic, and cuadratic relation on multiple namespaces, suffixes, etc. none helped! ¿Is there a way to measure the overfitting capability, apart from the loss factor provided?

is there a real POS-Tagger or NER/NEC extractor with an example to reproduce (including dataset) to see what may I be doing wrong?

thanks anyway!