search

MantisAI / nervaluate

Full named-entity (i.e., not tag/token) evaluation metrics based on SemEval’13
MIT License
154 stars 19 forks source link

Add command line interface #48

Closed Eleni170 closed 2 years ago

Eleni170 commented 3 years ago

This branch focuses on the creation of a client that can run in command line the evaluator. A standard rvl client is used and it is attempted to be triggered using the command:

rvl nervaluate model_path data_path

The example data that was used are the following:

{"text": "Apple is looking at buying U.K. startup for $1 billion", "meta":[{"label":"ORG", "start":0, "end":5},{"label":"GPE", "start":27, "end":31},{"label":"MONEY", "start":44, "end":54}]} {"text": "Zuckemberg remaims the Undisputed boss at Facebook.", "meta":[{"label":"PERSON", "start":0, "end":1}, {"label":"ORG", "start":6, "end":7}]}

the results of the evaluation should be added at the 2 json files in the results folder.

Feel free to make comments about the implementation.

Eleni170 commented 3 years ago

Ok so I made some of the changes that @ivyleavedtoadflax suggested. Since there is no logic to create the lists, I removed spaCy for now. I also have to consider passing into the function the lists in a better way than being in a dictionary in a data_path.