feat(agent):Add okapibm25 agent

fossology / atarashi

Atarashi scans for license statements in open source software, focusing on text statistics. Designed to work stand-alone and with FOSSology.

http://fossology.github.io/atarashi

GNU General Public License v2.0

26 stars 23 forks source link

feat(agent):Add okapibm25 agent #101

Open its-sushant opened 2 years ago

its-sushant commented 2 years ago

Description

Add a agent for bm25Okapi agent.

Files

agents/bm25.py

How to use

Running bm25 agent atarashi -a bm25 /path/to/file.c
Checking accuracy using evaluator.py python3 path/to/evaluator.py -a bm25

Accuracy

The accuracy score that has been tested using evaluator.py is 62%

CC: @Kaushl2208 @GMishx @hastagAB @ag4ums @vasudevmaduri

GMishx commented 2 years ago

Thank you for your contribution @its-sushant but the library rank-bm25 is Apache-2.0 licensed which is not compatible with GPL-2.0 license of Atarashi.

Please find an alternate of the library which is not GPL-3.0, Apache- and AGPL-.

hastagAB commented 2 years ago

Thank you for your contribution @its-sushant but the library rank-bm25 is Apache-2.0 licensed which is not compatible with GPL-2.0 license of Atarashi.

Please find an alternate of the library which is not GPL-3.0, Apache- and AGPL-.

@GMishx @its-sushant Can we implement BM25 algo in-house for Atarashi?

its-sushant commented 2 years ago

Thank you for your contribution @its-sushant but the library rank-bm25 is Apache-2.0 licensed which is not compatible with GPL-2.0 license of Atarashi. Please find an alternate of the library which is not GPL-3.0, Apache- and AGPL-.

@GMishx @its-sushant Can we implement BM25 algo in-house for Atarashi?

Sure, we can implement bm25 transformer itself in Atarashi. I have already started working on it and will revert the changes as soon as possible.