hastagAB commented 4 years ago

Description

Added Nirjas for code comment extraction. Nirjas was created under FOSSology primarily to replace the existing code comment extractor
It extracts and passes only the license part out of all comments to atarashi
Evaluator file is fixed and updated
Evaluator TestFiles are zipped and during the testing, it will be unzipped and zipped again
.gitignore file is added

GMishx commented 4 years ago

@hastagAB , can you please update pyproject.toml and add Nirjas to it. You can remove the pip install -r requirements.txt from .travis.yml

hastagAB commented 4 years ago

@hastagAB , can you please update pyproject.toml and add Nirjas to it. You can remove the pip install -r requirements.txt from .travis.yml

@GMishx Thanks, I totally missed pyproject.toml. I've modified the required changes, please review.

hastagAB commented 4 years ago

Few changes, commented on every line to make sure we don't miss anything.

Most of the try except block was for testing only, forgot to remove those from the PR. Removed a few unwanted try except. The evaluator takes the output from the CLI itself, so I didn't raised error there on purpose.

hastagAB commented 4 years ago

Tested, Working fine on my system. @Kaushl2208 Please Test the changes and merge this PR.

Kaushl2208 commented 4 years ago

@hastagAB Tested. Everything seems to work fine on my system too.

Also, I don't have merge access.

CC: @GMishx @ag4ums

GMishx commented 4 years ago

I see a new commit @hastagAB. Is it just a squash commit or something changed?

hastagAB commented 4 years ago

I see a new commit @hastagAB. Is it just a squash commit or something changed?

Yes, I squashed the commits so that we can merge.

GMishx commented 4 years ago

I am getting follow error. Any idea why that is happening for me and not on travis?

$ atarashi --help
Traceback (most recent call last):
  File "/home/gaurav/development/atarashi/.venv/bin/atarashi", line 5, in <module>
    from atarashi.atarashii import main
  File "/home/gaurav/development/atarashi/.venv/lib/python3.7/site-packages/atarashi/atarashii.py", line 28, in <module>
    from atarashi.agents.tfidf import TFIDF
  File "/home/gaurav/development/atarashi/.venv/lib/python3.7/site-packages/atarashi/agents/tfidf.py", line 31, in <module>
    from sklearn.feature_extraction.text import TfidfVectorizer
  File "/home/gaurav/development/atarashi/.venv/lib/python3.7/site-packages/sklearn/__init__.py", line 80, in <module>
    from .base import clone
  File "/home/gaurav/development/atarashi/.venv/lib/python3.7/site-packages/sklearn/base.py", line 21, in <module>
    from .utils import _IS_32BIT
  File "/home/gaurav/development/atarashi/.venv/lib/python3.7/site-packages/sklearn/utils/__init__.py", line 20, in <module>
    from scipy.sparse import issparse
  File "/home/gaurav/development/atarashi/.venv/lib/python3.7/site-packages/scipy/__init__.py", line 68, in <module>
    from ._lib.deprecation import _deprecated
ModuleNotFoundError: No module named 'scipy._lib'

hastagAB commented 4 years ago

I am getting follow error. Any idea why that is happening for me and not on travis?

$ atarashi --help
Traceback (most recent call last):
  File "/home/gaurav/development/atarashi/.venv/bin/atarashi", line 5, in <module>
    from atarashi.atarashii import main
  File "/home/gaurav/development/atarashi/.venv/lib/python3.7/site-packages/atarashi/atarashii.py", line 28, in <module>
    from atarashi.agents.tfidf import TFIDF
  File "/home/gaurav/development/atarashi/.venv/lib/python3.7/site-packages/atarashi/agents/tfidf.py", line 31, in <module>
    from sklearn.feature_extraction.text import TfidfVectorizer
  File "/home/gaurav/development/atarashi/.venv/lib/python3.7/site-packages/sklearn/__init__.py", line 80, in <module>
    from .base import clone
  File "/home/gaurav/development/atarashi/.venv/lib/python3.7/site-packages/sklearn/base.py", line 21, in <module>
    from .utils import _IS_32BIT
  File "/home/gaurav/development/atarashi/.venv/lib/python3.7/site-packages/sklearn/utils/__init__.py", line 20, in <module>
    from scipy.sparse import issparse
  File "/home/gaurav/development/atarashi/.venv/lib/python3.7/site-packages/scipy/__init__.py", line 68, in <module>
    from ._lib.deprecation import _deprecated
ModuleNotFoundError: No module named 'scipy._lib'

I'm not sure, I haven't encountered this error till now. Maybe you have an older (python2) version of SciPy.

GMishx commented 4 years ago

I'm not sure, I haven't encountered this error till now. Maybe you have an older (python2) version of SciPy.

This was issue with my virtual environment. Just discarded my venv and installed again and it worked.

GMishx commented 4 years ago

@hastagAB , please squash the commits for PR merge.

fossology / atarashi

feat(Nirjas): Replace code comment extractor with Nirjas #68

Description