Missing metrics in database

[Hk-tang]

When running the scripts from a clean install an error is encountered where the error: librarycomparison.models.DoesNotExist: Metric matching query does not exist. is given. The solution is in #39 and this is related to issue #27

(venv) smr@smr-Inspiron-3668:/media/smr/7EA0E9174405BFDC/Projects/Python/LibraryMetricScripts$ python -m scripts
Retrieving list of top repos... 

pushed:>2019-09-09 language:Java
The rate limit is 30
You have 30/30 API calls remaining
Base query: pushed:>2019-09-09 language:Java
Collected  0  repos so far
pushed:>2019-09-09 language:Java stars:>100
Writing  10  repos to file

Finally ..... Execution is over 

Searching for imports in top repos...
1 out of 10 Query : "import org.springframework.security" in:file language:Java repo:Netflix/Hystrix
2 out of 10 Query : "import org.springframework.security" in:file language:Java repo:alibaba/druid
3 out of 10 Query : "import org.springframework.security" in:file language:Java repo:scwang90/SmartRefreshLayout
4 out of 10 Query : "import org.springframework.security" in:file language:Java repo:SeleniumHQ/selenium
5 out of 10 Query : "import org.springframework.security" in:file language:Java repo:alibaba/fastjson
6 out of 10 Query : "import org.springframework.security" in:file language:Java repo:CymChad/BaseRecyclerViewAdapterHelper
7 out of 10 Query : "import org.springframework.security" in:file language:Java repo:eugenp/tutorials
8 out of 10 Query : "import org.springframework.security" in:file language:Java repo:ReactiveX/RxAndroid
9 out of 10 Query : "import org.springframework.security" in:file language:Java repo:NationalSecurityAgency/ghidra
10 out of 10 Query : "import org.springframework.security" in:file language:Java repo:ctripcorp/apollo

 Finally ..... Execution is over 

Getting release data for  shiro
Getting release data for  spring-security
Loading data
Getting license for  shiro
Getting license for  spring-security
Loading last modification data
/media/smr/7EA0E9174405BFDC/Projects/Python/librarycomparisonswebsite/venv/lib/python3.8/site-packages/sklearn/feature_extraction/text.py:383: UserWarning: Your stop_words may be inconsistent with your preprocessing. Tokenizing the stop words generated tokens ['abov', 'afterward', 'alon', 'alreadi', 'alway', 'ani', 'anoth', 'anyon', 'anyth', 'anywher', 'becam', 'becaus', 'becom', 'befor', 'besid', 'cri', 'describ', 'dure', 'els', 'elsewher', 'empti', 'everi', 'everyon', 'everyth', 'everywher', 'fifti', 'formerli', 'forti', 'ha', 'henc', 'hereaft', 'herebi', 'hi', 'howev', 'hundr', 'inde', 'latterli', 'mani', 'meanwhil', 'moreov', 'mostli', 'nobodi', 'noon', 'noth', 'nowher', 'onc', 'onli', 'otherwis', 'ourselv', 'perhap', 'pleas', 'seriou', 'sever', 'sinc', 'sincer', 'sixti', 'someon', 'someth', 'sometim', 'somewher', 'themselv', 'thenc', 'thereaft', 'therebi', 'therefor', 'thi', 'thu', 'togeth', 'twelv', 'twenti', 'veri', 'wa', 'whatev', 'whenc', 'whenev', 'wherea', 'whereaft', 'wherebi', 'wherev', 'whi', 'yourselv'] not in stop_words.
  warnings.warn('Your stop_words may be inconsistent with '
/media/smr/7EA0E9174405BFDC/Projects/Python/librarycomparisonswebsite/venv/lib/python3.8/site-packages/sklearn/feature_extraction/text.py:383: UserWarning: Your stop_words may be inconsistent with your preprocessing. Tokenizing the stop words generated tokens ['abov', 'afterward', 'alon', 'alreadi', 'alway', 'ani', 'anoth', 'anyon', 'anyth', 'anywher', 'becam', 'becaus', 'becom', 'befor', 'besid', 'cri', 'describ', 'dure', 'els', 'elsewher', 'empti', 'everi', 'everyon', 'everyth', 'everywher', 'fifti', 'formerli', 'forti', 'ha', 'henc', 'hereaft', 'herebi', 'hi', 'howev', 'hundr', 'inde', 'latterli', 'mani', 'meanwhil', 'moreov', 'mostli', 'nobodi', 'noon', 'noth', 'nowher', 'onc', 'onli', 'otherwis', 'ourselv', 'perhap', 'pleas', 'seriou', 'sever', 'sinc', 'sincer', 'sixti', 'someon', 'someth', 'sometim', 'somewher', 'themselv', 'thenc', 'thereaft', 'therebi', 'therefor', 'thi', 'thu', 'togeth', 'twelv', 'twenti', 'veri', 'wa', 'whatev', 'whenc', 'whenev', 'wherea', 'whereaft', 'wherebi', 'wherev', 'whi', 'yourselv'] not in stop_words.
  warnings.warn('Your stop_words may be inconsistent with '
Getting JIRA issue data
Current repository in jira issues:  shiro
Getting GitHub issue data
========Current repository:  spring-projects/spring-security
Getting breaking changes for  shiro
Cloned repo
Calculating for  SHIRO-121_pre_commit
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000300000000, 19327352832, 0) failed; error='Not enough space' (errno=12)
b'#\n# There is insufficient memory for the Java Runtime Environment to continue.\n# Native memory allocation (mmap) failed to map 19327352832 bytes for committing reserved memory.\n# An error report file with more information is saved as:\n# /media/smr/7EA0E9174405BFDC/Projects/Python/LibraryMetricScripts/hs_err_pid15824.log\n'
Getting breaking changes for  spring-security
Cloned repo
Calculating for  1.0.1
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000300000000, 19327352832, 0) failed; error='Not enough space' (errno=12)
b'#\n# There is insufficient memory for the Java Runtime Environment to continue.\n# Native memory allocation (mmap) failed to map 19327352832 bytes for committing reserved memory.\n# An error report file with more information is saved as:\n# /media/smr/7EA0E9174405BFDC/Projects/Python/LibraryMetricScripts/hs_err_pid15829.log\n'
Filling popularity...
DID NOT CREATE new entry for: spring-security
Calculating release frequency...
calculating release frequency for library..  shiro
calculating release frequency for library..  spring-security
Calculating breaking changes...
Filling last modification data...
Loading last modification data
Filling last discussed on SO....
getting last discussed date for shiro
getting last discussed date for spring-security
Filling license data...
Loading data
Calculating issue data...
Calculating overall score...
Creating charts...
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/media/smr/7EA0E9174405BFDC/Projects/Python/LibraryMetricScripts/scripts/__main__.py", line 4, in <module>
    app.run()
  File "/media/smr/7EA0E9174405BFDC/Projects/Python/LibraryMetricScripts/scripts/app.py", line 46, in run
    filldb()
  File "/media/smr/7EA0E9174405BFDC/Projects/Python/LibraryMetricScripts/scripts/filldb.py", line 652, in filldb
    createCharts()
  File "/media/smr/7EA0E9174405BFDC/Projects/Python/LibraryMetricScripts/scripts/filldb.py", line 625, in createCharts
    create_popularity_chart(domain)
  File "/media/smr/7EA0E9174405BFDC/Projects/Python/LibraryMetricScripts/scripts/filldb.py", line 79, in create_popularity_chart
    save_chart_in_db(line_chart, domain, metric_name="popularity")
  File "/media/smr/7EA0E9174405BFDC/Projects/Python/LibraryMetricScripts/scripts/filldb.py", line 84, in save_chart_in_db
    metric = Metric.objects.get(name=metric_name)
  File "/media/smr/7EA0E9174405BFDC/Projects/Python/librarycomparisonswebsite/venv/lib/python3.8/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/media/smr/7EA0E9174405BFDC/Projects/Python/librarycomparisonswebsite/venv/lib/python3.8/site-packages/django/db/models/query.py", line 429, in get
    raise self.model.DoesNotExist(
librarycomparison.models.DoesNotExist: Metric matching query does not exist.

[snadi]

@XiaoleZ I believe you also ran into this issue and fixed it right? Is it fixed in the Dockerfile PR? (I still need to test it out on the server which is why I didn't merge it but I'll likely be doing that in May)

[XiaoleZ]

@snadi yes. There is a script creating the tables (Metric) and import values (security, etc..)

[snadi]

This issue should be fixed with the eventual merging of PR #43