search

chaoss / grimoirelab-elk

GNU General Public License v3.0
60 stars 121 forks source link

[qm-elk] Add support for QM data enrichment #926

Closed vchrombie closed 3 years ago

vchrombie commented 4 years ago

This PR contains the following enrichers

metrics discussion: metric labelled issues more information: https://github.com/vchrombie/gsoc/issues/6

micro-mordred configurations

project.json

{
  "potioncraft": {
    "gitqm": [
      "https://gitlab.com/Bitergia/devops/indexwarrior"
    ] ,
    "gitlabqm:issue": [
      "https://gitlab.com/Bitergia/devops%2Findexwarrior"
    ] ,
    "gitlabqm:merge": [
      "https://gitlab.com/Bitergia/devops%2Findexwarrior"
    ] ,
    "pipermailqm": [
      "https://lists.linuxfoundation.org/pipermail/grimoirelab-discussions/"
    ] ,
    "meetupqm": [
      "GitLab-Meetup-Group"
    ] ,
    "githubqm:issue": [
      "https://github.com/chaoss/grimoirelab"
    ] ,
    "githubqm:pull": [
      "https://github.com/chaoss/grimoirelab"
    ]
  }
}

setup.cfg

[gitqm]
raw_index = gitqm_raw
enriched_index = gitqm_enriched
category = commit

[gitlabqm:issue]
api-token = xxxx
raw_index = gitlabqm_issues_raw
enriched_index = gitlabqm_issues_enriched
sleep-for-rate = true
category = issue
no-archive = true

[gitlabqm:merge]
api-token = xxxx
raw_index = gitlabqm_mrs_raw
enriched_index = gitlabqm_mrs_enriched
sleep-for-rate = true
category = merge_request
no-archive = true

[pipermailqm]
raw_index = pipermailqm_raw
enriched_index = pipermailqm_enriched

[meetupqm]
raw_index = meetupqm_raw
enriched_index = meetupqm_enriched
api-token = xxxx
sleep-for-rate = true
sleep-time = "300"
no-archive = true

[githubqm:issue]
raw_index = githubqm_issues_raw
enriched_index = githubqm_issues_enriched
api-token = xxxx
sleep-for-rate = true
sleep-time = 300
category = issue
no-archive = true

[githubqm:pull]
raw_index = githubqm_prs_raw
enriched_index = githubqm_prs_enriched
api-token = xxxx
sleep-for-rate = true
sleep-time = 300
category = pull_request
no-archive = true
vchrombie commented 4 years ago

Hi everyone, I have updated the PR with the suggested changes and also the latest enricher work.

I will add the tests, schema and also format the project structure to similar of the SCMS structure tomorrow.

coveralls commented 4 years ago

Pull Request Test Coverage Report for Build 2375

Files with Coverage Reduction New Missed Lines %
/home/travis/build/chaoss/grimoirelab-elk/grimoire_elk/enriched/github.py 2 75.06%
/home/travis/build/chaoss/grimoirelab-elk/grimoire_elk/utils.py 45 66.42%
<!-- Total: 47 -->
Totals Coverage Status
Change from base Build 2373: -3.9%
Covered Lines: 8451
Relevant Lines: 10797
💛 - Coveralls
vchrombie commented 3 years ago

The work will be continued at glab-prosoul/grimoirelab-elk.