search

DefectDojo / django-DefectDojo

DevSecOps, ASPM, Vulnerability Management. All on one platform.
https://defectdojo.com
BSD 3-Clause "New" or "Revised" License
3.62k stars 1.52k forks source link

Unable to create EPIC - issue type is required #4316

Closed xanhacks closed 3 years ago

xanhacks commented 3 years ago

DD Version : 1.14.1 Installation : docker-compose

Hello,

Here is the error when I would like to create an epic via DD (my jira instance is already well configured and Enable engagement epic mapping is ON) :

uwsgi_1         | [pid: 23|app: 0|req: 20/33] 172.18.0.14 () {52 vars in 995 bytes} [Mon Apr 19 12:29:08 2021] GET /engagement/21/edit => generated 57216 bytes in 222 msecs (HTTP/1.1 200) 7 headers in 369 bytes (1 switches on core 1)
nginx_1         | 172.18.0.14 - - [19/Apr/2021:12:29:08 +0000] "GET /static/easymde/dist/easymde.min.css HTTP/1.1" 200 12394 "https://defectdojo-url/engagement/21/edit" "Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0" "-"
uwsgi_1         | [19/Apr/2021 12:29:13] DEBUG [dojo.jira_link.helper:161] not delegating to product PRO for Engagement: [SonarQube] project (Jan 01, 2021)
uwsgi_1         | [19/Apr/2021 12:29:13] DEBUG [dojo.jira_link.helper:1114] jform has changed: False
uwsgi_1         | [19/Apr/2021 12:29:13] DEBUG [dojo.jira_link.helper:1174] checking jira epic form for engagement: 21:Engagement: [SonarQube] project (Jan 01, 2021)
uwsgi_1         | [19/Apr/2021 12:29:13] DEBUG [dojo.jira_link.helper:158] delegating to product PRO for Engagement: [SonarQube] project (Jan 01, 2021)
uwsgi_1         | [19/Apr/2021 12:29:13] DEBUG [dojo.jira_link.helper:170] found jira_project 1: SAN(http://jira-url) for PRO
uwsgi_1         | [19/Apr/2021 12:29:13] DEBUG [dojo.jira_link.helper:1184] pushing engagement to JIRA
uwsgi_1         | [19/Apr/2021 12:29:13] DEBUG [dojo.decorators:134] model_or_id: Engagement: [SonarQube] project (Jan 01, 2021)
uwsgi_1         | [19/Apr/2021 12:29:13] DEBUG [dojo.decorators:61] converting model_or_id to id: Engagement: [SonarQube] project (Jan 01, 2021)
uwsgi_1         | [19/Apr/2021 12:29:13] DEBUG [dojo.jira_link.helper:1186] Push to JIRA for Epic queued succesfully
celeryworker_1  | [19/Apr/2021 12:29:13] INFO [celery.worker.strategy:157] Received task: dojo.jira_link.helper.add_epic[2271fbcd-de7d-4f17-8649-e056f8c48070]  
nginx_1         | 172.18.0.14 - - [19/Apr/2021:12:29:13 +0000] "POST /engagement/21/edit HTTP/1.1" 302 0 "https://defectdojo-url/engagement/21/edit" "Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0" "-"
uwsgi_1         | [pid: 23|app: 0|req: 21/34] 172.18.0.14 () {58 vars in 1181 bytes} [Mon Apr 19 12:29:13 2021] POST /engagement/21/edit => generated 0 bytes in 116 msecs (HTTP/1.1 302) 8 headers in 558 bytes (1 switches on core 0)
uwsgi_1         | [19/Apr/2021 12:29:13] DEBUG [dojo.jira_link.helper:158] delegating to product PRO for Engagement: [SonarQube] project (Jan 01, 2021)
uwsgi_1         | [19/Apr/2021 12:29:13] DEBUG [dojo.jira_link.helper:170] found jira_project 1: SAN(http://jira-url) for PRO
celeryworker_1  | [19/Apr/2021 12:29:13] ERROR [dojo.jira_link.helper:989] JiraError HTTP 400 url: http://jira-url/rest/api/2/issue
celeryworker_1  |   text: issue type is required
celeryworker_1  |   
celeryworker_1  |   response headers = {'Server': 'nginx/1.16.1', 'Date': 'Mon, 19 Apr 2021 12:23:18 GMT', 'Content-Type': 'application/json;charset=UTF-8', 'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', 'X-AREQUESTID': '863x1018505x1', 'X-ASESSIONID': '1ut55xb', 'X-XSS-Protection': '1; mode=block', 'X-Content-Type-Options': 'nosniff', 'X-Frame-Options': 'SAMEORIGIN', 'Content-Security-Policy': "frame-ancestors 'self'", 'X-ASEN': 'SEN-18287910', 'X-Seraph-LoginReason': 'OK', 'X-AUSERNAME': 'jira-username', 'Cache-Control': 'no-cache, no-store, no-transform', 'Content-Encoding': 'gzip', 'Vary': 'User-Agent'}
celeryworker_1  |   response text = {"errorMessages":[],"errors":{"issuetype":"issue type is required"}}
celeryworker_1  | Traceback (most recent call last):
celeryworker_1  |   File "/app/dojo/jira_link/helper.py", line 975, in add_epic
celeryworker_1  |     new_issue = jira.create_issue(fields=issue_dict)
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/jira/client.py", line 1107, in create_issue
celeryworker_1  |     r = self._session.post(url, data=json.dumps(data))
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/jira/resilientsession.py", line 154, in post
celeryworker_1  |     return self.__verb('POST', url, **kwargs)
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/jira/resilientsession.py", line 147, in __verb
celeryworker_1  |     raise_on_error(response, verb=verb, **kwargs)
celeryworker_1  |   File "/usr/local/lib/python3.6/site-packages/jira/resilientsession.py", line 57, in raise_on_error
celeryworker_1  |     r.status_code, error, r.url, request=request, response=r, **kwargs)
celeryworker_1  | jira.exceptions.JIRAError: JiraError HTTP 400 url: http://jira-url/rest/api/2/issue
celeryworker_1  |   text: issue type is required
celeryworker_1  |   
celeryworker_1  |   response headers = {'Server': 'nginx/1.16.1', 'Date': 'Mon, 19 Apr 2021 12:23:18 GMT', 'Content-Type': 'application/json;charset=UTF-8', 'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', 'X-AREQUESTID': '863x1018505x1', 'X-ASESSIONID': '1ut55xb', 'X-XSS-Protection': '1; mode=block', 'X-Content-Type-Options': 'nosniff', 'X-Frame-Options': 'SAMEORIGIN', 'Content-Security-Policy': "frame-ancestors 'self'", 'X-ASEN': 'SEN-18287910', 'X-Seraph-LoginReason': 'OK', 'X-AUSERNAME': 'jira-username', 'Cache-Control': 'no-cache, no-store, no-transform', 'Content-Encoding': 'gzip', 'Vary': 'User-Agent'}
celeryworker_1  |   response text = {"errorMessages":[],"errors":{"issuetype":"issue type is required"}}
celeryworker_1  | [19/Apr/2021 12:29:13] INFO [celery.app.trace:125] Task dojo.jira_link.helper.add_epic[2271fbcd-de7d-4f17-8649-e056f8c48070] succeeded in 0.26718668546527624s: False
nginx_1         | 172.18.0.14 - - [19/Apr/2021:12:29:13 +0000] "GET /alerts/count HTTP/1.1" 200 12 "https://defectdojo-url/alerts" "Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0" "-"
uwsgi_1         | [pid: 23|app: 0|req: 23/35] 172.18.0.14 () {52 vars in 1206 bytes} [Mon Apr 19 12:29:13 2021] GET /alerts/count => generated 12 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 180 bytes (1 switches on core 0)
uwsgi_1         | [19/Apr/2021 12:29:14] DEBUG [dojo.jira_link.helper:219] getting jira project url
uwsgi_1         | [19/Apr/2021 12:29:14] DEBUG [dojo.jira_link.helper:158] delegating to product PRO for Engagement: [SonarQube] project (Jan 01, 2021)
uwsgi_1         | [19/Apr/2021 12:29:14] DEBUG [dojo.jira_link.helper:170] found jira_project 1: SAN(http://jira-url) for PRO
uwsgi_1         | [19/Apr/2021 12:29:14] DEBUG [dojo.jira_link.helper:226] getting jira project url2
uwsgi_1         | [19/Apr/2021 12:29:14] DEBUG [dojo.jira_link.helper:158] delegating to product PRO for Engagement: [SonarQube] project (Jan 01, 2021)
uwsgi_1         | [19/Apr/2021 12:29:14] DEBUG [dojo.jira_link.helper:170] found jira_project 1: SAN(http://jira-url) for PRO
uwsgi_1         | [19/Apr/2021 12:29:14] DEBUG [dojo.jira_link.helper:183] found jira_instance Jira SAN | http://jira-url | jira-username for Engagement: [SonarQube] project (Jan 01, 2021)

In my Jira configuration, the issue type for an epic is 'EPIC' so I edit the issue_dict and rebuilt the uwsgi docker image, but I still have the same error. (If the issuetype field could go to the settings.dist.py in the future, it will be great)

dojo/jira_link/helper.py

        issue_dict = {
            'project': {
                'key': jira_project.project_key
            },
            'summary': engagement.name,
            'description': engagement.name,
            'issuetype': {
                'name': 'EPIC'
            },
            get_epic_name_field_name(jira_instance): engagement.name,
        }

If I test locally this script, it works fine :

from jira import JIRA

BASE_URL = "http://jira-url" 
jira = JIRA(BASE_URL, auth=('jira-username', 'jira-password'))

issue_dict = {
    'project': {
        'key': 'SAN'
    },
    'summary': 'Toto scan',
    'description': 'Test',
    'issuetype': {
        'name': 'EPIC'
    },
    "customfield_10102": "Toto scan"
}

new_issue = jira.create_issue(fields=issue_dict)
print(new_issue)

Regards

valentijnscholten commented 3 years ago

Can you provide the full logs? It should contain a log with the actual data being sent to JIRA:

image

JIRA is very strange in its errors, so your error might mean something else like an invalid project key or using an account with no permissions.

xanhacks commented 3 years ago

Unfortunately, I don't have any more logs. I also find it strange that this debug line is not displayed.

I really don't see what the problem is with Jira, because I can push findings to Jira but I can't create an Epic.

valentijnscholten commented 3 years ago

Looks like the celery work is not on DEBUG log level.

xanhacks commented 3 years ago 
  uwsgi:
    image: custom/defectdojo-uwsgi:1.14.1
    depends_on:
      - mysql
    entrypoint: ['/wait-for-it.sh', 'mysql:3306', '-t', '30', '--', '/entrypoint-uwsgi.sh']
    environment:
      DD_DEBUG: 'True'
      DD_LOG_LEVEL: 'DEBUG'
      ...
      DD_CELERY_LOG_LEVEL: 'DEBUG'

There is the env variable for uwsgi.

valentijnscholten commented 3 years ago

you need to set it for all containers

xanhacks commented 3 years ago

I set DD_CELERY_LOG_LEVEL: 'DEBUG' in every dockers in the docker-compose file but I still don't have the logger.debug('add_epic ...') line.

valentijnscholten commented 3 years ago

You need to set DD_DEBUG and DD_LOG_LEVEL

xanhacks commented 3 years ago 
celeryworker_1  | [21/Apr/2021 07:47:11] DEBUG [dojo.jira_link.helper:974] add_epic:
{'project': {'key': 'SAN'}, 'summary': '[SonarQube] repo', 'description': '[SonarQube] repo', 'issuetype': {'name': 'Epic'}, 'customfield_10102': '[SonarQube] repo'}

Thanks, the line below appears ! The error it's because {'name': 'Epic'} should be {'name': 'EPIC'} but I already change it in the code of the uwsgi docker.

$ docker exec -it 37dac944f028 bash
defectdojo@37dac944f028:/app$ cat dojo/jira_link/helper.py | grep "def add_epic(" -A 20
def add_epic(engagement):
    logger.debug('trying to create a new jira EPIC for %d:%s', engagement.id, engagement.name)

    if not is_jira_configured_and_enabled(engagement):
        return False

    logger.debug('config found')

    jira_project = get_jira_project(engagement)
    jira_instance = get_jira_instance(engagement)
    if jira_project.enable_engagement_epic_mapping:
        issue_dict = {
            'project': {
                'key': jira_project.project_key
            },
            'summary': engagement.name,
            'description': engagement.name,
            'issuetype': {
                'name': 'EPIC'
            },
            get_epic_name_field_name(jira_instance): engagement.name,

Is there any cache or something ? I already purge all the volumes and down/up docker-compose.

valentijnscholten commented 3 years ago

You need to rebuild all containers after making code changes. This code is executed in the celery-worker container.

xanhacks commented 3 years ago

Now it works ! Very big thanks !