Closed ChrisMcKee closed 1 year ago
Hi @ChrisMcKee. It looks like SQ responded "correctly" (there was no HTTP-4xx or HTTP-5xx) but it didn't reply with the expected JSON. Are you able to test your deployment with this change: https://github.com/DefectDojo/django-DefectDojo/pull/7394? It should be more talkative regarding real SQ responses.
If you will confirm, it is working for you, I will offer it as the solution.
@kiblik Great; I've just built the Dockerfile.django file in your fork to test. I'll update once it's pushed and I've swapped the image out
Not sure that your rebuild of the image from my fork was successful. I see in log number of line from the old version, not from my fix (372 vs. 374). Can you check it, please?
Yep sorry; I didnt clock github cli flipping back to master 🤦 ... Rebuilding 🏗️
UI
Test request was successful (there was no HTTP-4xx or HTTP-5xx) but response doesn't contain expected JSON response. SonarQube responded with HTTP-200 (). This is full response: <!DOCTYPE html> <html lang="en"> <head> <meta http-equiv="content-type" content="text/html; charset=UTF-8" charset="UTF-8" /> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <link rel="apple-touch-icon" href="/apple-touch-icon.png"> <link rel="apple-touch-icon" sizes="57x57" href="/apple-touch-icon-57x57.png"> <link rel="apple-touch-icon" sizes="60x60" href="/apple-touch-icon-60x60.png"> <link rel="apple-touch-icon" sizes="72x72" href="/apple-touch-icon-72x72.png"> <link rel="apple-touch-icon" sizes="76x76" href="/apple-touch-icon-76x76.png"> <link rel="apple-touch-icon" sizes="114x114" href="/apple-touch-icon-114x114.png"> <link rel="apple-touch-icon" sizes="120x120" href="/apple-touch-icon-120x120.png"> <link rel="apple-touch-icon" sizes="144x144" href="/apple-touch-icon-144x144.png"> <link rel="apple-touch-icon" sizes="152x152" href="/apple-touch-icon-152x152.png"> <link rel="apple-touch-icon" sizes="180x180" href="/apple-touch-icon-180x180.png"> <link rel="icon" type="image/x-icon" href="/favicon.ico"> <meta name="application-name" content="SonarQube" /> <meta name="msapplication-TileColor" content="#FFFFFF" /> <meta name="msapplication-TileImage" content="/mstile-512x512.png" /> <title>SonarQube</title> <link rel="stylesheet" href="/js/out726653E4.css" /> </head> <body> <div id="content"> <div class="global-loading"> <i class="spinner global-loading-spinner"></i> <span aria-live="polite" class="global-loading-text">Loading...</span> </div> </div> <script> window.baseUrl = ''; window.serverStatus = 'UP'; window.instance = 'SonarQube'; window.official = true; </script> <script type="module" src="/js/outDUFI7UAQ.js"></script> </body> </html>
Log
[10/Jan/2023 15:43:34] ERROR [dojo.tool_config.views:39]
Test request was successful (there was no HTTP-4xx or HTTP-5xx) but response doesn't contain expected JSON response.
SonarQube responded with HTTP-200 ().
This is full response:
<!DOCTYPE html>
<html lang="en">
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8" charset="UTF-8" />
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<link rel="apple-touch-icon" href="/apple-touch-icon.png">
<link rel="apple-touch-icon" sizes="57x57" href="/apple-touch-icon-57x57.png">
<link rel="apple-touch-icon" sizes="60x60" href="/apple-touch-icon-60x60.png">
<link rel="apple-touch-icon" sizes="72x72" href="/apple-touch-icon-72x72.png">
<link rel="apple-touch-icon" sizes="76x76" href="/apple-touch-icon-76x76.png">
<link rel="apple-touch-icon" sizes="114x114" href="/apple-touch-icon-114x114.png">
<link rel="apple-touch-icon" sizes="120x120" href="/apple-touch-icon-120x120.png">
<link rel="apple-touch-icon" sizes="144x144" href="/apple-touch-icon-144x144.png">
<link rel="apple-touch-icon" sizes="152x152" href="/apple-touch-icon-152x152.png">
<link rel="apple-touch-icon" sizes="180x180" href="/apple-touch-icon-180x180.png">
<link rel="icon" type="image/x-icon" href="/favicon.ico">
<meta name="application-name" content="SonarQube" />
<meta name="msapplication-TileColor" content="#FFFFFF" />
<meta name="msapplication-TileImage" content="/mstile-512x512.png" />
<title>SonarQube</title>
<link rel="stylesheet" href="/js/out726653E4.css" />
</head>
<body>
<div id="content">
<div class="global-loading">
<i class="spinner global-loading-spinner"></i>
<span aria-live="polite" class="global-loading-text">Loading...</span>
</div>
</div>
<script>
window.baseUrl = '';
window.serverStatus = 'UP';
window.instance = 'SonarQube';
window.official = true;
</script>
<script type="module" src="/js/outDUFI7UAQ.js"></script>
</body>
</html>
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/requests/models.py", line 971, in json
return complexjson.loads(self.text, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 2 column 1 (char 1)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/dojo/tools/sonarqube_api/api_client.py", line 374, in test_connection
num_projects = response.json()['paging']['total']
^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/requests/models.py", line 975, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 2 column 1 (char 1)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/dojo/tool_config/views.py", line 27, in new_tool_config
result = api.test_connection()
^^^^^^^^^^^^^^^^^^^^^
File "/app/dojo/tools/sonarqube_api/api_client.py", line 376, in test_connection
raise Exception(f"""
Exception:
Test request was successful (there was no HTTP-4xx or HTTP-5xx) but response doesn't contain expected JSON response.
SonarQube responded with HTTP-200 ().
This is full response:
<!DOCTYPE html>
<html lang="en">
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8" charset="UTF-8" />
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<link rel="apple-touch-icon" href="/apple-touch-icon.png">
<link rel="apple-touch-icon" sizes="57x57" href="/apple-touch-icon-57x57.png">
<link rel="apple-touch-icon" sizes="60x60" href="/apple-touch-icon-60x60.png">
<link rel="apple-touch-icon" sizes="72x72" href="/apple-touch-icon-72x72.png">
<link rel="apple-touch-icon" sizes="76x76" href="/apple-touch-icon-76x76.png">
<link rel="apple-touch-icon" sizes="114x114" href="/apple-touch-icon-114x114.png">
<link rel="apple-touch-icon" sizes="120x120" href="/apple-touch-icon-120x120.png">
<link rel="apple-touch-icon" sizes="144x144" href="/apple-touch-icon-144x144.png">
<link rel="apple-touch-icon" sizes="152x152" href="/apple-touch-icon-152x152.png">
<link rel="apple-touch-icon" sizes="180x180" href="/apple-touch-icon-180x180.png">
<link rel="icon" type="image/x-icon" href="/favicon.ico">
<meta name="application-name" content="SonarQube" />
<meta name="msapplication-TileColor" content="#FFFFFF" />
<meta name="msapplication-TileImage" content="/mstile-512x512.png" />
<title>SonarQube</title>
<link rel="stylesheet" href="/js/out726653E4.css" />
</head>
<body>
<div id="content">
<div class="global-loading">
<i class="spinner global-loading-spinner"></i>
<span aria-live="polite" class="global-loading-text">Loading...</span>
</div>
</div>
<script>
window.baseUrl = '';
window.serverStatus = 'UP';
window.instance = 'SonarQube';
window.official = true;
</script>
<script type="module" src="/js/outDUFI7UAQ.js"></script>
</body>
</html>
The URL is just set to the face url of sonarqube on our cluster; it needed to be
https://sq.mydomain.com/api
saved fine with that set.
I will add it as a possible reason of failure in to the message.
Ta; I was flipping between this, dependency-track, kubeclarity, dependency-check, burp-enterprise. The docs do show it with the /api but as the path isn't configurable in sonarqube it seems redundant to require it to be specified. Obviously more of a faff to change now though as its been out a fair while.
I extended the error message and change the status of PR to "ready for review". Let's wait for the approval and merge.
UI
Test request was successful (there was no HTTP-4xx or HTTP-5xx) but response doesn't contain expected JSON response. SonarQube responded with HTTP-200 (). This is full response: <!DOCTYPE html> <html lang="en"> <head> <meta http-equiv="content-type" content="text/html; charset=UTF-8" charset="UTF-8" /> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <link rel="apple-touch-icon" href="/apple-touch-icon.png"> <link rel="apple-touch-icon" sizes="57x57" href="/apple-touch-icon-57x57.png"> <link rel="apple-touch-icon" sizes="60x60" href="/apple-touch-icon-60x60.png"> <link rel="apple-touch-icon" sizes="72x72" href="/apple-touch-icon-72x72.png"> <link rel="apple-touch-icon" sizes="76x76" href="/apple-touch-icon-76x76.png"> <link rel="apple-touch-icon" sizes="114x114" href="/apple-touch-icon-114x114.png"> <link rel="apple-touch-icon" sizes="120x120" href="/apple-touch-icon-120x120.png"> <link rel="apple-touch-icon" sizes="144x144" href="/apple-touch-icon-144x144.png"> <link rel="apple-touch-icon" sizes="152x152" href="/apple-touch-icon-152x152.png"> <link rel="apple-touch-icon" sizes="180x180" href="/apple-touch-icon-180x180.png"> <link rel="icon" type="image/x-icon" href="/favicon.ico"> <meta name="application-name" content="SonarQube" /> <meta name="msapplication-TileColor" content="#FFFFFF" /> <meta name="msapplication-TileImage" content="/mstile-512x512.png" /> <title>SonarQube</title> <link rel="stylesheet" href="/js/out726653E4.css" /> </head> <body> <div id="content"> <div class="global-loading"> <i class="spinner global-loading-spinner"></i> <span aria-live="polite" class="global-loading-text">Loading...</span> </div> </div> <script> window.baseUrl = ''; window.serverStatus = 'UP'; window.instance = 'SonarQube'; window.official = true; </script> <script type="module" src="/js/outDUFI7UAQ.js"></script> </body> </html>
Log
[10/Jan/2023 15:43:34] ERROR [dojo.tool_config.views:39] Test request was successful (there was no HTTP-4xx or HTTP-5xx) but response doesn't contain expected JSON response. SonarQube responded with HTTP-200 (). This is full response: <!DOCTYPE html> <html lang="en"> <head> <meta http-equiv="content-type" content="text/html; charset=UTF-8" charset="UTF-8" /> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <link rel="apple-touch-icon" href="/apple-touch-icon.png"> <link rel="apple-touch-icon" sizes="57x57" href="/apple-touch-icon-57x57.png"> <link rel="apple-touch-icon" sizes="60x60" href="/apple-touch-icon-60x60.png"> <link rel="apple-touch-icon" sizes="72x72" href="/apple-touch-icon-72x72.png"> <link rel="apple-touch-icon" sizes="76x76" href="/apple-touch-icon-76x76.png"> <link rel="apple-touch-icon" sizes="114x114" href="/apple-touch-icon-114x114.png"> <link rel="apple-touch-icon" sizes="120x120" href="/apple-touch-icon-120x120.png"> <link rel="apple-touch-icon" sizes="144x144" href="/apple-touch-icon-144x144.png"> <link rel="apple-touch-icon" sizes="152x152" href="/apple-touch-icon-152x152.png"> <link rel="apple-touch-icon" sizes="180x180" href="/apple-touch-icon-180x180.png"> <link rel="icon" type="image/x-icon" href="/favicon.ico"> <meta name="application-name" content="SonarQube" /> <meta name="msapplication-TileColor" content="#FFFFFF" /> <meta name="msapplication-TileImage" content="/mstile-512x512.png" /> <title>SonarQube</title> <link rel="stylesheet" href="/js/out726653E4.css" /> </head> <body> <div id="content"> <div class="global-loading"> <i class="spinner global-loading-spinner"></i> <span aria-live="polite" class="global-loading-text">Loading...</span> </div> </div> <script> window.baseUrl = ''; window.serverStatus = 'UP'; window.instance = 'SonarQube'; window.official = true; </script> <script type="module" src="/js/outDUFI7UAQ.js"></script> </body> </html> Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/requests/models.py", line 971, in json return complexjson.loads(self.text, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/json/__init__.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 2 column 1 (char 1) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/app/dojo/tools/sonarqube_api/api_client.py", line 374, in test_connection num_projects = response.json()['paging']['total'] ^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/requests/models.py", line 975, in json raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) requests.exceptions.JSONDecodeError: Expecting value: line 2 column 1 (char 1) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/app/dojo/tool_config/views.py", line 27, in new_tool_config result = api.test_connection() ^^^^^^^^^^^^^^^^^^^^^ File "/app/dojo/tools/sonarqube_api/api_client.py", line 376, in test_connection raise Exception(f""" Exception: Test request was successful (there was no HTTP-4xx or HTTP-5xx) but response doesn't contain expected JSON response. SonarQube responded with HTTP-200 (). This is full response: <!DOCTYPE html> <html lang="en"> <head> <meta http-equiv="content-type" content="text/html; charset=UTF-8" charset="UTF-8" /> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <link rel="apple-touch-icon" href="/apple-touch-icon.png"> <link rel="apple-touch-icon" sizes="57x57" href="/apple-touch-icon-57x57.png"> <link rel="apple-touch-icon" sizes="60x60" href="/apple-touch-icon-60x60.png"> <link rel="apple-touch-icon" sizes="72x72" href="/apple-touch-icon-72x72.png"> <link rel="apple-touch-icon" sizes="76x76" href="/apple-touch-icon-76x76.png"> <link rel="apple-touch-icon" sizes="114x114" href="/apple-touch-icon-114x114.png"> <link rel="apple-touch-icon" sizes="120x120" href="/apple-touch-icon-120x120.png"> <link rel="apple-touch-icon" sizes="144x144" href="/apple-touch-icon-144x144.png"> <link rel="apple-touch-icon" sizes="152x152" href="/apple-touch-icon-152x152.png"> <link rel="apple-touch-icon" sizes="180x180" href="/apple-touch-icon-180x180.png"> <link rel="icon" type="image/x-icon" href="/favicon.ico"> <meta name="application-name" content="SonarQube" /> <meta name="msapplication-TileColor" content="#FFFFFF" /> <meta name="msapplication-TileImage" content="/mstile-512x512.png" /> <title>SonarQube</title> <link rel="stylesheet" href="/js/out726653E4.css" /> </head> <body> <div id="content"> <div class="global-loading"> <i class="spinner global-loading-spinner"></i> <span aria-live="polite" class="global-loading-text">Loading...</span> </div> </div> <script> window.baseUrl = ''; window.serverStatus = 'UP'; window.instance = 'SonarQube'; window.official = true; </script> <script type="module" src="/js/outDUFI7UAQ.js"></script> </body> </html>
The URL is just set to the face url of sonarqube on our cluster; it needed to be
https://sq.mydomain.com/api
saved fine with that set.
can i know how you fixed this? @ChrisMcKee
I just added the /api to my sonar address.
Unable to find the project https://uat-sonarqube.com/dashboard?id=sts due to 404 - {"errors":[{"msg":"Component key \u0027https://uat-sonarqube.com/dashboard?id\u003dsts\u0027 not found"}]}
while adding Sonarqube API Scan Configuration, I'm getting this error can u help me to fix this. @ChrisMcKee
Key is only sts
, not the whole url.
Unable to find the project https://uat-sonarqube.com/dashboard?id=sts due to 404 - {"errors":[{"msg":"Component key \u0027https://uat-sonarqube.com/dashboard?id\u003dsts\u0027 not found"}]}
while adding Sonarqube API Scan Configuration, I'm getting this error can u help me to fix this. @ChrisMcKee
but still it shows this error Unable to find the project sts due to 403 - {"errors":[{"msg":"Insufficient privileges"}]}
@ChrisMcKee
but still it shows this error Unable to find the project sts due to 403 - {"errors":[{"msg":"Insufficient privileges"}]}
@ChrisMcKee
@kiblik kindly review this and give me an update.
but still it shows this error Unable to find the project sts due to 403 - {"errors":[{"msg":"Insufficient privileges"}]} @ChrisMcKee
@kiblik kindly review this and give me an update.
double-check that sts
is really the correct key (maybe you cut some characters) and also if used API key has permission to access this project in SQ.
but still it shows this error Unable to find the project sts due to 403 - {"errors":[{"msg":"Insufficient privileges"}]}
@ChrisMcKee
I've crossed this part, But still after that i couldn't find any report to be imported from sonarqube.
@kiblik kindly check this and update me that how to get report from sonarqube.
but still it shows this error Unable to find the project sts due to 403 - {"errors":[{"msg":"Insufficient privileges"}]} @ChrisMcKee
I've crossed this part, But still after that i couldn't find any report to be imported from sonarqube.
@kiblik kindly check this and update me that how to get report from sonarqube.
If you are still receiving Unable to find the project sts due to 403 - {"errors":[{"msg":"Insufficient privileges"}]}
you can not expect any new data in DD. This error says "I'm not able to fetch data".
Try to generate a new API token or check some possible permission issues in SQ. Messages Unable to find the project sts due to 403 - {"errors":[{"msg":"Insufficient privileges"}]}
is coming from SQ and it is just displayed in DD.
@kiblik noo that's not the issue now for me, project sts is successfully added, but after adding still the report is not generated in defectdojo. can u provide me the entire steps for integrating sonarqube in defectdojo.
I done all these steps successfully but still couldn't fetch any SQ report here.
I done all these steps successfully but still couldn't fetch any SQ report here.
Can you check the logs?
In this part ....
After this is done, you can import the findings on the Product page through Findings -> Import Scan Results. Select “Edgescan Scan” as the Scan type, the API scan configuration from the last step, and click Import.
How should i apply credential?
while importing i used to get this error
An exception error occurred during the report import:403 Client Error: Forbidden for url: https://sonarqube/api/v1/vulnerabilities/export.json?c%5Basset_id_in%5D=cms&c%5Bstatus%5D=open
@kiblik
Sorry, there was a typo in the documentation. I already prepared a fix: https://github.com/DefectDojo/django-DefectDojo/pull/8389/files
Please set Scan type
as SonarQube API Import
and API Scan Configuration
to your credentials (which you created).
Bug description Saving sonarqube tool config returns
Expecting value: line 2 column 1 (char 1)
and fails to save.Steps to reproduce
Django log
Expected behaviour Sonarqube login happens and the service stores the change; or if the issue is during communication with sonarqube the response is logged/appropriate error returned.
Deployment method (select with an
X
)Environment information