search

integrated-application-development / delphilint

Delphi IDE package providing on-the-fly code analysis and linting, powered by SonarDelphi
GNU Lesser General Public License v3.0
79 stars 9 forks source link

Unable to use our SonarQube install to analyse files #53

Open NeeeeB opened 2 months ago

NeeeeB commented 2 months ago

Prerequisites

Delphi IDE version

Delphi 12 Version 29.0.51961.7529

DelphiLint version

1.1.0

SonarDelphi version

1.1.0

SonarQube version

Version 9.9.3 (build 79811)

Issue description

java version "21.0.2" 2024-01-16 LTS Windows 11 22H2 Build 22621.3737

Tried with all 3 User Token available (Project, Global and Personal), always the same error as shown in the screenshot. image

logs as requested : delphilint-server-2024-06-26-5.log

Ini file can not be uploaded in Github so here is what it looks like (minus the token)

[Resources] ServerJarOverride= JavaExeOverride=C:\Program Files\Java\jdk-21\bin\java.exe [Debug] ShowConsole=0 ExternalServer=0 [Client] AutoShowToolWindow=1 SaveBeforeAnalysis=1 [SonarHost] Tokens_Size=1 Tokens_0=MD.desktop_client@https://sonar.bow.local/sonar=sqp_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX [Server] SonarDelphiVersionOverride= JvmOptions=-server -Djava.net.useSystemProxies=true [Standalone] UseDefaultRules=1 DisabledRules_Size=1 DisabledRules_0=

Steps to reproduce

Just run the analysis on a file.

Minimal Delphi code exhibiting the issue

No response

fourls commented 2 months ago

Hi @NeeeeB, thanks for raising this issue!

I think it's very likely that this is an incompatibility with the SonarQube 9.9 API - I've found that the API can be very inconsistent even between minor versions.

I'll investigate and get back to you.

denieu commented 2 months ago

When I was trying to configure it in my IDE, after correcting the problem that I reported in issue #51, I had the same problem, after about an hour doing several tests I was successful.

It seems to have worked for me when I removed the Project Key from the plugin global configuration.

I was using SonarQube 10.5.1 and Java 21.0.2.

NeeeeB commented 2 months ago

Oh, I might try this. What kind of user token did you use ?

Le mer. 26 juin 2024, 21:35, Daniel Wojcickoski @.***> a écrit :

When I was trying to configure it in my IDE, after correcting the problem that I reported in issue #51 https://github.com/integrated-application-development/delphilint/issues/51, I had the same problem, after about an hour doing several tests I was successful.

It seems to have worked for me when I removed the Project Key from the plugin global configuration.

I was using SonarQube 10.5.1 and Java 21.0.2.

— Reply to this email directly, view it on GitHub https://github.com/integrated-application-development/delphilint/issues/53#issuecomment-2192488757, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADABJMT5VD66DJRXD7E4CXTZJMJYRAVCNFSM6AAAAABJ52G52CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJSGQ4DQNZVG4 . You are receiving this because you were mentioned.Message ID: @.*** .com>

NeeeeB commented 2 months ago

@fourls well @denieu 's suggestion did the trick. As soon as I removed the project key of our SonarQube instance in DelphiLint configuration, it started to work. Now I do have another problem, but I'll ask in the Discord group because it is more a "use" problem than a bug.

fourls commented 2 months ago

As soon as I removed the project key of our SonarQube instance in DelphiLint configuration, it started to work.

To clarify, @NeeeeB and @denieu, you removed it from the Connected Mode authorization settings, like this?

image

That would make the authorization token not apply at all - all these settings do is indicate to DelphiLint that it should include the "Authorization Token" when making requests for the project "Project Key" to "Server URL". The only way this setup would work is if "Force user authentication" is disabled on the SonarQube instance - if this is the case for you both then this will work fine.

The problem still stands that the authorization token doesn't seem to be working. I have been using SonarQube 10.5.1 + DelphiLint with no issues for a couple of months now, so that lessens my theory about the SonarQube version.

I will still test SQ 9.9.1 tomorrow and see if I can reproduce.

NeeeeB commented 2 months ago

That's exactly what I did. Well, about the "Force user authentication" I'll have to check (can not for the moment), but given that I need to authenticate to access the SonarQube instance, I would believe it is enabled...

I'll try to update SonarQube and SonarDelphi today, because I see that everything is outdated on our side...(we are in the process of adding all of this code analyzing to our project so it is just "tests" for the moment.)

I need to find documentation about making custom rules for sonarDelphi too, because our coding convention is somewhat special.

fourls commented 2 months ago

Interesting - thanks for all the details, tomorrow I'll try a few things and see what I get.

For getting started with SonarDelphi and custom rules, I'd recommend reading SonarDelphi's manual, particularly the section on custom rules. This includes:

denieu commented 2 months ago

What kind of user token did you use?

I used a "User Token" (Personal), but it was trial and error, I didn't find anywhere indicating which token to use in the documentation. image

To clarify, @NeeeeB and @denieu, you removed it from the Connected Mode authorization settings, like this?

Exactly like that @fourls. Due to my organization's policies "Force User Authentication" needs to be enabled, in my opinion it is a bit counterintuitive to try to analyze the project and fail because of this, since I was analyzing the project with the same Project Key setted in Connected Mode authorization settings.

NeeeeB commented 2 months ago

@denieu same policy here, "Force user anthentication" is enabled too (@fourls I've checked). I also used a Personal User Token, and like you it was trial and error. It works now btw, can not explain why since @fourls says it should not ahah.

fourls commented 2 months ago

OK, so I think there are two separate problems here:

1: SQ 9.9 incompatibility

I've done some testing and I've found that SQ 9.9 uses a different authentication scheme to later versions, making DelphiLint incompatible. I've just merged #55 to fix this.

@NeeeeB, this could be your original problem from a few months ago.

2: ???

Unfortunately I'm not able to reproduce the behaviour you describe where the token doesn't work if the project key is set, but does work if it's replaced with an empty string.

I'm wondering if this is a locale-specific bug. If so, the possible failure points - which I rewrote fairly recently in #37 - are generally pretty well covered by unit tests.

@denieu and @NeeeeB, could you run the DelphiLint client tests and let me know if any of them fail? You can run the tests by

  1. Cloning + installing build prerequisites
  2. Opening client/test/DelphiLintClientTest280.dproj (or 290 if Delphi 12) in the Delphi IDE
  3. Compiling and running
  4. Clicking "Run" in the top left corner of the test explorer
NeeeeB commented 2 months ago

I'll try to run the tests this morning.

EdIt : @fourls

image

That file is missing in the source.

fourls commented 2 months ago

@NeeeeB, this looks like the JS that's compiled in the prebuild script didn't succeed. Is there any output from the prebuild script?

NeeeeB commented 2 months ago

Ah sorry, missed a step in the prerequisites about running the script. It failed BTW.

image

@fourls

fourls commented 1 month ago

Hi @NeeeeB, this looks like a bug with the Java formatter we use - according to https://github.com/diffplug/spotless/issues/1791 this has been fixed in a recent version. I'll update that dependency ASAP.

Can you run the client tests? I think it's most likely that the problem is in the client.

You can run the tests by

  1. Cloning + installing build prerequisites
  2. Opening client/test/DelphiLintClientTest280.dproj (or 290 if Delphi 12) in the Delphi IDE
  3. Compiling and running
  4. Clicking "Run" in the top left corner of the test explorer
NeeeeB commented 1 month ago

@fourls Looks like there is still a problem even with updated DelphiLint sources.

image

Sorry, can't help with this myself because I'm definitely not used with java and co...

fourls commented 1 month ago

Hi @NeeeeB, good catch! I'll fix that up tomorrow.

Regardless, you don't need to run any Java stuff or scripts to run the client tests, which are in the Delphi project at client/test/DelphiLintClientTest280.dproj (or 290 if Delphi 12). Could you please compile that project using the Delphi IDE, then run the generated executable?

NeeeeB commented 1 month ago

Hi @NeeeeB, good catch! I'll fix that up tomorrow.

Regardless, you don't need to run any Java stuff or scripts to run the client tests, which are in the Delphi project at client/test/DelphiLintClientTest280.dproj (or 290 if Delphi 12). Could you please compile that project using the Delphi IDE, then run the generated executable?

Oh damn, didn't check if the prebuild script had compiled the missing .js from my previous attempt. I thought it would not if it failed. But it is here now, so I'll try and run the tests ASAP.

NeeeeB commented 1 month ago

@fourls image

Well, that will be a stopper for me. DUnitX is missing, and we do not use it here, so I won't install it. I'm not working on a hobby project here and thus can not modify my Delphi installation like I want... Sorry.

PS: except if you have any way to include the missing units in your source.

fourls commented 1 month ago

Hi @NeeeeB, OK. I've built the tests myself:

DelphiLintClientTest290.zip

Could you run and see if you encounter any problems?

NeeeeB commented 1 month ago

@fourls image

Many thanks for your time and your patience BTW.

fourls commented 1 month ago

Damn, seems like my theory was wrong... thank you for running the tests. I still can't reproduce. Perhaps it's Windows 11? I'm trying to reproduce on Windows 10. Seems unlikely, though.

I'll do some more investigation and get back to you.

Many thanks for your time and your patience BTW.

No worries! Thanks for raising and helping out to debug this issue.

denieu commented 1 month ago

That would make the authorization token not apply at all - all these settings do is indicate to DelphiLint that it should include the "Authorization Token" when making requests for the project "Project Key" to "Server URL". The only way this setup would work is if "Force user authentication" is disabled on the SonarQube instance - if this is the case for you both then this will work fine.

I ran some new tests, it really seems to me that it was a configuration problem and not actually a bug in the plugin.

I only managed to get the same error by configuring the Project Key in the connected mode settings and leaving the Project Key blank in the project settings.

I believe the confusion occurs for two main reasons:

image

Considering I'm using a user token and only have one token for the SonarQube instance, I'll keep an empty project key everywhere, this will work fine for me.

I don't know if it's possible, but I believe it would be great to have some "automatic" configuration mode similar to what is done in SonarLint connected mode configuration in VSCode. I believe that, in addition, improving the configuration documentation would be very useful.

Below are some screenshots of how SonarLint connected mode configuration works:

image

image

fourls commented 1 month ago

Hi @denieu, thanks for doing this extra testing! This is very helpful.

I only managed to get the same error by configuring the Project Key in the connected mode settings and leaving the Project Key blank in the project settings.

@NeeeeB, do you have the same configuration? Could this be the problem?

  • I didn't find any section in the plugin configuration documentation explaining which type of token should be used

This is a good point, we don't have any indication about the token to use. This is because that any of the tokens should work, but SonarQube requires user-level authorization for the endpoint to retrieve security hotspot information (the API documentation lies about this, which makes me think it's a bug).

  • By selecting "Read sonar-project.properties if present" I expected the Project Key in the Project Options to be automatically populated and I wouldn't need to worry about it.

Hmm, to be honest this has never occurred to me. They're totally different concerns internally - the project key is used to communicate with SonarQube, while the sonar-project.properties is processed by SonarDelphi. It's a good idea - I've raised #58 to look at reading the sonar-project.properties if no project key is supplied.

I don't know if it's possible, but I believe it would be great to have some "automatic" configuration mode similar to what is done in SonarLint connected mode configuration in VSCode.

The most we can do is a button that opens the URL to create a token - other than that, we can't do the automatic configuration. We used to have a "Create Token" button, but it got removed when the authorization token management got refactored a few months ago.

I believe that, in addition, improving the configuration documentation would be very useful.

Agreed. I think this is probably something that could be fixed internally to the plugin, e.g. having more description or a hint button in the window to create tokens. I've raised #59 for this.

NeeeeB commented 1 month ago

@fourls This is what my configuration looks like for project options : image

And for the settings : image

Still the same problem : image

EDIT: Now, no matter what I do or setup, I can not make it work anymore. Deleting the project key as suggested before does not work. That's really frustrating. I've setup/restarted DelphiLint server and IDE at least 20 times without success.

fourls commented 3 weeks ago

Hi @NeeeeB, @denieu,

Apologies for the delay - I have been busy with other projects. I haven't been able to find anything else that could be causing this problem.

I have just merged a PR adding support for all the token types. I'm not sure if it will resolve these issues or not, but it could be worth trying this again with latest master?

denieu commented 3 weeks ago

I tested using different types of tokens and it worked well for me.

With the change, however, the project key became mandatory in the "Project options...", so I was unable to use just one "Authorization Token" for all my projects. For each project I need to access "Settings..." and configure the token relative to the project key, this for me affects the usability of the plugin a little.

fourls commented 3 weeks ago

Hi @denieu, I'm glad all works well.

With the change, however, the project key became mandatory in the "Project options...", so I was unable to use just one "Authorization Token" for all my projects. For each project I need to access "Settings..." and configure the token relative to the project key, this for me affects the usability of the plugin a little.

That was an intentional change - if you don't provide a project key then Connected Mode doesn't know which project to connect to, so you won't get all the benefits of being connected:

I appreciate that it's a little frustrating having to set all the tokens individually. Perhaps we could add a wildcard, e.g.

Server URL Project Key Authorization Token
https://my.sonar.server * squ_1234

that DelphiLint falls back to using if there's not a matching entry for the project key? What do you think?

denieu commented 2 weeks ago

For me, the possibility of using wildcards solves the problem very well. Especially when !58 is completed, the overall configuration will become much simpler and more intuitive.