search

SonarSource / sonarqube-roslyn-sdk

SDK for SonarQube Roslyn Analyzer Plugins
GNU Lesser General Public License v3.0
85 stars 43 forks source link

Unable to build jar, External packages cannot depend on packages that target projects #129

Open kevin-whalley-gc opened 5 years ago

kevin-whalley-gc commented 5 years ago

Using VS2017 15.9.14 I created a project from the Roslyn template and have been attempting to package it.

I am getting stuck on it's NuGet step of resolving dependencies

.\RoslynSonarQubePluginGenerator /a:GreatCall.CodeAnalyzer s Roslyn Analyzer Plugin Generator for SonarQube 2.0.0.0 Maximum supported Roslyn version: 2.8.0.0 Minimum supported SonarQube version: 6.7 (highest version tested against: 7.3-alpha1) [DEBUG] Parsed NuGet reference. Id: GreatCall.CodeAnalyzers, version: [DEBUG] Fetching NuGet config files... [DEBUG] Enabled package sources: [DEBUG] https://www.nuget.org/api/v2/, machine-wide: False [DEBUG] C:\LocalNuGetFeed, machine-wide: False [DEBUG] http://proget.greatcall.com/nuget/LibNugetFeed/, machine-wide: False [DEBUG] C:\Program Files (x86)\Microsoft SDKs\NuGetPackages\, machine-wide: False Attempting to locate package with id 'GreatCall.CodeAnalyzers' [DEBUG] Number of packages located: 1 [DEBUG] Package versions: 1.0.0 - latest version

[DEBUG] Using version marked as latest. Version was not specified. Using version 1.0.0. [NuGet] Attempting to resolve dependency 'Microsoft.CodeAnalysis.Analyzers (≥ 2.6.1)'. [NuGet] Attempting to resolve dependency 'Microsoft.CodeAnalysis.CSharp.Workspaces (≥ 2.9.0)'. [NuGet] Attempting to resolve dependency 'Microsoft.CodeAnalysis.CSharp (= 2.9.0)'. [NuGet] Attempting to resolve dependency 'Microsoft.CodeAnalysis.Common (= 2.9.0)'. [NuGet] Attempting to resolve dependency 'System.AppContext (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Collections (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'Microsoft.NETCore.Platforms (≥ 1.1.0)'. [NuGet] Attempting to resolve dependency 'Microsoft.NETCore.Targets (≥ 1.1.0)'. [NuGet] Attempting to resolve dependency 'System.Runtime (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Resources.ResourceManager (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Globalization (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Reflection (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.IO (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Text.Encoding (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Threading.Tasks (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Reflection.Primitives (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Threading (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Collections.Concurrent (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Diagnostics.Debug (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Diagnostics.Tracing (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Runtime.Extensions (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Collections.Immutable (≥ 1.5.0)'. [NuGet] Attempting to resolve dependency 'NETStandard.Library (≥ 1.6.1)'. [NuGet] Attempting to resolve dependency 'System.Diagnostics.Tools (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Linq (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Linq.Expressions (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Reflection.Emit.ILGeneration (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Reflection.Emit.Lightweight (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Reflection.Extensions (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Reflection.TypeExtensions (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Diagnostics.Contracts (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.ObjectModel (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Reflection.Emit (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Net.Primitives (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Runtime.Handles (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Text.Encoding.Extensions (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Text.RegularExpressions (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Xml.ReaderWriter (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.IO.FileSystem (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.IO.FileSystem.Primitives (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Runtime.InteropServices (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Threading.Tasks.Extensions (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Xml.XDocument (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.IO.Compression (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'runtime.native.System.IO.Compression (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Buffers (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'runtime.native.System (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Net.Http (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Diagnostics.DiagnosticSource (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Security.Cryptography.X509Certificates (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Security.Cryptography.Algorithms (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Security.Cryptography.Primitives (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Security.Cryptography.Encoding (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'runtime.native.System.Security.Cryptography.OpenSsl (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'runtime.debian.8-x64.runtime.native.System.Security.Cryptography.OpenSsl (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'runtime.fedora.23-x64.runtime.native.System.Security.Cryptography.OpenSsl (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'runtime.fedora.24-x64.runtime.native.System.Security.Cryptography.OpenSsl (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'runtime.opensuse.13.2-x64.runtime.native.System.Security.Cryptography.OpenSsl (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'runtime.opensuse.42.1-x64.runtime.native.System.Security.Cryptography.OpenSsl (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.OpenSsl (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'runtime.rhel.7-x64.runtime.native.System.Security.Cryptography.OpenSsl (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'runtime.ubuntu.14.04-x64.runtime.native.System.Security.Cryptography.OpenSsl (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'runtime.ubuntu.16.04-x64.runtime.native.System.Security.Cryptography.OpenSsl (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'runtime.ubuntu.16.10-x64.runtime.native.System.Security.Cryptography.OpenSsl (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'runtime.native.System.Security.Cryptography.Apple (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.Apple (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Runtime.Numerics (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Globalization.Calendars (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Security.Cryptography.Cng (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'runtime.native.System.Net.Http (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Security.Cryptography.Csp (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Security.Cryptography.OpenSsl (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Runtime.WindowsRuntime (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'Microsoft.Win32.Primitives (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Globalization.Extensions (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Runtime.InteropServices.RuntimeInformation (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Threading.Timer (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Console (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.IO.Compression.ZipFile (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Net.Sockets (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Diagnostics.FileVersionInfo (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Reflection.Metadata (≥ 1.4.1)'. [NuGet] Attempting to resolve dependency 'System.Diagnostics.StackTrace (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Dynamic.Runtime (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Reflection.Metadata (≥ 1.6.0)'. [NuGet] Attempting to resolve dependency 'System.Text.Encoding.CodePages (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Threading.Tasks.Parallel (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Threading.Thread (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.ValueTuple (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Xml.XmlDocument (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Xml.XPath.XDocument (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'System.Xml.XPath (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'Microsoft.CodeAnalysis.Workspaces.Common (= 2.9.0)'. [NuGet] Attempting to resolve dependency 'System.Composition (≥ 1.0.31)'. [NuGet] Attempting to resolve dependency 'System.Composition.AttributedModel (≥ 1.0.31)'. [NuGet] Attempting to resolve dependency 'System.Composition.Convention (≥ 1.0.31)'. [NuGet] Attempting to resolve dependency 'System.Composition.Hosting (≥ 1.0.31)'. [NuGet] Attempting to resolve dependency 'System.Composition.Runtime (≥ 1.0.31)'. [NuGet] Attempting to resolve dependency 'System.Composition.TypedParts (≥ 1.0.31)'. [NuGet] Attempting to resolve dependency 'System.Linq.Parallel (≥ 4.3.0)'. [NuGet] Attempting to resolve dependency 'SQLitePCLRaw.bundle_green (≥ 1.1.2)'. [NuGet] Attempting to resolve dependency 'SQLitePCLRaw.core (≥ 1.1.2)'. [NuGet] Attempting to resolve dependency 'SQLitePCLRaw.provider.e_sqlite3.android (≥ 1.1.2)'. [NuGet] Attempting to resolve dependency 'SQLitePCLRaw.lib.e_sqlite3.android (≥ 1.1.2)'. [NuGet] Attempting to resolve dependency 'SQLitePCLRaw.provider.sqlite3.ios_unified (≥ 1.1.2)'. [NuGet] Attempting to resolve dependency 'SQLitePCLRaw.provider.e_sqlite3.macos (≥ 1.1.2)'. [NuGet] Attempting to resolve dependency 'SQLitePCLRaw.lib.e_sqlite3.osx (≥ 1.1.2)'. [NuGet] Attempting to resolve dependency 'SQLitePCLRaw.provider.e_sqlite3.wpa81 (≥ 1.1.2)'. [NuGet] Attempting to resolve dependency 'SQLitePCLRaw.lib.e_sqlite3.v120_wp81 (≥ 1.1.2)'. [NuGet] Attempting to resolve dependency 'SQLitePCLRaw.provider.e_sqlite3.wp80 (≥ 1.1.2)'. [NuGet] Attempting to resolve dependency 'SQLitePCLRaw.lib.e_sqlite3.v110_wp80 (≥ 1.1.2)'. [NuGet] Attempting to resolve dependency 'SQLitePCLRaw.provider.e_sqlite3.win8 (≥ 1.1.2)'. [NuGet] Attempting to resolve dependency 'SQLitePCLRaw.lib.e_sqlite3.v110 (≥ 1.1.2)'. [NuGet] Attempting to resolve dependency 'SQLitePCLRaw.provider.e_sqlite3.win81 (≥ 1.1.2)'. [NuGet] Attempting to resolve dependency 'SQLitePCLRaw.lib.e_sqlite3.v120 (≥ 1.1.2)'. [NuGet] Attempting to resolve dependency 'SQLitePCLRaw.provider.e_sqlite3.uwp10 (≥ 1.1.2)'. [NuGet] Attempting to resolve dependency 'SQLitePCLRaw.lib.e_sqlite3.v140 (≥ 1.1.2)'. [NuGet] Attempting to resolve dependency 'SQLitePCLRaw.provider.e_sqlite3.net35 (≥ 1.1.2)'. [NuGet] Attempting to resolve dependency 'SQLitePCLRaw.lib.e_sqlite3.v110_xp (≥ 1.1.2)'. [NuGet] Attempting to resolve dependency 'SQLitePCLRaw.lib.e_sqlite3.linux (≥ 1.1.2)'. [NuGet] Attempting to resolve dependency 'SQLitePCLRaw.provider.e_sqlite3.net40 (≥ 1.1.2)'. [NuGet] Attempting to resolve dependency 'SQLitePCLRaw.provider.e_sqlite3.net45 (≥ 1.1.2)'. [NuGet] Attempting to resolve dependency 'SQLitePCLRaw.provider.e_sqlite3.netstandard11 (≥ 1.1.2)'. Failed to install package: External packages cannot depend on packages that target projects. Check that there are released versions of the package, or specify a pre-release version identifier. PS C:\Users\kevin.whalley\Downloads\SonarQube.Roslyn.SDK-2.0>

GreatCall.CodeAnalyzers.1.0.0.zip

ferdiozturk commented 4 years ago

I have the same issue, when trying to pluginize "Puma.Security.Rules" package... first there were circular dependencies (which I resolved by editing the .nuspecs inside the installed .nupkgs) - then I get "Failed to install package: External package [...]" after attempt to resolve dep Microsoft.CodeAnalysis.VisualBasic

I suggested it somewhere else already, please try the PluginGenerator.exe on all the roslyn analyzers, and release the jars in this repo also? That way not each and every one of us has to run into these problems...

second suggestion: better logging, please? "Failed to install package: External [...]" without any hint what package seems to be the problem is kind of very very minimal and does not name that errorneous package/dependency at all.

BTW: I installed the microsoft.codeanalysis.visualbasic to my local Nuget repo, the PluginGenerator can find it. I know that for sure. The error is about something else, (very old) bug in NuGet? see https://danlimerick.wordpress.com/2011/10/01/getting-around-nugets-external-package-dependency-problem/

sebastien-marichal commented 3 months ago

Hello @kevin-whalley-gc, @ferdiozturk,

Adter some investigation, it seems that, indeed, the issue is coming from NuGet itself (as you said @ferdiozturk).

I suggested it somewhere else already, please try the PluginGenerator.exe on all the roslyn analyzers, and release the jars in this repo also? That way not each and every one of us has to run into these problems...

@ferdiozturk, I am not sure I fully understand what you are suggesting. Do you suggest finding all analyzer packages on NuGet, bundling them, and providing them in the repository?

second suggestion: better logging, please? "Failed to install package: External [...]" without any hint what package seems to be the problem is kind of very very minimal and does not name that errorneous package/dependency at all.

Unfortunately, improving the error message would be very difficult as the issue is coming from NuGet itself. When getting the exception, we are missing the resolution context. We can only assume that the last resolution attempt was the faulty package.

My suggestion for the future is to rework how the SDK resolves package dependency. It is a big rework. Instead of referencing Nuget.exe we should use the APIs provided by the following packages:

Small POC (does not resolve dependencies):

string packageId = "Puma.Security.Rules";
string packageVersion = "2.4.11";
var logger = new ConsoleLogger();
var settings = new Settings(@"C:\NugetPackages");

var packageSource = new PackageSource("https://api.nuget.org/v3/index.json");
var sourceRepository = new SourceRepository(packageSource, Repository.Provider.GetCoreV3());
var package = new PackageIdentity(packageId, NuGetVersion.Parse(packageVersion));
var consoleProjectContext = new ConsoleProjectContext(logger)
{
    PackageExtractionContext = new PackageExtractionContext(
        PackageSaveMode.Defaultv3,
        XmlDocFileSaveMode.Skip,
        ClientPolicyContext.GetClientPolicy(settings, logger),
        logger)
};

var cacheSourceProvider = new CachingSourceProvider(new PackageSourceProvider(settings));
var nuGetPackageManager = new NuGetPackageManager(cacheSourceProvider, settings, @"C:\NugetPackages");
await nuGetPackageManager.RestorePackageAsync(
    package,
    consoleProjectContext,
    new PackageDownloadContext(new SourceCacheContext()),
    [sourceRepository], CancellationToken.None);

class ConsoleLogger : LoggerBase
{
    public override void Log(ILogMessage message)
    {
        Console.WriteLine($"[{message.Level}] {message.Message}");
    }

    public override Task LogAsync(ILogMessage message)
    {
        Log(message);
        return Task.CompletedTask;
    }
}