[Improvement] Allow custom Nuget source for tools library

mike9182 commented 1 year ago

Is your feature request related to a problem? Please describe. Some organizations do not permit software to be downloaded directly from nuget.org

Describe the solution you'd like Allow the Tool Library to be configured to use an internal nuget repository such as Nexus.

Describe alternatives you've considered It may be possible to install the tools manually, but this is a bit of a pain.

Additional context None

MscrmTools commented 1 year ago

I don't have access nor I know anything about Nexus.

Can you explain if Nexus web services work the exact same way as the original Nuget one (same search service url, same response format, etc)?

If so, that should be possible to precise what url to use for the repository

mike9182 commented 1 year ago

This is very similar to specifying an alternative package source in Visual Studio. This package manager works identically to the nuget.org feed. I think the only change required is to allow specifying an alternative URL instead of hardcoding the "https://api.nuget.org/v3/index.json" url in StoreFromPortal.cs when retrieving packages.

Screenshot is from Visual Studio 2022

MscrmTools commented 1 year ago

The point is there is a prevalidation of tools available performed manually in my XrmToolBox portal. If I allow a custom Nuget feed, that means I don't have any control over tools listed. These tools could break XrmToolBox and users would come here to report their issue. Also, this custom Nuget feed should contains only XrmToolBox tools AND I should handle parsing content of tools from the Nuget feed instead of XrmToolBox portal. It feels a lot of work for a minority of users and companies (not mentioning I have no access to such a custom feed)

mike9182 commented 1 year ago

I think there is some part of this that I don't understand since the change feels simple to me. If I have time, I may try creating a NuGet feed in Azure Devops and test with that. Either way, if this is difficult to implement, or if you choose not to implement for other reasons, such as maintenance or support, it seems simple enough to just download and install the tools manually. :relaxed:

To explain Nexus some more, it is used as an internal repository and connects to other feeds, such as Nuget.org. It then scans the packages available in that feed before making them available within the organization. So, the end result is that all approved packages from nuget.org become available within the organization.

MscrmTools commented 1 year ago

I don't see a lot of complexity of implementing BUT it's just that Tool Library relies primarily on XrmToolBox portal feed to get the list and metadata of tools available. Nuget comes after to get extra informations like versions and download url. But if you remove XrmToolBox portal feed, it just does not work.

Adding a nuget feed directly means :

rewriting the way tools are retrieved (bypass XrmToolBox portal feed)
getting all information from Nuget feed instead of XrmToolBox portal feed

mike9182 commented 1 year ago

Thanks for the clarification. I don't see the use of the XrmToolbox portal feed as an issue. For the problem that I am trying to solve, the important part is that the plugin code be downloaded from an internal Nuget repo instead of from Nuget.org directly.

As long as all packages being used have been approved by the organization, then a repository such as Nexus is a drop-in replacement for nuget.org. For example, I could take a Visual Studio project that was using nuget.org, change the package source to the internal Nexus repo, and then redownload all packages without any changes to the project itself.

MscrmTools commented 1 year ago

"the important part is that the plugin code be downloaded from an internal Nuget repo instead of from Nuget.org directly." => I do understand this but the point is that currently, there is no direct access to nuget.org (well, yes, for downloading packages). There is first an access to XrmToolBox portal feed and bypassing it would require some workload.

mike9182 commented 1 year ago

My point is that the only step where the alternate Nuget Feed matters is in the step of downloading the packages. Downloading the packages from an internal repository such as Nexus ensures that the organization has the opportunity to scan the packages. These organizations do not allow using packages directly from Nuget since they have not been scanned.

I do not see any issues with the step of first accessing the XrmToolbox portal feed.

MscrmTools commented 1 year ago

Hum... ok... I was more thinknig about alternate nuget feed with tools not registered on XrmToolBox (live private company tools).

But I now got your point.

Is there any chance you are aware about documentation that would explain what are the links to use for alternate nuget feed? I would like to understand what is the "custom" part of an alternate feed url and what is common with the standard nuget feed.

michaelmittermair commented 1 year ago

Same from my side. Package download directly from nuget.org will be declined, as they have be scanned by another platform before using it. Therefore loading the packages from another url would help.

josefreitas commented 12 months ago

Hi, I have the same issue with Nexus.

I created an API to merge the Nuget and the Nexus packages into one list, which works.

However, to get the versions, the Nuget.org endpoint is hardcoded. Is it possible to make that configurable? Because the Nexus uses the same API version and properties as Nuget.

MscrmTools commented 12 months ago

That is something that can be done but I would need a test endpoint to be sure everything is working correctly. Or you can fork the repository, do the change, test it and submit a PR

MscrmTools commented 10 months ago

@mike9182 @michaelmittermair @josefreitas I'm starting to work on this issue. What I see so far are two urls that are in use:

url for downloading a package. This information comes from XrmToolBox portal directly and should be replaced dynamically for an alternative url. Url is like the following : https://api.nuget.org/v3-flatcontainer/mscrmtools.accesschecker/1.2022.3.6/mscrmtools.accesschecker.1.2022.3.6.nupkg
url for searching additional inforamtion useful for the Tool Library. It's hard coded for now with https://api-v2v3search-0.nuget.org/query?q={packageName}

So we have two urls. Can you tell me what the same url would look like with Nexus? This is to understand if I need to create two new settings (one for each url) or if the hostname will be the same for both url in Nexus

Thanks for your help

michaelmittermair commented 9 months ago

We already created a custom startup-tool, that is checking the version of the nuget on our teamcity. New versions are automatically downloading and extracted. Therefore, we have an updated version of our custom plugins automatically.

Our teamcity has the following url: https://teamcity.intern/httpAuth/app/nuget/feed/Dynamics365/v3/index.json (which would be the second our in your case).

For the first url, it would be interessting, if the information of the nuspec-file (package info) can be used to display. The required information for each package is added to the nuspec-file, for example:

`<?xml version="1.0" encoding="utf-8"?>

MscrmTools / XrmToolBox

[Improvement] Allow custom Nuget source for tools library #1217