search

loic-sharma / BaGet

A lightweight NuGet and symbol server
https://loic-sharma.github.io/BaGet/
MIT License
2.59k stars 650 forks source link

[Question] BaGet as PowerShell module repository #427

Open OCram85 opened 4 years ago

OCram85 commented 4 years ago

❓ Question

Did anyone managed to run BaGet as PowerShell module repository sucessfully?

📑 Details

It seems like there are several discussions about the usage of BaGet and PowerShell in general.

Especially finding and installing packages via PowerShell (PackageManagement & PowerShellGet) seems to be unstable.

BaGet would be the first PackageRespository type using NuGet v3 API. Could the issues relate to this?

💣 Testing Details

🖼 Screenshots

BaGet

🔖 Refs

tiksn commented 4 years ago

Right now we have the same issue

OCram85 commented 4 years ago

@loic-sharma: We really need this to be fixed 😀

Is there any way to help? - Maybe we could provide additional analysis data of specific test scenarios?

OCram85 commented 4 years ago

BUMP.

@loic-sharma: Any updates about this?

loic-sharma commented 4 years ago

Hi, I apologize for the very long response. This requires supporting the legacy NuGet V2 APIs, which is tracked by https://github.com/loic-sharma/BaGet/issues/43. @Mizipzor and @jgagnaire made good progress on this (see https://github.com/loic-sharma/BaGet/pull/294 and https://github.com/loic-sharma/BaGet/pull/178).

I'll admit that I've been dragging my feet here as I feel torn on adding "legacy" APIs to BaGet. However, I believe that supporting Powershell modules and fully replacing NuGet.Server's feature set feels valuable enough. I will tackle this as part of BaGet V1 soon.

Jaykul commented 4 years ago

It's worth pointing out that PowerShellGet v3 is already in progres and doesn't intend to remain fully backward compatible @loic-sharma -- perhaps (assuming it uses the new v3 APIs, since they're commiting to using NuGet.Client) PowerShell is not a good enough reason to re-implement old APIs ...

softlion commented 4 years ago

Especially when powershell core is out and fully incompatible ;) new name: pwsh ! Includes SSH client !!!

Jaykul commented 4 years ago

@softlion except that PowerShell Core is backward compatible, and still ships the same old (v2) NuGet-based PowerShellGet dependency manager, so it doesn't change anything in this regard...

OCram85 commented 4 years ago

So If I get this right, the problem could be fixed when:

jeroenlandheer commented 4 years ago

I have tried to install PowerShellGet v3 (Beta 4) and my results were not that encouraging 😢 

PS> Get-PSResourceRepository

Name          Url                                           Trusted Priority
----          ---                                           ------- --------
MyRepository  https://[REDACTED]/v3/index.json               true    50
PSGallery     https://www.powershellgallery.com/api/v2      false   50

PS> Get-PSResourceRepository MyRepository | fl

Name     : MyRepository
Url      : https://[REDACTED]/v3/index.json
Trusted  : true
Priority : 50

PS> Find-PSResource -Repository MyRepository
No cursor found. Defaulting to 01/01/0001 00:00:00.
Find-PSResource: An invalid request URI was provided. The request URI must either be an absolute URI or BaseAddress must be set.
PS> Set-PSResourceRepository MyRepository -URL https://[REDACTED]/api/v2/package
PS> Find-PSResource -Repository MyRepository
Find-PSResource: 'doctype' is an unexpected token. The expected token is 'DOCTYPE'. Line 1, position 3.
PS> Set-PSResourceRepository MyRepository -URL https://[REDACTED]/api/v2
PS> Find-PSResource -Repository MyRepository
Find-PSResource: 'doctype' is an unexpected token. The expected token is 'DOCTYPE'. Line 1, position 3.
PS> Set-PSResourceRepository MyRepository -URL https://[REDACTED]/api/v3
PS> Find-PSResource -Repository MyRepository
Find-PSResource: 'doctype' is an unexpected token. The expected token is 'DOCTYPE'. Line 1, position 3.
PS> Set-PSResourceRepository MyRepository -URL https://[REDACTED]/api/v3/package
PS> Find-PSResource -Repository MyRepository
Find-PSResource: 'doctype' is an unexpected token. The expected token is 'DOCTYPE'. Line 1, position 3.
PS> Set-PSResourceRepository MyRepository -URL https://[REDACTED]/api/
PS> Find-PSResource -Repository MyRepository
Find-PSResource: 'doctype' is an unexpected token. The expected token is 'DOCTYPE'. Line 1, position 3.
PS>

I've tried several different URL's, none of them really work. Am I missing something? (The first attempt did give a different message compared to the rest...)

TylerLeonhardt commented 3 years ago

Hello everyone 👋 I work on the PowerShell team and wanted to say a couple things:

  1. BaGet is amazing. A really sleek NuGet server AND client that works on .NET Core... I'm in love
  2. PowerShellGet v2 is what ships in Windows as a part of Windows PowerShell so needless to say... PSGet v2 is here to stay for a long long time...
  3. I was able to get Publish-Module working which is really exciting 😁
  4. However, like others here, Find and Install do not work

It would be awesome to see the addition of the last couple of v2 APIs to get Find/Install working 🙂

brajjan commented 3 years ago

I got it to install the module using PowershellGet v3.0.0-beta10 Published the module through v2, have not tried to publish through v3

~ ❯ Find-PSResource -Name NetworkingDsc -Repository Baget

Name          Version Repository Description
----          ------- ---------- -----------
NetworkingDsc 8.1.0.0 Baget      DSC resources for configuring settings related to networking.

~ ❯ Install-PSResource -Name NetworkingDsc -Repository Baget -Verbose -Debug

DEBUG: Parameters passed in >>> Name: 'NetworkingDsc'; Version: ''; Prerelease: 'False'; Repository: 'Baget'; Scope: ''; AcceptLicense: 'False'; Quiet: 'False'; Reinstall: 'False'; TrustRepository: 'False'; NoClobber: 'False';
DEBUG: Entering InstallHelper::ProcessInstallParams
DEBUG: Console is elevated: 'True'
DEBUG: Console is Windows PowerShell: 'False'
DEBUG: Current user scope installation path: 'C:\Users\username\Documents\PowerShell'
DEBUG: All users scope installation path: 'C:\Program Files\PowerShell'
VERBOSE: Scope is: CurrentUser
DEBUG: Checking to see if paths exist
DEBUG: Path: 'C:\Users\username\Documents\PowerShell\Modules'  >>> exists? 'True'
DEBUG: Path: 'C:\Users\username\Documents\PowerShell\Scripts'  >>> exists? 'True'
DEBUG: Path: 'C:\Users\username\Documents\PowerShell\Scripts\InstalledScriptInfos'  >>> exists? 'True'
DEBUG: Attempting to search for packages in 'Baget'
DEBUG: Untrusted repository accepted as trusted source.
VERBOSE: Begin installing package: 'NetworkingDsc'
DEBUG: Successfully able to download package from source to: 'C:\Users\username\AppData\Local\Temp\bc139dfc-f5c4-4057-96ea-fd22bd918bbe'
DEBUG: Deleting 'C:\Users\username\AppData\Local\Temp\bc139dfc-f5c4-4057-96ea-fd22bd918bbe\networkingdsc\8.1.0\networkingdsc.8.1.0.nupkg.sha512'
DEBUG: Deleting 'C:\Users\username\AppData\Local\Temp\bc139dfc-f5c4-4057-96ea-fd22bd918bbe\networkingdsc\8.1.0\networkingdsc.nuspec'
DEBUG: Deleting 'C:\Users\username\AppData\Local\Temp\bc139dfc-f5c4-4057-96ea-fd22bd918bbe\networkingdsc\8.1.0\networkingdsc.8.1.0.nupkg'
DEBUG: Installation path is: 'C:\Users\username\Documents\PowerShell\Modules\NetworkingDsc'
VERBOSE: Full installation path is: 'C:\Users\username\AppData\Local\Temp\bc139dfc-f5c4-4057-96ea-fd22bd918bbe\networkingdsc'
DEBUG: Attempting to move 'C:\Users\username\AppData\Local\Temp\bc139dfc-f5c4-4057-96ea-fd22bd918bbe\networkingdsc' to 'C:\Users\username\Documents\PowerShell\Modules\NetworkingDsc
VERBOSE: Successfully installed package NetworkingDsc
DEBUG: Attempting to delete 'C:\Users\username\AppData\Local\Temp\bc139dfc-f5c4-4057-96ea-fd22bd918bbe'

~ ❯ Get-DscResource -Module NetworkingDsc -Name NetIPInterface

ResourceType  : DSC_NetIPInterface
Name          : NetIPInterface
FriendlyName  : NetIPInterface
Module        : NetworkingDsc
ModuleName    : NetworkingDsc
Version       : 8.1.0
Path          : C:\Users\username\Documents\PowerShell\Modules\NetworkingDsc\8.1.0\DscResources\DSC_NetIPInterface\DSC_NetIPInterface.psm1
ParentPath    : C:\Users\username\Documents\PowerShell\Modules\NetworkingDsc\8.1.0\DscResources\DSC_NetIPInterface
ImplementedAs : PowerShell
CompanyName   : DSC Community
Properties    : {AddressFamily, InterfaceAlias, AdvertiseDefaultRoute, Advertising…}
VertigoRay commented 3 years ago

@brajjan, I see you got it to work with PowerShell Core using DSC. Did it also work for Install-Module or Find-Module? What about PowerShell Desktop (5.1)?

I tried BaGet and LiGet. Was getting the issues on both, but I didn't make the issues over here, yet:

~I'll try to make time today to make similar detailed output for BaGet.~ We're using sunside/simple-nuget-server instead. I'd prefer BaGet because it's beautiful, as @TylerLeonhardt said. However, we need it functional now. I'll keep an eye on this thread. I hope things get resolved here and we can swap to BaGet in the future.

TylerLeonhardt commented 3 years ago

@VertigoRay there's no frontend for that one is there?

VertigoRay commented 3 years ago

@TylerLeonhardt, you are correct. The meta and downloads are available though. I imagine something could be made relatively easy.

$module = 'ActiveDiretory'
$url = "https://simple-nuget-server.contoso.com/FindPackagesById()?id=%27${module}%27&$skip=0&$top=40
(Invoke-RestMethod $url)[0].properties.DownloadCount.'#text'

I would prefer to not fork that project. I would rather put time into this one. I just need to have time to do that. 😏

Xeevis commented 3 years ago

PowershellGet v3.0.0-beta10 seems to be working fine.

# Install latest prelease version of PowershellGet
Install-Module -Name PowerShellGet -AllowPrerelease -Force
# Register BaGet repository
Register-PSResourceRepository -Name 'BaGet' -Url 'http://127.0.0.1:5555/v3/index.json' -Trusted
# Publish resource
Publish-PSResource -Path E:\PowerShell\PSCalendar -Repository BaGet
# Find module
Find-PSResource PSCalendar -Repository BaGet
# Install module
Install-PSResource PSCalendar -Repository BaGet
# Run module function
Show-Calendar
MS-LUF commented 3 years ago

Hello, I have tried PowershellGet v3.0.0-beta10. It works in HTTP but not using TLS. Each time the client is trying to communicate without TLS, even if the repo is set with a HTTPS uri... Any idea . Perhaps Xeevis, did you try to expose your repo using a TLS configuration ?

jeroenlandheer commented 3 years ago

@MS-LUF I've ran a couple of tests and I can confirm that using TLS this is not working on Beta 10 atm. (We have BaGet deployed on a linux kubernetes cluster with an NGINX front-end proxy.)

Once a new version of PowerShellGet is released I'll try again.

TylerLeonhardt commented 3 years ago

Might be worth getting an issue going on PowerShellGet?

MS-LUF commented 3 years ago

Might be worth getting an issue going on PowerShellGet?

Already done :) https://github.com/PowerShell/PowerShellGet/issues/314

It's a shame that in 2020 a web component does not support TLS ! Moreover, TLS is supported for default repository (PSGallery). I don't understand why they have splitted the behavior in their code...

OCram85 commented 2 years ago

Hosting a private NuGet service for Powershell is was a hell in 2021. The problem here isn't the server part at all.

Every time I had to connect a new client to the NuGet based service it scared the shit out of me because:

jeroenlandheer commented 2 years ago

@OCram85 The issues you describe are very distinct.

Hope this clarifies a few things.

OCram85 commented 2 years ago

You're absolutely right. My comment wasn't really clear and much more impulsive than it was intended.

Just wanted to describe the situation I'm facing in a heterogen environment with:

I've invested weeks in the last years to get these systems hooked up with a private NutGet Server like PowershellGallery (BaGet, Nexus Repository....). In some cases it works instant and others you end up reading PowerShellGet and PackageManagement issues / workarounds.

So to sum up I just want to tell my personal opinion about this situation:

I really like working and developing modules for pwsh. But If you need to deploy your modules internally you're really faced with some additional challenges. It seems to me like the PowerShellGet based package management is the only valid way to publish and consume pwsh modules. Unfortunately I don't see much progress in solving the existing issues in PowerShellGet v2, or even a clear roadmap when to expect the PowerShellGet v3.

😄

commutecomfort commented 5 months ago

The following worked for me. It can be a workaround/ solution depending on your requirements:

Download the Nuget client from the following URL. Nuget client is a .exe file. It does not need any installation, not does it need admin rights to execute.

https://learn.microsoft.com/en-us/nuget/install-nuget-client-tools?tabs=windows#install-nugetexe

Then, open a PS window with the folder where nuget.exe was downloaded. Run the following command to add BaGet as a source for NuGet client (nuget.exe).

.\nuget sources Add -Name "BaGet" -Source "http://localhost:5000/v3/index.json"

If the source BaGet has already been added, then run the following command to configure it's settings. (Optional)

.\nuget sources Update -Name "BaGet" -Source "http://localhost:5000/v3/index.json"

Use the following commands to push packages to Baget. These packages are downloaded in the form of .nupkg file from the PowerShell Gallery. The -ApiKey parameter is the value set in appsettings.json for the ApiKey.

.\nuget push -Source BaGet -ApiKey SuperSecret packagemanagement.1.4.8.1.nupkg

.\nuget push -Source BaGet -ApiKey SuperSecret powerstig.4.20.0.nupkg

.\nuget push -Source BaGet -ApiKey SuperSecret get-windowsautopilotinfo.3.9.0.nupkg

Run the following command to install the package on local machine (where you access the BaGet webpage from). This works on Windows. It can possibly work on Linux as well with the equivalent of NuGet.exe for the linux world.

.\nuget install < < name of the package > > -Source BaGet

.\nuget install get-windowsautopilotinfo -Source BaGet

Then, navigate to the directory where the package (PS module) was installed. If you are installing it for -Scope CurrentUser, the directory would be: "C:\users\< < username > >\documents\windowspowershell\modules\ < < name of the PS Module > > ".

Now the .psm1 file needs to be imported. Run the following command to do so. You may also browse to the module location with Windows Explorer and look for the name of .psm1 file. Usually, it is the same as the module name. In some cases, you many need to import .psd1 file as well, for the PS module to work.

import-module < <Path to the .psm1 file> >

import-module .\powerstig.4.20.0.psm1 import-module .\get-windowsautopilotinfo.3.9.0.psm1

Run the following command to see the newly added PS cmdlets from the installed module(s). Get-Command -Module < < Name of the installed module > >

The following is a link to Nuget CLI documentation. https://learn.microsoft.com/en-us/nuget/reference/cli-reference/cli-ref-sources

Jaykul commented 5 months ago

For what it's worth...

After all these years, the new version (named PSResourceGet) has finally released with support for the v3 feed format. It might be worth testing with that (and opening an issue on the new repo if it doesn't work) 😉

Also, about the compatibility thing... Although backward compatibility is obviously never trivial, writing PowerShell modules that work across multiple versions back to Windows PowerShell (5.1) and on Windows, Linux and OSX is not that hard -- and PSResourceGet for instance, should work across them all.