search

WarningImHack3r / npm-update-dependencies

Update NPM dependencies from your IDE
https://plugins.jetbrains.com/plugin/21105-npm-update-dependencies
9 stars 2 forks source link

Private Packages? #27

Closed yachtwave closed 9 months ago

yachtwave commented 1 year ago

What problem are you trying to solve?

Need to detect if there are newer version of private packages.

Describe the solution you’d like

Allow setting credentials for NPM access to private package

Describe any alternatives you’ve also considered

No response

WarningImHack3r commented 1 year ago

Hi there! The latest version does support private/custom registries, but it indeed does not support credentials.
I'm not familiar with private packages, can you explain to me how it works so I can figure out a simple way to add support for them in my plugin?

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 7 days with no activity.

github-actions[bot] commented 1 year ago

This issue was closed because it has been inactive for 7 days since being marked as stale. Don't hesitate to reopen it if you think it's still relevant.

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 7 days with no activity.

github-actions[bot] commented 1 year ago

This issue was closed because it has been inactive for 7 days since being marked as stale. Don't hesitate to reopen it if you think it's still relevant.

DefiCzech commented 10 months ago

It would be nice to add. We use azure, but I am not sure what exactly you need to know to implement this feature. I can provide result of npm config ls -l or whatever you need.

WarningImHack3r commented 10 months ago

@DefiCzech I need to know how private packages work. How do you add them? Do you log in each time you want to upgrade them? Are the credentials saved somewhere?

What's missing with the current behavior of the extension? Is it crashing? Simply not reporting any updates from them? Does it need to be bind to the custom registries feature? Can the npm cli easily provide some ways for me to fetch the info I would need?

DefiCzech commented 10 months ago

How do you add them?

https://learn.microsoft.com/en-us/azure/devops/artifacts/get-started-npm?view=azure-devops&tabs=Other

Do you log in each time you want to upgrade them?

No, the credentials are saved globally (in example above are credentials saved in .npmrc file in same folder as package.json)

Are the credentials saved somewhere?

Yes, the credentials are saved globally in npm config or in .npmrc file

What's missing with the current behavior of the extension? Is it crashing? Simply not reporting any updates from them?

No it is not crashing. Yes simply not reporting any updates from private package

Does it need to be bind to the custom registries feature?

I dont fully undestand the question. But I thing the answer is yes :D

Can the npm cli easily provide some ways for me to fetch the info I would need?

Yes of course you can use npm view <package> version just for latest version of package or npm view <package> versions for all versions. It works for private package (info from here https://bobbyhadz.com/blog/list-all-versions-of-npm-package it works). npm outdated works too.

WarningImHack3r commented 10 months ago

Thanks for all this info @DefiCzech! Gonna take a look this weekend maybe or next week

github-actions[bot] commented 9 months ago

This issue is stale because it has been open for 7 days with no activity.

WarningImHack3r commented 9 months ago

@DefiCzech @jeokeefe Due to professional reasons I won't be able to work on that until a few days/weeks :/

DefiCzech commented 9 months ago

Ok no problem ;)

github-actions[bot] commented 9 months ago

This issue is stale because it has been open for 7 days with no activity.

WarningImHack3r commented 9 months ago

@DefiCzech @jeokeefe that latest commit should add support for private packages. It works by gathering info from the command line (e.g.: npm v <package> version) when the fetching from an online repository fails. You can try out the new version here, and I'm waiting for your positive feedback before releasing it publicly. You can find the installation methods at the bottom of the README (Manually: ... part). (I also encourage you to delete the plugin, restart WS, and reinstall the plugin the normal way after that test is successful to make sure you'll keep getting official updates as they flow)

DefiCzech commented 9 months ago

After installing and opening package.json file, error occured. stacktrace.txt

WebStorm 2023.3 Build #WS-233.11799.229, built on December 1, 2023 Runtime version: 17.0.9+7-b1087.7 amd64 VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o. Windows 11.0

WarningImHack3r commented 9 months ago

Hmm @DefiCzech it's due to an incomplete JAR file, which is weird... I'm gonna use the usual release pipeline and hide the release on the store while I don't have your approval. You'll be able to try with the official GitHub release JAR file, which should include everything it needs. Expect a ping in ~5h!

DefiCzech commented 9 months ago

@WarningImHack3r installing plugin (zip file) from https://github.com/WarningImHack3r/npm-update-dependencies/releases/tag/v2.0.1 works great. But after adding your new (https://nopaste.net/ohz9xQIccV?d=1&f=npm-update-dependencies-2.1.0.jar) jar plugin file into zip (and remove old one), after installing the new version of private package is not detected.

WarningImHack3r commented 9 months ago

@DefiCzech yeah just wait for a complete JAR in a few hours, you'll be able to reload WS without any error and properly check the update behavior

WarningImHack3r commented 9 months ago

@DefiCzech you can try it out in GitHub releases now!

DefiCzech commented 9 months ago

@WarningImHack3r it looks not working properly (Plugin installed manualy, from https://github.com/WarningImHack3r/npm-update-dependencies/releases/tag/v2.1.0 ) image

WarningImHack3r commented 9 months ago

It makes no sense, I'm running this very command if the network fetch fails...

I tried with a dummy private package on a custom registry I have, I can try again on a dummy package I'm going to upload to npmjs.org. In the meantime, if you know Kotlin a bit you can try to take a look in the code

DefiCzech commented 9 months ago

It makes no sense, I'm running this very command if the network fetch fails...

Undestood, but is confirmed, that "network fetch fails"? I have no idea, but me personally as noob in this, I will use only npm v <package-name> version for all packages (as npm will handle this). But there is maybe some drawbacks etc. I dont know.

I am using azure as private packages registry.

In the meantime, if you know Kotlin a bit you can try to take a look in the code

Sadly I dont know Kotlin at all :(

WarningImHack3r commented 9 months ago

Undestood, but is confirmed, that "network fetch fails"?

It means when querying https://<package-registry>/<package-name>/latest fails, then I fallback to `npm v version

Also, I tried to create an npmjs account but publishing a private package is paid :/ I don't know how I'm gonna debug your issue

DefiCzech commented 9 months ago

It means when querying https://<package-registry>/<package-name>/latest fails

https://registry.npmjs.org/@npm-tee/form-elements/latest returns "not found" - but it means fails?

I can provide you Personal access token to our packages (only read access), are you able to setup on your side https://learn.microsoft.com/en-us/azure/devops/artifacts/get-started-npm?view=azure-devops&tabs=Other#setup-credentials ?

WarningImHack3r commented 9 months ago

https://registry.npmjs.org/@npm-tee/form-elements/latest returns "not found" - but it means fails?

The registry is determined from npm v <package-name> dist.tarball, which should not be the default one for that package

I can provide you Personal access token to our packages (only read access), are you able to setup on your side https://learn.microsoft.com/en-us/azure/devops/artifacts/get-started-npm?view=azure-devops&tabs=Other#setup-credentials ?

This is very nice from you, I'm gonna try that likely tomorrow after work.
I just copied it, you can delete the token part from your message.

Thank you for your availability and sorry for the time it takes to implement the feature :/

Edit: As far as I can see @DefiCzech, it seems that your registry URL is simply not extracted entirely as I'm just taking the Scheme+Root and nothing from the path! I'm gonna try this fix ASAP and will confirm if something else is wrong!

DefiCzech commented 9 months ago

The registry is determined from npm v <package-name> dist.tarball, which should not be the default one for that package

ok for me the command with my package, returns valid url. npm view @npm-tee/form-elements dist.tarball returns something like this https://pkgs.dev.azure.com/..../_packaging/.../npm/registry/@npm-tee/form-elements/-/form-elements-7.2.4.tgz

WarningImHack3r commented 9 months ago

@DefiCzech hi! I just managed to install one of your packages, and I am now working on debugging my extension! Here's my live TODO list (I'll update this message as I progress):

WarningImHack3r commented 9 months ago

@DefiCzech 👀

👀

(vite is for checking regular packages still work well)

DefiCzech commented 9 months ago

@WarningImHack3r amazing

WarningImHack3r commented 9 months ago

Update pushed, on review process! Thanks for all @DefiCzech, you've been really helpful (and patient 😅), hope you'll enjoy the feature!

I'll obviously delete your credentials from my PC and my notes, thanks for that!

DefiCzech commented 9 months ago

Nice I am looking forward for update ;) I am revoking token tho. Thank you.

DefiCzech commented 9 months ago

@WarningImHack3r I dont know why (maybe some settings, Webstorm version, operating system Win etc.), but plugin does not check my private packages. I have no idea what is wrong :( image

WarningImHack3r commented 9 months ago

@DefiCzech oh no... can you see the repository you gave me access to when running npm config ls? Could you let me know where your credentials are for this repo stored? In a global or local .npmrc?

DefiCzech commented 9 months ago

can you see the repository you gave me access to when running npm config ls?

yes I can see @npm-tee:registry = "<URL_TO_MY_REGISTRY>"

Could you let me know where your credentials are for this repo stored? In a global or local .npmrc?

Credentials are stored globaly (in user .npmrc file C:\Users\defic\.npmrc), but registry info is stored in project local .npmrc file E:\Projects\DTCZ\Frontend\DTCZ.FE.Web\.npmrc

WarningImHack3r commented 9 months ago

@DefiCzech Let me describe to you step by step what my code does, maybe I'm missing something or our configurations are different because I literally tested with this package and your credentials:

So maybe try to see if by mentally (or programmatically) reproducing these exact steps you get something, else what I got wrong. Note that I don't need any credentials because the npm v command is supposed to bind creds to registries by itself.

Sorry for the issue again...

DefiCzech commented 9 months ago

... because I literally tested with this package and your credential

I know, so I am so surprised why it is not working for me.

Before any first scan for updates or deprecations, I make one round around all registries I can find, using the following process

I get this - on my side if I use your algorithm

https://pkgs.dev.azure.com/<ORGANIZATION_NAME>/_packaging/<FEED_NAME>/npm/registry/
https://pkgs.dev.azure.com/<ORGANIZATION_NAME>/_packaging/<FEED_NAME>/npm/registry/
https://pkgs.dev.azure.com/<ORGANIZATION_NAME>/<ID_OF_SOMETHING>/_packaging/<FEED_NAME_NORMAILZED>/npm/registry/
https://pkgs.dev.azure.com/<ORGANIZATION_NAME>/<ID_OF_SOMETHING>/_packaging/<FEED_NAME_NORMAILZED>/npm/registry/
https://pkgs.dev.azure.com/<ORGANIZATION_NAME>/<ID_OF_SOMETHING>/_packaging/<FEED_NAME_NORMAILZED>/npm/registry/
https://pkgs.dev.azure.com/<ORGANIZATION_NAME>/_packaging/<FEED_NAME>/npm/registry/

Lines with ID are not valid registries, may be it could couse problems, dont know. Only first, second and last lines are valid registry. For your credetinals for my packages all lines were valid registries. I am using different method for creating credentials - vsts-npm-auth tool for creating credentials vsts-npm-auth -config .npmrc

Everything else should be the same.

I know you told me before. But why you just dont check version by npm v <package-name> version and let npm handle it (packages, credentials etc.)

I will check if plugin works if all lines in .npmrc are valid registries.

Edit: I checked plugin with same config as I provided to you without luck. So I assume it will be some OS (windows) issue, or webstorm or something on my end.

WarningImHack3r commented 9 months ago

Lines with ID are not valid registries, may be it could couse problems, dont know. Only first, second and last lines are valid registry.

Yeah I know that but as all these registry URLs are saved and then looped on, it will only be successful when the right --registry is used with the package. Incorrect URLs will just don't work and the loop will just use the next one until it finds the correct URL, so it's not an issue.

I know you told me before. But why you just dont check version by npm v <package-name> version and let npm handle it (packages, credentials etc.)

Because with a simple npm v <package> ..., the default registry (https://registry.npmjs.org) is used, not the registry the package actually comes from, unfortunately. Maybe adding the @npm-tee:registry=... helps npm figure out which registry to use, but on my end with the lines you gave me, it did not get bind correctly.
I'm just doing additional work to get around and pass the --registry flag, but the outcome should be the same.
npm handles the credentials automatically when it uses the right registry though.

So I assume it will be some OS (windows) issue, or webstorm or something on my end.

Again I don't think so: as long as the right version of the plugin is running, it is platform-agnostic. I'm only using the npm command and a cross-platform fetch for web requests. That's why I took so long figuring out the best approach to gather all the info I needed: I'm always using the most generic code possible, and I never rely on things that wouldn't work for all situations/users. That's the key to the least problems and the highest quality software possible.

I will boot up my Windows PC thought, and see if the shell commands are properly launched from the program. Can you un-revoke/provide me with some new credentials so I can test again, please? I can retrieve the ones you gave me if you can make them work again.

WarningImHack3r commented 9 months ago

Thanks! Trying

Edit 1: I was right, when you don't specify a registry attached to a package (@org:registry=url), npm picks the default registry by default. If you specify it, however, npm v without --registry works fine and picks the right one.

Edit 2: everything works fine with the extension and your credentials (with or without @org:registry) on my Mac. Gonna try it on Windows now.

Edit 3: It indeed seems to not be working on Windows!! Investigating

Edit 4: It throws a "File not found" error while trying to spawn the npm command, gonna fix that in a few hours

WarningImHack3r commented 9 months ago

Spawning shell commands actually fails on Windows, reopening until it's fixed despite not being tightly linked to this feature

WarningImHack3r commented 9 months ago

@DefiCzech fixed by 0bef600, releasing 2.1.1!

Thanks again for all your help :)

DefiCzech commented 9 months ago

@WarningImHack3r whooha, amazing job - it works. Love it, Thank you.