Does not support private npm registries with auth

swells commented 2 months ago

Before filling a bug

[x] have you checked the faq for known issues.
[x] have you checked existing issues

Describe the bug

On these OneBranch compliant CI/CD pipelines we must use a private internal authenticated npm registry and fails tiring to acquire autorest-core's version during bootstrapping.

Expected behavior

autorest should support this as a typical use-case and use .npmrc on private authenticated npm registries .

Here are the pipeline steps:

network isolation

extends:
  template: v2/OneBranch.Official.CrossPlat.yml@templates
    featureFlags:
      EnableCDPxPAT: false
      WindowsHostVersion:
        Network: KS3

Npm Authenticate via ADO

  - task: npmAuthenticate@0
    displayName: "Authenticate npm"
    inputs:
      workingFile: '$(Build.SourcesDirectory)\src\.npmrc'

.npmrc

registry=https://msdata.pkgs.visualstudio.com/................../npm/registry/
always-auth=true

Set env and run autorest

NOTE: The setting of the env autorest_registry: https://msdata.pkgs.visualstudio.com/................../npm/registry/

 - script: >-
      autorest
         --input-file=.some-spec.json
         --csharp 
         --namespace=SOME-NAMESPACE
         --legacy 
        --debug
    env: # Variables to map into the process's environment.
      autorest_registry: https://msdata.pkgs.visualstudio.com/................../npm/registry/

Failure on getting azure-core version

"C:\Windows\system32\cmd.exe" /D /E:ON /V:OFF /S /C "CALL "C:\__w\_temp\548d9984-5f23-47b4-8d71-afa9773675f2.cmd""
AutoRest code generation utility [cli version: 3.7.1; node: v20.11.1]
(C) 2018 Microsoft Corporation.
https://aka.ms/autorest
info    | AutoRest core version selected from configuration: ~2.0.4413.
debug   | [0.97 s] Network Enabled: true
debug   | [1.13 s] No @autorest/core (or @microsoft.azure/autorest-core) is installed.
debug   | [1.13 s] ~2.0.4413 was not satisfied directly by a previous installation.
debug   | [1.39 s] Error trying to resolve @autorest/core version ~2.0.4413: Error: Unable to resolve package '@autorest/core@~2.0.4413'.
Failure:
Error: Unable to find a valid AutoRest core package '@autorest/core' @ '~2.0.4413'.
Error: Unable to find a valid AutoRest core package '@autorest/core' @ '~2.0.4413'.
    at selectVersion (C:\__t\node\20.11.1\x64\node_modules\autorest\dist\src_autorest-as-a-service_ts.js:393:23)
    at resolveCoreVersion (C:\__t\node\20.11.1\x64\node_modules\autorest\dist\src_autorest-as-a-service_ts.js:590:29)
    at main (C:\__t\node\20.11.1\x64\node_modules\autorest\dist\app.js:98:33)

Additional context

I think what is happening is after setting the env autorest_registry autorest'ss use of pacote manifest fetch to get the azure-core version fails since it can not properly authenticate:

export async function fetchPackageMetadata(spec: string): Promise<pacote.ManifestResult> {
  try {
    return await pacote.manifest(spec, {
      cache: `${tmpdir()}/cache`,
      registry: process.env.autorest_registry || "https://registry.npmjs.org",
      "full-metadata": true,
    });
  } catch (error) {
    logger.error(`Error resolving package ${spec}`, error);
    throw new UnresolvedPackageException(spec);
  }
}

There is no way to deal with the _auth scopes from .npmrc and pacote.manifest. This is my speculation anyhow since this is difficult to debug this in CI pipelines.

timotheeguerin commented 1 month ago

Just posting here too, autorest is currently in maintanaince mode and we don't have time to commit. Using private registry with authentication is also often tied to being locked from installing dependencies after the install stage(which breaks the JIT install that autorest does)

The current recommendation for this is to setup a package.json with the autorest dependencies in it

{
 "name": "my-autorest-private-setup",
  "devDependencies": {
    "@autorest/core": "latest",
   "@autorest/modelerfour": "latest",
   "@autorest/csharp": "latest"
  }
}

npm ci

And tell autorest where to find the local versions

autorest --version "$PWD/node_modules/@autorest/core" --use "$PWD/node_modules/@autorest/modelerfour" --use "$PWD/node_modules/@autorest/csharp"

swells commented 1 month ago

Thanks @timotheeguerin for getting back to me. Where can I find out what the suggested alternative to autorest is, I don't see it here in github

I will try your suggestion above also.

Azure / autorest