npm-leech do not download dependencies with tilde (~) and caret (^) in versions

[roarbr]

Hi, While using npm-leech to download a package with a lot of dependencies, some dependencies failed to download with an error message like this:

pkg-err { Error: Request failed with status code 405
    at createError (/usr/lib/node_modules/npm-leech/node_modules/axios/lib/core/createError.js:16:15)
    at settle (/usr/lib/node_modules/npm-leech/node_modules/axios/lib/core/settle.js:17:12)
    at IncomingMessage.handleStreamEnd (/usr/lib/node_modules/npm-leech/node_modules/axios/lib/adapters/http.js:236:11)
    at emitNone (events.js:91:20)
    at IncomingMessage.emit (events.js:185:7)
    at endReadableNT (_stream_readable.js:978:12)
    at _combinedTickCallback (internal/process/next_tick.js:80:11)
    at process._tickCallback (internal/process/next_tick.js:104:9)
  config: 
   { url: 'http://registry.npmjs.org/accepts/~1.3.7',
...
     _header: 'GET /accepts/~1.3.7 HTTP/1.1\r\nAccept: application/json, text/plain, */*\r\nUser-Agent: axios/0.19.2\r\nHost: registry.npmjs.org\r\nConnection: close\r\n\r\n',
...
        responseUrl: 'http://registry.npmjs.org/accepts/~1.3.7',
        redirects: [],
        read: [Function] } },
  response: 
   { status: 405,
     statusText: 'Method Not Allowed',

The package.json used with npm-leech version 1.2.1:

  "name": "foo",
  "version:": "1.0.0",
  "dependencies": {
    "express": "latest"
  }
}

The package 'express' dependencies, only showing the two first:

npm show express --json dependencies
{
  "accepts": "~1.3.7",
  "array-flatten": "1.1.1",

The dependency 'array-flatten' has a "normal" semver and is downloaded without problems. But the dependency 'accepts' has a tihlde in the semver. This causes npm-leech to download this version without resolving the tihlde to a "normal" semver first. But the registry does not understand "GET http://registry.npmjs.org/accepts/~1.3.7".

Using wireshark, this is the TCP stream when downloading the package 'accepts':

GET /accepts/~1.3.7 HTTP/1.1
Accept: application/json, text/plain, */*
User-Agent: axios/0.19.2
Host: registry.npmjs.org
Connection: close

HTTP/1.1 405 Method Not Allowed
Date: Wed, 05 May 2021 15:47:03 GMT
Content-Type: application/json
Content-Length: 63
Connection: close
Set-Cookie: __cfduid=db421dde183435235cd4cb62992f582121620229622; expires=Fri, 04-Jun-21 15:47:02 GMT; path=/; domain=.npmjs.org; HttpOnly; SameSite=Lax
CF-Ray: 64ab1ae7687ffac0-OSL
Allow: PUT
CF-Cache-Status: DYNAMIC
cf-request-id: 09decf24a30000fac0cd049000000001
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Server: cloudflare

{"code":"MethodNotAllowedError","message":"GET is not allowed"}

So, shouldn't npm-leech resolve tihlde and caret semvers into normal semvers before downloading packages?

[larsw]

Have you looked into (e.g. with Wireshark or Fiddler) what the npm client does when downloading caret and tilde versions?

[larsw]

I would suggest you base your downloads on the package-lock.json instead of the package.json file.