DownloadError: Download failed for url "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-5.0.19.tgz"

arpitgoyal43 commented 10 months ago

Versions

NodeJS: 18.16.0
mongodb-memory-server-*: 8.16.0
mongodb(the binary version): 5.0.19
system: linux (in gitlab pipeline)

package: mongo-memory-server : 8.16.0

What is the Problem?

Getting the error for unit tests while running in gitlab pipeline. It was working as expected till now and recently started seeing this error in pipeline continously.

console.error MongoDB connection FAIL DownloadError: Download failed for url "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-5.0.19.tgz", Details: Status Code is 403 (MongoDB's 404) This means that the requested version-platform combination doesn't exist Try to use different version 'new MongoMemoryServer({ binary: { version: 'X.Y.Z' } })' List of available versions can be found here: https://www.mongodb.com/download-center/community/releases/archive at RedirectableRequest. (/builds/mainstream-eng-cloud/bff-rfq/node_modules/mongodb-memory-server-core/src/util/MongoBinaryDownload.ts:426:17) at RedirectableRequest.emit (node:events:517:28) at RedirectableRequest.emit (node:domain:489:12) at RedirectableRequest.Object..RedirectableRequest._processResponse (/builds/mainstream-eng-cloud/bff-rfq/node_modules/follow-redirects/index.js:368:10) at ClientRequest.RedirectableRequest._onNativeResponse (/builds/mainstream-eng-cloud/bff-rfq/node_modules/follow-redirects/index.js:65:10) at Object.onceWrapper (node:events:632:26) at ClientRequest.emit (node:events:529:35) at ClientRequest.emit (node:domain:489:12) at HTTPParser.parserOnIncomingClient (node:_http_client:700:27) at HTTPParser.parserOnHeadersComplete (node:_http_common:119:17) at TLSSocket.socketOnData (node:_http_client:541:22) at TLSSocket.emit (node:events:517:28) at TLSSocket.emit (node:domain:489:12) at addChunk (node:internal/streams/readable:335:12) at readableAddChunk (node:internal/streams/readable:308:9) at TLSSocket.Readable.push (node:internal/streams/readable:245:10) { url: 'https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-5.0.19.tgz', msg: "Status Code is 403 (MongoDB's 404)\n" + "This means that the requested version-platform combination doesn't exist\n" + "Try to use different version 'new MongoMemoryServer({ binary: { version: 'X.Y.Z' } })'\n" + 'List of available versions can be found here: https://www.mongodb.com/download-center/community/releases/archive' }

Code Example

package.json -> dependeny: "mongodb-memory-server": "8.16.0", "config": { "mongodbMemoryServer": { "debug": "0" } },

jest code ->

const { MongoMemoryServer } = require("mongodb-memory-server");

`describe(" sample API test", () => {
  beforeAll(async () => {
    try {
      got.mockResolvedValue({ body: '{"accessLevel":"success"}' });
      await mongod.start();
      const uri = await mongod.getUri();

      const mongooseOpts = {
        useNewUrlParser: true,
        useUnifiedTopology: true
      };

      if (mongoose.connection.name != "") await mongoose.connection.close();

      await mongoose.connect(uri, mongooseOpts);
});
});

Debug Output

Tried solution for on exisitng forumns but error still persist with same error or different.

https://stackoverflow.com/questions/62226563/error-status-code-is-403-mongodbs-404-this-means-that-the-requested-version

https://github.com/nodkz/mongodb-memory-server/issues/316

https://stackoverflow.com/questions/70907062/starting-the-instance-of-mongodb-memory-server-failed-on-unit-testing-with-jest

Debug Output

```txt Put The Debugout output here ```

Do you know why it happenes?

no, but this is recenlty started in pipeline and locally it's working properly.

Please suggest any possible solutions for this.

hasezoey commented 10 months ago

please provide debug output, see Enable Debug Mode and post the full log

also please provide what environment you are running your pipeline in (like node:12 or node:12-debian). it would also be helpful if you could provide the full output of the commands in Distribution Support.

in addition you could also try to upgrade to mongodb-memory-server 9.0

arpitgoyal43 commented 10 months ago

@hasezoey: Please find the required details. Using node:18.18.2_alpine3.18.4 docker image in pipeline

Got only below logs:

DownloadError: Download failed for url "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-5.0.19.tgz", Details:
    Status Code is 403 (MongoDB's 404)
    This means that the requested version-platform combination doesn't exist
    Try to use different version 'new MongoMemoryServer({ binary: { version: 'X.Y.Z' } })'
    List of available versions can be found here: https://www.mongodb.com/download-center/community/releases/archive
        at RedirectableRequest.<anonymous> (/builds/mainstream-eng-cloud/bff-rfq/node_modules/mongodb-memory-server-core/src/util/MongoBinaryDownload.ts:426:17)
        at RedirectableRequest.emit (node:events:517:28)
        at RedirectableRequest.emit (node:domain:489:12)
        at RedirectableRequest.Object.<anonymous>.RedirectableRequest._processResponse (/builds/mainstream-eng-cloud/bff-rfq/node_modules/follow-redirects/index.js:368:10)
        at ClientRequest.RedirectableRequest._onNativeResponse (/builds/mainstream-eng-cloud/bff-rfq/node_modules/follow-redirects/index.js:65:10)
        at Object.onceWrapper (node:events:632:26)
        at ClientRequest.emit (node:events:529:35)
        at ClientRequest.emit (node:domain:489:12)
        at HTTPParser.parserOnIncomingClient (node:_http_client:700:27)
        at HTTPParser.parserOnHeadersComplete (node:_http_common:119:17)
        at TLSSocket.socketOnData (node:_http_client:541:22)
        at TLSSocket.emit (node:events:517:28)
        at TLSSocket.emit (node:domain:489:12)
        at addChunk (node:internal/streams/readable:335:12)
        at readableAddChunk (node:internal/streams/readable:308:9)
        at TLSSocket.Readable.push (node:internal/streams/readable:245:10) {
      url: 'https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-5.0.19.tgz',
      msg: "Status Code is 403 (MongoDB's 404)\n" +
        "This means that the requested version-platform combination doesn't exist\n" +
        "Try to use different version 'new MongoMemoryServer({ binary: { version: 'X.Y.Z' } })'\n" +
        'List of available versions can be found here: https://www.mongodb.com/download-center/community/releases/archive'
    }
      at MongoMemoryServer.<anonymous> (node_modules/mongodb-memory-server-core/src/MongoMemoryServer.ts:293:17)
      at node_modules/mongodb-memory-server-core/node_modules/tslib/tslib.js:169:75
      at __awaiter (node_modules/mongodb-memory-server-core/node_modules/tslib/tslib.js:165:16)
      at node_modules/mongodb-memory-server-core/src/MongoMemoryServer.ts:284:68

arpitgoyal43 commented 10 months ago

Tried with "mongodb-memory-server": "9.0.X", but getting below error for that. ENOENT: no such file or directory, open 'node:http'

hasezoey commented 10 months ago

Got only below logs:

there should be more logs if debug mode is enabled starting with a line of MongoMS:resolveConfig: Debug Mode Enabled

Tried with "mongodb-memory-server": "9.0.X", but getting below error for that. ENOENT: no such file or directory, open 'node:http'

this should be supported for a long time, could you provide more error backtrace?

Using node:18.18.2_alpine3.18.4 docker image in pipeline

alpine is completely unsupported because mongodb does not have any binaries for that and there are no known working binaries that could be used instead (like on arch you can use the ubuntu binaries), please use ubuntu or debian nodejs images.

arpitgoyal43 commented 10 months ago

Found these log:

There is no offical build of MongoDB for Alpine!
      at MongoBinaryDownloadUrl.getLinuxOSVersionString (node_modules/mongodb-memory-server-core/src/util/MongoBinaryDownloadUrl.ts:239:15)
      at MongoBinaryDownloadUrl.<anonymous> (node_modules/mongodb-memory-server-core/src/util/MongoBinaryDownloadUrl.ts:173:23)
      at node_modules/mongodb-memory-server-core/node_modules/tslib/tslib.js:169:75
      at __awaiter (node_modules/mongodb-memory-server-core/node_modules/tslib/tslib.js:165:16)
      at MongoBinaryDownloadUrl.getArchiveNameLinux (node_modules/mongodb-memory-server-core/lib/util/MongoBinaryDownloadUrl.js:126:38)
      at MongoBinaryDownloadUrl.<anonymous> (node_modules/mongodb-memory-server-core/src/util/MongoBinaryDownloadUrl.ts:94:21)
      at node_modules/mongodb-memory-server-core/node_modules/tslib/tslib.js:169:75
      at __awaiter (node_modules/mongodb-memory-server-core/node_modules/tslib/tslib.js:165:16)
      at MongoBinaryDownloadUrl.getArchiveName (node_modules/mongodb-memory-server-core/lib/util/MongoBinaryDownloadUrl.js:55:38)
      at MongoBinaryDownloadUrl.<anonymous> (node_modules/mongodb-memory-server-core/src/util/MongoBinaryDownloadUrl.ts:56:32)
      at node_modules/mongodb-memory-server-core/node_modules/tslib/tslib.js:169:75
      at __awaiter (node_modules/mongodb-memory-server-core/node_modules/tslib/tslib.js:165:16)
      at MongoBinaryDownloadUrl.getDownloadUrl (node_modules/mongodb-memory-server-core/lib/util/MongoBinaryDownloadUrl.js:30:38)
      at MongoBinaryDownload.<anonymous> (node_modules/mongodb-memory-server-core/src/util/MongoBinaryDownload.ts:170:38)
      at fulfilled (node_modules/mongodb-memory-server-core/node_modules/tslib/tslib.js:166:62)
  console.warn
    Unknown/unsupported linux "alpine(undefined)". Falling back to legacy MongoDB build!
      at MongoBinaryDownloadUrl.getLinuxOSVersionString (node_modules/mongodb-memory-server-core/src/util/MongoBinaryDownloadUrl.ts:270:13)
      at MongoBinaryDownloadUrl.<anonymous> (node_modules/mongodb-memory-server-core/src/util/MongoBinaryDownloadUrl.ts:173:23)
      at node_modules/mongodb-memory-server-core/node_modules/tslib/tslib.js:169:75
      at __awaiter (node_modules/mongodb-memory-server-core/node_modules/tslib/tslib.js:165:16)
      at MongoBinaryDownloadUrl.getArchiveNameLinux (node_modules/mongodb-memory-server-core/lib/util/MongoBinaryDownloadUrl.js:126:38)
      at MongoBinaryDownloadUrl.<anonymous> (node_modules/mongodb-memory-server-core/src/util/MongoBinaryDownloadUrl.ts:94:21)
      at node_modules/mongodb-memory-server-core/node_modules/tslib/tslib.js:169:75
      at __awaiter (node_modules/mongodb-memory-server-core/node_modules/tslib/tslib.js:165:16)
      at MongoBinaryDownloadUrl.getArchiveName (node_modules/mongodb-memory-server-core/lib/util/MongoBinaryDownloadUrl.js:55:38)
      at MongoBinaryDownloadUrl.<anonymous> (node_modules/mongodb-memory-server-core/src/util/MongoBinaryDownloadUrl.ts:56:32)
      at node_modules/mongodb-memory-server-core/node_modules/tslib/tslib.js:169:75
      at __awaiter (node_modules/mongodb-memory-server-core/node_modules/tslib/tslib.js:165:16)
      at MongoBinaryDownloadUrl.getDownloadUrl (node_modules/mongodb-memory-server-core/lib/util/MongoBinaryDownloadUrl.js:30:38)
      at MongoBinaryDownload.<anonymous> (node_modules/mongodb-memory-server-core/src/util/MongoBinaryDownload.ts:170:38)
      at fulfilled (node_modules/mongodb-memory-server-core/node_modules/tslib/tslib.js:166:62)

arpitgoyal43 commented 10 months ago

This was working fine till Oct 31, 2023 on alpine linux and started failing automatically.

Also observed that there is one url received in error which is not accessible and it return 403 status in pipeline.

DownloadError: Download failed for url "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-5.0.19.tgz", Details: Status Code is 403 (MongoDB's 404)

hasezoey commented 10 months ago

i dont know why the alpine build was working before and not now anymore, but to my knowledge there has never been a working alpine build on mongodb's archives.

please use node's debian or ubuntu images.

arpitgoyal43 commented 10 months ago

Issue is resolved now.

YannikG commented 6 months ago

I'm very late to the party but ran into a similar issue today trying to download the database tools in a Docker file. So I want to help anyone searching frustrated the web for a solution or an explanation.

In the end, my curl command was faulty. I forgot to add a space between the url in quotations and the parameter -o. Wrong: curl "http://example.com"-o "myfile.deb" Correct curl "http://example.com" -o "myfile.deb"

hasezoey commented 6 months ago

@YannikG, what has your comment to do with this issue? this project does not make use of curl or have a curl mentioned workaround.

typegoose / mongodb-memory-server