search

typegoose / mongodb-memory-server

Manage & spin up mongodb server binaries with zero(or slight) configuration for tests.
https://typegoose.github.io/mongodb-memory-server/
MIT License
2.56k stars 185 forks source link

Postinstall Failure `linux/mongodb-linux-x86_64-debian81-latest.tgz` #131

Closed Moggers closed 4 years ago

Moggers commented 5 years ago

Not sure if this is upstream or mongodb themselves, but the above resolved binary is throwing:

<Error>
<Code>AccessDenied</Code>
<Message>Access Denied</Message>
<RequestId>3BB9EE1261D433D5</RequestId>
<HostId>
z8fk3EzYBQAo83J9pdvWOmxLqiROkYc116+r8jpAOWlAQ7yBYz1jlFPlxHt+1OESUeB2cMy1AAU=
</HostId>
</Error>

This happens when not specifying a version - as a workaround specifying 3.4.19-rc1 seems to work.

nkhlmn commented 5 years ago

My travis builds recently started failing as well. Related?

> mongodb-memory-server@2.9.3 postinstall /home/travis/build/nikhilkamineni/codex/node_modules/mongodb-memory-server
> node ./postinstall.js
mongodb-memory-server: checking MongoDB binaries cache...
failed to download/install MongoDB binaries. The error:
Error: MongoBinaryDownload: md5 check is failed
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! mongodb-memory-server@2.9.3 postinstall: `node ./postinstall.js`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the mongodb-memory-server@2.9.3 postinstall script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
dankelleher commented 5 years ago

@nikhilkamineni yes it's the same issue - the download server is reporting access denied when downloading the binary or looking up the md5.

PacoDu commented 5 years ago

Same error here when using npm install --save-dev mongodb-memory-server

node: v11.6.0 npm: 6.5.0

$ npm install mongodb-memory-server --save-dev

> mongodb-memory-server@2.9.3 postinstall /home/*******/node_modules/mongodb-memory-server
> node ./postinstall.js

mongodb-memory-server: checking MongoDB binaries cache...
failed to download/install MongoDB binaries. The error:
Error: MongoBinaryDownload: md5 check is failed
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! mongodb-memory-server@2.9.3 postinstall: `node ./postinstall.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the mongodb-memory-server@2.9.3 postinstall script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
rocknrolla777 commented 5 years ago

workaround: you can use "MONGOMS_DISABLE_POSTINSTALL=1" variable.

MONGOMS_DISABLE_POSTINSTALL=1 npm install

hpgmiskin commented 5 years ago

I am having the same issue. I am able to get past the post install script using MONGOMS_DISABLE_POSTINSTALL=true yarn install which @rocknrolla777 mentions. However when running tests the binary attempts to download again and fails with the same md5 check is failed.

The output to getos() command is { os: 'linux', dist: 'Debian', release: '8.11' }

hpgmiskin commented 5 years ago

I have managed to install with the following command (thanks to @Moggers for suggesting to specifically define the version of mongodb)

MONGOMS_VERSION=4.0.0 yarn add mongodb-memory-server --dev
rocknrolla777 commented 5 years ago

@hpgmiskin Is MongoDB version specified in tests? 

  binary: {
    version: '4.0.5',
  },
hpgmiskin commented 5 years ago

@rocknrolla777 I have not got the binary version specified in tests. Thank for pointing out 👍

chemitaxis commented 5 years ago

Hi @hpgmiskin and @rocknrolla777, Do I need to be the same version in MONGOMS_VERSION and the binary version option? Thanks.

hpgmiskin commented 5 years ago

Hi @chemitaxis you can either install with the MONGOMS_DISABLE_POSTINSTALL=true flag and set the mongo version in the test script using binary version. Alternatively you can install and run with the MONGOMS_VERSION=4.0.3. I would recommend what @rocknrolla777 has done which is disable post install and then set the binary version in your test script.

rocknrolla777 commented 5 years ago

Hi @chemitaxis, I absolutely agree with @hpgmiskin. Just disable post install and set binary version. If MongoDB version is not declared in your test script and you install node modules with a specific version (not latest), then your test script installs the latest version anyway. It means binaries will be downloaded twice.

chemitaxis commented 5 years ago

Ok, thanks ;)

nodkz commented 5 years ago

It looks like that some problems happen in MongoDB distro with binaries and md5 files.

So bumping a new MAJOR version with breaking changes:

nodkz commented 5 years ago

Current problem with latest packages for UBUNTU.

For example: http://downloads.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1804-latest.tgz https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1404-latest.tgz http://downloads.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1404-latest.tgz http://downloads.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1404-v4.0-latest.tgz

screen shot 2019-01-16 at 10 01 18 pm

Somebody can ping MongoDB team?

rocknrolla777 commented 5 years ago

The same for DEBIAN - http://downloads.mongodb.org/linux/mongodb-linux-x86_64-debian81-latest.tgz

nodkz commented 5 years ago

Maybe affected other OS and versions.

See all version binaries, some links are broken: https://www.mongodb.org/dl/linux/x86_64-ubuntu1404 https://www.mongodb.org/dl/linux/x86_64-ubuntu1804 https://www.mongodb.org/dl/osx/x86_64

For now as workaround you need to find working binary manually. Eg.

MONGOMS_VERSION=4.0.0 yarn add mongodb-memory-server --dev

And use this version in your tests:

new MongoMemoryServer({
  binary: {
    version: '4.0.0', // or you use process.env.MONGOMS_VERSION = '4.0.0'
  },
});

Or use process.env.MONGOMS_VERSION = '4.0.0' instead of binary.version.

Or use dotenv

nodkz commented 5 years ago

TO ALL Please Retweet and Like this tweet https://twitter.com/nodkz/status/1085575353442144258 MongoDB's Jira is too slow for fixing this issue and should be used other channels.

chemitaxis commented 5 years ago

Done ✅

Sent from my iPhone

On 16 Jan 2019, at 17:33, Pavel Chertorogov notifications@github.com wrote:

TO ALL Please Retweet and Like this tweet https://twitter.com/nodkz/status/1085575353442144258 MongoDB's Jira is too slow for fixing this issue and should be used other channels.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

nodkz commented 5 years ago

Currently fixing this issue via

refactor: add LATEST_VERSION constant in code for global changing latest version on some value. Change latest version on 4.0.3 😂 cause latest version on download return 403 http-status code. But 4.0.3 version downloads normally 🙃

Due to broken latest packages I cannot run all tests on CI and publish a new version. Damn it!

nodkz commented 5 years ago

Just published new MAJOR 3.0.0. version for fixing this issue.

screen shot 2019-01-16 at 10 46 11 pm

Please check it and confirm that all works as expected in your codebases. Except that you are working on 4.0.3 version instead of latest.

rocknrolla777 commented 5 years ago

@nodkz thanks for quick issue resolving! 👍

nodkz commented 5 years ago

TO ALL Please Retweet and Like this tweet https://twitter.com/nodkz/status/1085575353442144258

MongoDB's Jira is too slow for fixing this issue and should be used other channels.

mhassan1 commented 5 years ago

I passed this tweet to some friends at MongoDB. Let's see what they say.

rvillane commented 5 years ago

I just tried v3.0.0 but I'm getting the following error, any ideas?

20137 verbose stack Error: mongodb-memory-server@3.0.0 postinstall: `node ./postinstall.js`
20137 verbose stack Exit status 1
20137 verbose stack     at EventEmitter.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:301:16)
20137 verbose stack     at EventEmitter.emit (events.js:159:13)
20137 verbose stack     at ChildProcess.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
20137 verbose stack     at ChildProcess.emit (events.js:159:13)
20137 verbose stack     at maybeClose (internal/child_process.js:943:16)
20137 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:220:5)
20138 verbose pkgid mongodb-memory-server@3.0.0
20139 verbose cwd /Users/spdhw/git/direct-order-transformer
20140 verbose Darwin 17.7.0
20141 verbose argv "/usr/local/bin/node" "/usr/local/bin/npm" "install"
20142 verbose node v9.3.0
20143 verbose npm  v6.5.0
20144 error code ELIFECYCLE
20145 error errno 1
20146 error mongodb-memory-server@3.0.0 postinstall: `node ./postinstall.js`
20146 error Exit status 1
20147 error Failed at the mongodb-memory-server@3.0.0 postinstall script.
20147 error This is probably not a problem with npm. There is likely additional logging output above.
20148 verbose exit [ 1, true ]
rvillane commented 5 years ago

updating to node 10 fixes it

mattlord commented 5 years ago

Thank you for reporting the issue to us! We've been looking into this today and now have a fix in place. The next -latest build links should be working correctly and in the meantime we're trying to manually correct the ones already in place.

I also want to note that the -latest builds are ~ nightly builds from a given GitHub branch -- where no version means master. So linux/mongodb-linux-x86_64-debian81-latest.tgz would in fact be a ~ nightly build from the master branch, which is the 4.1 development branch today (not GA). It seems that perhaps you were assuming that -latest was a static link to the latest GA release (4.0.5 as of today)? It's reasonable given the build name, but -latest is just a build from the branch at the time -- it's not an official release. I'll look into renaming these to something more clear, e.g. "-nightly" (even though they're generated on events vs a schedule) or instead leaving those as-is and creating and maintaining new static links to get the latest GA release and/or the latest GA release for a specific major version, so for example as of right now:

linux/mongodb-linux-x86_64-debian81-stable.tgz   --> linux/mongodb-linux-x86_64-debian81-4.0.5.tgz
linux/mongodb-linux-x86_64-debian81-36stable.tgz --> linux/mongodb-linux-x86_64-debian81-3.6.9.tgz

That way you have a static link that can be used to get the latest official GA release, which is I think what you really wanted here.

Thanks again! Please let me know if you have any followup questions or issues.

nodkz commented 5 years ago

@mattlord thanks for fast response and clarification about latest builds. I thought that they work like latest tag in docker naming or like in cdn js lib delivery.

Anyway I will be glad to any naming for GA latest packages. Hope that -36stable.tgz is typo and you will use existed mask for package naming like it did for current latest names - -v3.6-latest. Please let me know when it will be implemented. Tnx.

And yep I also have the old issue abou similar binaries for Alpine linux - https://jira.mongodb.org/browse/SERVER-36790 Where can be download executable binary for Alpine, like it did eg. for Ubuntu. Of course if it's possible

For now this package works for most OS and CI with zero configuration and no need in preinstalled mongod on the system. Also it may spin up in parallel different versions and greatly works for parallel tests (when simultaneously worked several test processes with own mongod in-memory instance).

Thanks.

PS. In Twitter we trust! 😉

nodkz commented 5 years ago

@zhongjixiuxing can you open a new issue and provide small repro repo. I'll try to resolve it till the end of week. Tnx.

zhongjixiuxing commented 5 years ago

@zhongjixiuxing can you open a new issue and provide small repro repo. I'll try to resolve it till the end of week. Tnx.

@nodkz I'm so sorry, it's usage problem, I didn't look at the new document. I remove this my comment, thanks for your reply.

hasezoey commented 4 years ago

@nodkz could this be closed because it got fixed?

hasezoey commented 4 years ago

i just found export const LATEST_VERSION: string = '4.0.3'; and noticed the provided links by @nodkz are not denied anymore - should we revert to latest again?

nodkz commented 4 years ago

@hasezoey thanks for pointing 🙏 this fixed in https://github.com/nodkz/mongodb-memory-server/commit/5bfc9c004177f3142f6e809aef8e5b06e7e379fb

nodkz commented 4 years ago

@hasezoey reverted back due to CI build problems https://circleci.com/gh/nodkz/mongodb-memory-server/462 Have no time to figure this out. (PS: Shame to me to commit directly to the master 😬Rush is a bad thing. I bring trash to commit history. 😳)

hasezoey commented 4 years ago

@nodkz i just tested locally, and get a lot of STDOUT: {"t":{"$date":"2020-02-28T18:24:38.796+0100"},"s":"I", "c":"REPL_HB", "id":0,"ctx":"ReplCoord-0","msg":"{}","attr":{"message":"Heartbeat to 127.0.0.1:38639 failed after 2 retries, response status: HostUnreachable: Error connecting to 127.0.0.1:38639 :: caused by :: Connection refused"}}

i have looked into this more, and it seems that it is because of

  MongoMS:MongoInstance Mongo[35459]: STDOUT: {"t":{"$date":"2020-02-28T19:17:34.544+0100"},"s":"F", "c":"-",       "id":0,"ctx":"ReplCoord-0","msg":"{}","attr":{"message":"Invalid access at address: 0x20"}}
  MongoMS:MongoInstance  +0ms
  MongoMS:MongoInstance Mongo[35459]: STDOUT: {"t":{"$date":"2020-02-28T19:17:34.544+0100"},"s":"F", "c":"-",       "id":0,"ctx":"ReplCoord-0","msg":"{}","attr":{"message":"Got signal: 11 (Segmentation fault)."}}

but couldnt find a fix or bug report for this, and i dont know how mongodb's jira works PS: latest downloads an dev version nowdays ... (on my system v4.3.3-635-gabcbc1e)