Closed blyork closed 8 years ago
Hmm do you get this issue if you don't use New Relic? This issue means that for some reason bson is undefined in the MongoDB driver, which makes me suspect this issue is due to New Relic's monkey-patching the driver.
Thanks. I will try that. I already downgrade mongoose to 4.2 but that didn't solve the issue.
This is not a New Relic issue:
TypeError: Cannot read property 'calculateObjectSize' of null
at addToOperationsList (C:\sensorFact\npm\node_modules\mongodb\lib\bulk\unordered.js:147:30)
at UnorderedBulkOperation.raw (C:\sensorFact\npm\node_modules\mongodb\lib\bulk\unordered.js:390:7)
at bulkWrite (C:\sensorFact\npm\node_modules\mongodb\lib\collection.js:574:10)
at Collection.insertMany (C:\sensorFact\npm\node_modules\mongodb\lib\collection.js:477:44)
at Collection.insert (C:\sensorFact\npm\node_modules\mongodb\lib\collection.js:753:15)
at NativeCollection.(anonymous function) [as insert] (C:\sensorFact\npm\node_modules\mongoose\lib\drivers\node-mongodb-native\collection.js:136:28)
at model.Model.$__handleSave (C:\sensorFact\npm\node_modules\mongoose\lib\model.js:130:21)
at model.Model.$__save (C:\sensorFact\npm\node_modules\mongoose\lib\model.js:189:9)
at C:\sensorFact\npm\node_modules\mongoose\lib\model.js:288:11
at new Promise.ES6 (C:\sensorFact\npm\node_modules\mongoose\lib\promise.js:45:3)
at model.Model.save (C:\sensorFact\npm\node_modules\mongoose\lib\model.js:287:10)
at model._done (C:\sensorFact\npm\node_modules\hooks-fixed\hooks.js:101:24)
at _next (C:\sensorFact\npm\node_modules\hooks-fixed\hooks.js:64:28)
at fnWrapper (C:\sensorFact\npm\node_modules\hooks-fixed\hooks.js:186:18)
at model.Object.defineProperty.value.fn (C:\sensorFact\npm\node_modules\mongoose\lib\schema.js:251:9)
at _next (C:\sensorFact\npm\node_modules\hooks-fixed\hooks.js:62:30)
at fnWrapper (C:\sensorFact\npm\node_modules\hooks-fixed\hooks.js:186:18)
at model.<anonymous> (C:\sensorFact\npm\node_modules\mongoose\lib\schema.js:230:13)
at next_ (C:\sensorFact\npm\node_modules\hooks-fixed\hooks.js:89:34)
at fnWrapper (C:\sensorFact\npm\node_modules\hooks-fixed\hooks.js:186:18)
at complete (C:\sensorFact\npm\node_modules\mongoose\lib\document.js:1131:5)
at C:\sensorFact\npm\node_modules\mongoose\lib\document.js:1157:20`
Also, this error does not seem to be caught by mongoose's normal error handling. I bound a function to the error event on mongoose.connection and the function never gets called.
Weird. Do you get any errors when you do npm install
?
I didn't receive any error logs.
I forgot to mention this earlier but I am running node version 5.4.0 and npm 3.3.12.
First thing I'd try would be to reinstall mongoose, rm -rf node_modules/mongoose && npm cache clear && npm install
. If that doesn't work, check in node_modules/mongoose/node_modules/mongodb/node_modules/mongodb-core/node_modules/bson/lib/bson/parser/calculate_size.js
and make sure that file exists
Oh wait you're using npm 3. Open up node_modules/bson/lib/bson/parser/calculate_size.js
and make sure its there.
It is there. I have reinstalled this app a couple times since the original error. We still are consistently getting it.
I have gone back as mongoose 4.1 and I am still getting the error:
├─┬ mongoose@4.1.12
│ ├── async@0.9.0
│ ├── bson@0.4.21
│ ├── hooks-fixed@1.1.0
│ ├── kareem@1.0.1
│ ├─┬ mongodb@2.0.46
│ │ ├── es6-promise@2.1.1
│ │ ├─┬ mongodb-core@1.2.19
│ │ │ └─┬ kerberos@0.0.17
│ │ │ └── nan@2.0.9
│ │ └─┬ readable-stream@1.0.31
│ │ ├── core-util-is@1.0.2
│ │ ├── isarray@0.0.1
│ │ └── string_decoder@0.10.31
│ ├── mpath@0.1.1
│ ├── mpromise@0.5.4
│ ├─┬ mquery@1.6.3
│ │ └── bluebird@2.9.26
│ ├── ms@0.1.0
│ ├── muri@1.0.0
│ ├── regexp-clone@0.0.1
│ └── sliced@0.0.5
TypeError: Cannot read property 'calculateObjectSize' of null
at addToOperationsList (C:\sensorfact\npm\sensorfact-pclx-api\node_modules\mongodb\lib\bulk\unordered.js:147:30)
at UnorderedBulkOperation.raw (C:\sensorfact\npm\sensorfact-pclx-api\node_modules\mongodb\lib\bulk\unordered.js:390:7)
at bulkWrite (C:\sensorfact\npm\sensorfact-pclx-api\node_modules\mongodb\lib\collection.js:574:10)
at Collection.insertMany (C:\sensorfact\npm\sensorfact-pclx-api\node_modules\mongodb\lib\collection.js:477:44)
at wrapped [as insertMany] (C:\sensorfact\npm\sensorfact-pclx-api\node_modules\newrelic\lib\instrumentation\mongodb.js:185:25)
at Collection.insert (C:\sensorfact\npm\sensorfact-pclx-api\node_modules\mongodb\lib\collection.js:753:15)
at wrapped (C:\sensorfact\npm\sensorfact-pclx-api\node_modules\newrelic\lib\transaction\tracer\index.js:155:28)
at wrappedQuery (C:\sensorfact\npm\sensorfact-pclx-api\node_modules\newrelic\lib\instrumentation\mongodb.js:218:53)
at wrapped [as insert] (C:\sensorfact\npm\sensorfact-pclx-api\node_modules\newrelic\lib\instrumentation\mongodb.js:188:22)
at NativeCollection.(anonymous function) [as insert] (C:\sensorfact\npm\sensorfact-pclx-api\node_modules\mongoose\lib\drivers\node-mongodb-native\collection.js:142:28)
at model.Model.$__handleSave (C:\sensorfact\npm\sensorfact-pclx-api\node_modules\mongoose\lib\model.js:118:21)
at C:\sensorfact\npm\sensorfact-pclx-api\node_modules\mongoose\lib\model.js:218:11
at new Promise.ES6 (C:\sensorfact\npm\sensorfact-pclx-api\node_modules\mongoose\lib\promise.js:45:3)
at model.Model.save (C:\sensorfact\npm\sensorfact-pclx-api\node_modules\mongoose\lib\model.js:217:10)
at model._done (C:\sensorfact\npm\sensorfact-pclx-api\node_modules\hooks-fixed\hooks.js:101:24)
at _next (C:\sensorfact\npm\sensorfact-pclx-api\node_modules\hooks-fixed\hooks.js:64:28)
at fnWrapper (C:\sensorfact\npm\sensorfact-pclx-api\node_modules\hooks-fixed\hooks.js:186:18)…
This is one of the stranger bugs I've seen, it looks like for some reason the mongodb driver's state is corrupted. Do you have any idea where the code that's triggering this error is?
I am starting to think this is an error the mongodb module.
From what I can tell, this error is happening because the topology object being passed to the Collection constructor inside "lib/collection" of the mongodb module is missing the bson field. The topology object gets passed further and further down until is finally errors out in "lib/bulk/unordered" of the mongodb module.
Do you know where the topology object is passed in?
I posted a issue to the mongodb module's issue board:
Could it be that mongoose is doing something weird when it comes to creating a new Db instance ? I cannot see the passing of the topology chain being broken anywhere so unless something is corrupting the state the only other possibility is the creation of a new Db class instance where topology its not passed in.
@christkv the relevant code is here: https://github.com/Automattic/mongoose/blob/32194211654db7f1005a06590124e695e772bbdf/lib/drivers/node-mongodb-native/connection.js#L46-L63 and here: https://github.com/Automattic/mongoose/blob/32194211654db7f1005a06590124e695e772bbdf/lib/drivers/node-mongodb-native/connection.js#L181-L207 . Anything obvious leap out at you?
// doOpen
this.db.open(function(err) {
if (err) return fn(err);
listen(self);
fn();
});
// doOpenSet
this.db.open(function(err) {
if (err) return fn(err);
fn();
listen(self);
});
Any reason that the listen
block happens before the fn()
in one, but not the other?
I wanted to let you guys know that without updating, reinstalling, or reconfiguring, the issue in our environment has disappeared. The only possible place of change that I can think of is Azure and mongolab. Is it possible that the topology might not be set over networking issues?
it's possible, there could be a misconfigured replset config or some dns resolution issues that could cause weird behavior.
Glad it went away, will mark this as closed.
@vkarpov15 @blyork I am also facing this issue......Any updates on this. My mongoose version is 4.4.4
No updates. Please provide some code that reproduces this issue, otherwise not much I can do about it.
+1
Also getting this same issue.
TypeError: Cannot read property 'calculateObjectSize' of null
at addToOperationsList (/src/node_modules/mongoose/node_modules/mongodb/lib/bulk/unordered.js:148:30)
at UnorderedBulkOperation.raw (/src/node_modules/mongoose/node_modules/mongodb/lib/bulk/unordered.js:391:7)
at bulkWrite (/src/node_modules/mongoose/node_modules/mongodb/lib/collection.js:584:10)
at Collection.insertMany (/src/node_modules/mongoose/node_modules/mongodb/lib/collection.js:483:44)
at wrapped [as insertMany] (/src/node_modules/newrelic/lib/instrumentation/mongodb.js:177:25)
at Collection.insert (/src/node_modules/mongoose/node_modules/mongodb/lib/collection.js:766:15)
at wrapped [as insert] (/src/node_modules/newrelic/lib/instrumentation/mongodb.js:177:25)
at NativeCollection.(anonymous function) [as insert] (/src/node_modules/mongoose/lib/drivers/node-mongodb-native/collection.js:136:28)
at model.Model.$__handleSave (/src/node_modules/mongoose/lib/model.js:130:21)
at model.Model.$__save (/src/node_modules/mongoose/lib/model.js:189:9)
at model.Model.save (/src/node_modules/mongoose/lib/model.js:282:17)
at model._done (/src/node_modules/hooks-fixed/hooks.js:101:24)
at _next (/src/node_modules/hooks-fixed/hooks.js:64:28)
at fnWrapper (/src/node_modules/hooks-fixed/hooks.js:186:18)
at model.Object.defineProperty.value.fn (/src/node_modules/mongoose/lib/schema.js:250:9)
at _next (/src/node_modules/hooks-fixed/hooks.js:62:30)
at fnWrapper (/src/node_modules/hooks-fixed/hooks.js:186:18)
at /src/node_modules/mongoose/lib/schema.js:234:13
at complete (/src/node_modules/mongoose/lib/document.js:1149:5)
at /src/node_modules/mongoose/lib/document.js:1175:20
at ObjectId.SchemaType.doValidate (/src/node_modules/mongoose/lib/schematype.js:654:22)
at /src/node_modules/mongoose/lib/document.js:1171:9
@edwardysun please provide repro instructions
Started getting the issue on azure environment too. Seems to go away for a while when refreshing the mongo replicaset.
@danmo please provide repro instructions or whatever other relevant information you can. The "can't reproduce" label is there for a reason.
Well, I got it now too.
Stack trace:
160329/174416.675, [error], message: Uncaught error: Cannot read property 'calculateObjectSize' of null stack: TypeError: Cannot read property 'calculateObjectSize' of null
at addToOperationsList (/mnt/app/node_modules/mongoose/node_modules/mongodb/lib/bulk/unordered.js:147:30)
at UnorderedBulkOperation.raw (/mnt/app/node_modules/mongoose/node_modules/mongodb/lib/bulk/unordered.js:390:7)
at bulkWrite (/mnt/app/node_modules/mongoose/node_modules/mongodb/lib/collection.js:574:10)
at Collection.insertMany (/mnt/app/node_modules/mongoose/node_modules/mongodb/lib/collection.js:477:44)
at wrapped as insertMany
at Collection.insert (/mnt/app/node_modules/mongoose/node_modules/mongodb/lib/collection.js:753:15)
at wrapped (/mnt/app/node_modules/newrelic/lib/transaction/tracer/index.js:155:28)
at wrappedQuery (/mnt/app/node_modules/newrelic/lib/instrumentation/mongodb.js:218:53)
at wrapped as insert
at NativeCollection.(anonymous function) as insert
at model.Model.$handleSave (/mnt/app/node_modules/mongoose/lib/model.js:130:21)
at model.Model.$save (/mnt/app/node_modules/mongoose/lib/model.js:189:9)
at /mnt/app/node_modules/mongoose/lib/model.js:288:11
at new Promise.ES6 (/mnt/app/node_modules/mongoose/lib/promise.js:45:3)
at model.Model.save (/mnt/app/node_modules/mongoose/lib/model.js:287:10)
at model._done (/mnt/app/node_modules/mongoose/node_modules/hooks-fixed/hooks.js:101:24)
at _next (/mnt/app/node_modules/mongoose/node_modules/hooks-fixed/hooks.js:64:28)
at fnWrapper (/mnt/app/node_modules/mongoose/node_modules/hooks-fixed/hooks.js:186:18)
at model.
Package.json
"async": "1.5.0",
"bassmaster": "1.8.0",
"boom": "3.0.0",
"catbox": "7.0.0",
"catbox-memory": "2.0.1",
"good": "6.4.0",
"good-console": "5.2.0",
"good-loggly": "2.0.0",
"h2o2" : "4.0.2",
"hapi": "11.1.2",
"hapi-auth-anonymous": "3.0.1",
"hapi-auth-bearer-mw": "2.0.1",
"hapi-fields": "0.2.0",
"hapi-mandrill": "2.1.1",
"hapi-mongoose-db-connector": "2.1.3",
"hapi-oauth-store-multi-tenant": "2.1.1",
"hapi-routes-accounts": "2.1.2",
"hapi-routes-authorization-and-session-management": "3.0.2",
"hapi-routes-oauth-management": "3.1.2",
"hapi-routes-roles": "2.1.2",
"hapi-routes-status": "2.1.1",
"hapi-routes-tenants-setup": "2.0.1",
"hapi-routes-users": "2.2.2",
"hapi-store-accounts": "2.0.1",
"hapi-user-store-multi-tenant": "2.2.1",
"hapi-swaggered": "2.4.1",
"hoek": "3.0.4",
"inert" : "3.2.0",
"joi": "7.0.1",
"lout": "7.2.0",
"mongoose": "4.2.5",
"newrelic": "1.24.0",
"pkginfo": "0.3.1",
"underscore": "1.8.3",
"vision" : "4.0.1",
"kerberos" : "0.0.17"
Hosted on modulus.io Database on compose.io: 2.6.4
@vkarpov15 this should go away in the latest mongodb as it's now using the BSON parser directly from mongodb-core instead of the passed down one, meaning it should never be able to be null again.
Yes, we got the error in two independent applications which were using mongoose 4.4.8. In both cases the problem was solved by upgrading to 4.4.10.
Good thing I upgraded mongoose to latest mongodb yesterday
Having this problem too with mongoose 4.4.8 on Amazon AWS. See below:
I shall try upgrading to mongoose 4.4.10 and will report any status update.
It is not mongoose issue. It is mongodb driver I believe. I am seeing this error like once in a month. In order to reproduce the error, the code(bulk update & upsert) need to be run for a few weeks. I also update mongo jira. https://jira.mongodb.org/browse/NODE-639
│ └─┬ mongodb@2.0.55 │ ├─┬ mongodb-core@1.2.31
I recently started receiving this error:
And a while later after restarting:
Here are my mongoose dependencies.
Any ideas?