search

globalsign / mgo

The MongoDB driver for Go
Other
1.97k stars 232 forks source link

Request: BSON stabilize ordering of map keys (instead of GoLang's random ordering) #381

Open bgehman opened 4 years ago

bgehman commented 4 years ago

I'd like to leverage the Upsert() capabilities of Mongo, and know when a document has changed on the Upsert when using GoLang maps. Specifically I want ChangeInfo.Updated to be 0 when the document has not changed.

Background When it comes to JSON & BSON maps, there are differences in their respective specs:

Coupled with the fact that GoLang intentionally randomizes the order when walking over a map (ie: range <map>) means that 99% of my Upserts are coming back indicating that the document changed (when it really didn't). The only thing that changed was the randomized ordering of the keys by the lower level GoLang range function when walking maps.

This is the LOC that is the culprit (range'ing over the map): https://github.com/globalsign/mgo/blob/113d3961e7311526535a1ef7042196563d442761/bson/encode.go#L205

If the addMap() function would fetch the map's keys, sort them, and then addElem() in that sorted order I'd be "set". The current code is taking GoLang's map random ordering (more akin to JSON), and encoding it as a strict ordering in BSON.

I've also tried using bson.D with no luck. Whenever Upsert()'ing with bson.D the order is preserved (not random in Mongo) -- however ChangeInfo.Updated is never 0 on identical / repeated calls.

What works:

What doesn't work:

I'm not able to model the docs via GoLang struct as this middleware component is agnostic to the model of the docs (more akin to a generic JSON processor), but it does need to know if/when the document has been modified on the Upsert().

What version of MongoDB are you using (mongod --version)?

Doesn't matter

What version of Go are you using (go version)?

Doesn't matter (greater than 1.0)

What operating system and processor architecture are you using (go env)?

Doesn't matter

What did you do?

Upsert a bson.M document (ie: map[string]interface{}) Repeated calls of the same map result in ChangeInfo.Updated not equal to 0, depending on the randomized order of golangs range <map> and BSON's internal strict ordering.

Can you reproduce the issue on the latest development branch?

Haven't tried, but a code review show the same range <map> problem (ie: randomized ordering of map keys).

bgehman commented 4 years ago

Actually, struct is broken too. In all cases, an Upsert (via any method) results in ChangeInfo.Updated not equal to 0 even though the document is not changed.

Since this driver is abandon-ware, we are moving to mongo-go-driver. That other driver does still have the range <map> problem, but at least there are ways to detect if documents have changed on upsert with it. L8r.