search

facebookarchive / flashback

Capture and replay real mongodb workloads
Other
217 stars 72 forks source link

Fix $hint and $orderby for the query recording #32

Closed wojcikstefan closed 8 years ago

wojcikstefan commented 9 years ago

Fixes #31 (you can see the description of the problem under that issue)

wojcikstefan commented 9 years ago

Hi there @tredman! Are you by any chance the maintainer of this package? I still need to update/add some test cases in this PR, but wanted to get your feedback on it before I move forward.

wojcikstefan commented 9 years ago

@tmc I see you merged a PR in the latest commit to this repo. Are you still involved in its development? :)

tredman commented 8 years ago

Sorry for the delay on this!

This looks like it will work although it is a highly specific solution for just $hint and $orderBy and there are other cases where ordering is important and the tools break. This actually bit us at Parse when we were trying to use flashback with $centerSphere queries, where the order of fields matters, and we had to do some manual hacking of the output file to correct the order.

I think the lesson here is that JSON is a bad intermediate format for storage of ops between "record" and "replay" stages. To my knowledge there's no good way to marshall JSON in go and preserve order. We should write documents as raw BSON from the python tools and read them in as ordered BSON in go to avoid this.

Can't merge as there are conflicts, can you rebase and resolve? Thanks, Travis

tredman commented 8 years ago

Closing in favor of https://github.com/ParsePlatform/flashback/pull/35 which is a more thorough fix.

wojcikstefan commented 8 years ago

Thanks! Your solution looks much better!