findelabs / mongodb-stream-rs

2 stars 2 forks source link

Support GUID format for _id values #2

Open pbtrudel opened 1 year ago

pbtrudel commented 1 year ago

Hi,

I am in the same situation where we want to migrate from a standalone mongodb instance to a Atlas replicaset. I love the tool and it would help out immensely.

The issue I am having with the --continue flag is that it works well when the _id values are in standard mongodb format (ie 63502ddf370418ea1fd94307). In my case, some collections are in this format, but others are in a GUID format (ie 0c6b98a6-8dca-4e26-89f6-813f25152962). The biggest collections to migrate have this GUID format for _id and hence all the data of these collections gets transferred over again, even with the --continue flag. It seems the _id values are not properly read and cannot find the latest doc.

Would it be possible for the tool to support both _id formats?

Thanks for the help.

nutgood commented 1 year ago

For other people that will find this issue, we made this which handles the case above and is just as fast as the rust version: mongosync