Copying contents from one collection into another

[Lsnickels]

I'm not sure if this is the best place to post this, but I'm running into a functionality issue that I'm sure Mongolite is capable of, but I am unsure of how to implement.

I created a stackoverflow question located here.

Essentially, I have multiple databases that all have collections with the same names, and the same structure to them with different data within that structure. My goal is to the contents of all of the databases into one single database after adding some fields that identify which original dataset it came from. I stumbled across a MongoDB function called db.collection.copyTo() but I am uncertain how to use this using the mongolite run() functionality.

Any help would be greatly appreciated!

[Lsnickels]

Any insight on how to go about this @jeroen ? My backup plan is to use a bash script with the sys package to run a system command from within while passing in arguments (the names of the destination and source collections)

[ralmond]

Stumbled across this pretty late, but I think you can use an aggregate pipeline to do this.

[ 
  {"$match": { ...Query to select applicable records...}},
  {"$set":{"source": ...Set  ID...}},
  {"$out":{"db": ...destination DB..., "coll": ...destinationCollection...}}
]

The first line selects the applicable documents, the second line adds a field indicating the source collection and the third line specifies the output file.

On catch, using an $out stage in the pipeline going to a different database is not supported in older versions of Mongo (4.0 doesn't support it). You may need to use the export and import methods to copy between the databases.