Trigger + aggregation + $merge not working

[coodoo]

Following code runs fine with nodejs and aggregated results will be inserted into zz collection, but the same code running via trigger (atlas + stitch) always failed to insert results into zz, wondering is this expected or a bug?

db.getCollection('bakesales')
  .aggregate( [
    { $match: { date: { $gte: new Date('1970-01-01') } } },
    { $group: { _id: { $dateToString: { format: "%Y-%m", date: "$date" } }, sales_quantity: { $sum: "$quantity"}, sales_amount: { $sum: "$amount" } } },
    { $merge: { into: {db: 'test', coll:"zz"}, whenMatched: "replace" } }
  ] )

[tkaye407]

Hi @coodoo. I do not believe that stitch supports $merge due to the complications that arise in relation to rules. That being said, we have this page that details some of the limitations of functions in stitch (https://docs.mongodb.com/stitch/mongodb/mongodb-service-limitations/#aggregation-stages). Unfortunately, $merge is not listed as an unsupported operation, so I have filed a ticket with our docs team to reflect this. I would first suggest looking at what "user" is running the functions. A lot of the limitations are lifted when running as a "system user". Additionally, we use canny (https://mongodb.canny.io/mongodb-stitch) to track requests, so feel free to add support for $merge if you would like

[tkaye407]

I was just informed by one of the stitch core engineers that $merge should work when the function is run as a system user, please let me know here though if you do not find that to be the case

[coodoo]

Hey @tkaye407, thanks for getting back to me, I've tried the same code again with following settings, hopefully that's the correct settings to make the function runs as a system user:

But unfortunately that still didn't work, is there other place to correctly configure it?

[SriSivaC]

Hello,

I have an application, which can benefit from the working of $merge pipeline with Triggers and Stitch. Kindly update whether system user can implement $merge in functions for Triggers.

Thanks in advance

[gabriprat]

Hi, merge is not working for me as well for scheduled triggers. I tried running as application and server. What am I missing? Thank you for your help!

[drewdipalma]

Hi Folks – $merge should work for system users but not when using Application permissions. Can you verify that you are using a MongoDB 4.2+ cluster (the version where $merge was introduced)? If this is the case and you are still experiencing issues would you mind sending me a link to your application so I can look more closely into the issue – drew.dipalma@mongodb.com.

[gabriprat]

Hi @drewdipalma, I'm using a 4.2.8 cluster. In fact, when I manually executed the function it worked without any problems. But, when I let the scheduled trigger run it, the data was not merged even though the logs showed no errors. I'm sending you the link to my app via email. Thank you for your help!

[drewdipalma]

Using the following function with a scheduled Trigger (leveraging the Atlas sample data set) I was able to continually add new collections. I am working with folks who reached out directly to see if further investigation is needed.

exports = async function(arg){
  var ts = new Date();
  var collection = context.services.get("mongodb-atlas").db("sample_mflix").collection("users");
  var doc = await collection.aggregate([
    {
      "$match": { "name":"Ned Stark"}
    },
    {
      "$merge": {"into": "foo"+ts.getTime()}
    }]).toArray();
  return JSON.stringify(doc);
};

[Xfox1]

Using the following function with a scheduled Trigger (leveraging the Atlas sample data set) I was able to continually add new collections. I am working with folks who reached out directly to see if further investigation is needed.

exports = async function(arg){
  var ts = new Date();
  var collection = context.services.get("mongodb-atlas").db("sample_mflix").collection("users");
  var doc = await collection.aggregate([
    {
      "$match": { "name":"Ned Stark"}
    },
    {
      "$merge": {"into": "foo"+ts.getTime()}
    }]).toArray();
  return JSON.stringify(doc);
};

I used something like it. I just noticed something. If I don't put the .toArray()at the end of the aggregate it seems like it's not working. How come?

[tkaye407]

Aggregate() and find() return cursors (https://www.mongodb.com/docs/manual/reference/method/js-cursor/) so if you dont add toArray() then you just have a cursor which is not what you want.