Instagram Source: Respect Message Delete Webhook Events

[steffh]

Is your feature request related to a problem? Please describe. Instagram requires Apps to respect message delete webhooks: https://developers.facebook.com/docs/messenger-platform/instagram/app-review#requirements

Describe the solution you'd like

Once a message delete webhook event is received for a certain message, there should be a new message state deleted (currently: pending, delivered, failed) and the message content should be cleared.

Webhook Event:

   "object":"instagram",
   "entry":[
      {
         "id":"<IGID>",
         "time":1569262486134,
         "messaging":[
            {
               "sender":{
                  "id":"<IGSID>"
               },
               "recipient":{
                  "id":"<IGID>"
               },
               "timestamp":1569262485349,
               "message":{
                  "mid":"<MESSAGE_ID>",
                  "is_deleted":"true"
               }
            }
         ]
      }
   ]
}

Source: https://developers.facebook.com/docs/messenger-platform/instagram/features/webhook

Describe alternatives you've considered Physically deleting the message - but that might be difficult in an append-only infrastructure.

Additional context

[steffh]

This might be related to: https://github.com/airyhq/airy/issues/2132

[chrismatix]

@steffh Would it be ok to actually delete these messages? Given that this is a privacy feature and the HTTP API is just one of many ways of accessing a message this seems a bit more in line with what they want.

[steffh]

@chrismatix I mean of course (see above in the ticket - alternatives), but could we do that in an append-only world?

[chrismatix]

@steffh Yes, we could: We aggregate our streaming data in changelog tables that materialize only one value per key. In addition, Kafka has a feature called Log compaction that enables it to delete old values for a particular key or delete them altogether.

[steffh]

@chrismatix Okay, I mean sure.

You would still sent a message metadata event indicating that the message got deleted I assume?

The API would then just return a result without the message or a 404 respectively I assume?

I am just wondering about the user experience because maybe it would be actually better if we showed that a certain message was there but got deleted (e.g. in a conversation). Otherwise things could be rather confusing for agents?

In case we went for such a solution, we would update the message and set a metadata property to is_deleted: true or sth. like that so the clients could render that information e.g. in the relevant language?

[chrismatix]

You would still sent a message metadata event indicating that the message got deleted I assume?

Yes we would create a websocket/webhook event for that.

The API would then just return a result without the message or a 404 respectively I assume?

The message would be missing from the API and if we ever had a "get message by id" endpoint then that would return 404

I am just wondering about the user experience because maybe it would be actually better if we showed that a certain message was there but got deleted (e.g. in a conversation). Otherwise things could be rather confusing for agents?

In case we went for such a solution, we would update the message and set a metadata property to is_deleted: true or sth. like that so the clients could render that information e.g. in the relevant language?

We can totally go this way and I can see the value. Just wanted to point out that we can physically delete the message. I'm updating the ticket to reflect the deleted state approach, wherein we set the state to deleted and clear the original message content for privacy compliance.