search

mia-platform / integration-connector-agent

Apache License 2.0
2 stars 0 forks source link

[Feature]: Save of raw event data #14

Closed calairoh closed 3 hours ago

calairoh commented 6 days ago

Describe the feature

We would need that the agent saves all the raw event data in a specific collection. This is necessary for us to keep track of all the modifications which happen on the Jira entities to implement audit logs and entity history visualization.

Code of Conduct

davidebianchi commented 6 days ago

So, you need to avoid to overwrite the issue but to track all data as new document, correct?

It is already possible to save it raw: if you remove the processors, the data will be saved as json without any transformation

calairoh commented 6 days ago

Actually, for us it's ok to overwrite the issue because we need also the last snapshot. In addition to this, we require also to keep the entire event trail in order to access the history of the issue.

davidebianchi commented 3 hours ago

It is actually possible with #18 always insert a new element inside a mongo collection. Also, with #16 and #17 it is possible to define multiple pipeline per data source, and filter the data you want to manage per each pipeline