appsmithorg / appsmith

Platform to build admin panels, internal tools, and dashboards. Integrates with 25+ databases and any API.
https://www.appsmith.com
Apache License 2.0
34.7k stars 3.75k forks source link

[Feature]: Mongo Change Streams #25707

Open felix-appsmith opened 1 year ago

felix-appsmith commented 1 year ago

Is there an existing issue for this?

Summary

Appsmith currently offers a wide range of MongoDB functions, but we've identified a crucial feature that is not yet available for easy implementation: MongoDB Change Streams. This powerful functionality is essential for creating dynamic dashboards that automatically display real-time updates from MongoDB, making it a valuable tool for building responsive data-driven applications.

DOCS: https://www.mongodb.com/docs/manual/changeStreams/

Why should this be worked on?

This feature will empower our users to effortlessly implement real-time updating dashboards, utilizing the full capabilities of Change Streams in a simple and efficient manner. As a result, they will be able to explore and develop multiple use cases with ease and speed.

felix-appsmith commented 1 year ago

Workaround:

What it does is validate the state of the data, and when it finds changes, it updates the table. If no changes are found, it maintains the current state. This speeds up the function. However, it's important to consider that this workaround may have limitations and performance issues with massive data, as I tested it with small data segments where the execution time was 280 milliseconds.


export default {

MongoStreamActive () {

async function GetStreamMongo(){
try{
const t0 = performance.now();
let validator = false
let currentData
let newData
await mongoData.run()
if(!validator){
validator = true
currentData = mongoData.data
storeValue('MongoData', currentData)
}

if(validator){
newData =  mongoData.data
}
if(newData == currentData){
console.log(`No updates`)
}else{
storeValue('MongoData', newData)
currentData =  newData
}
const t1 = performance.now();
console.log(`Call to doSomething took ${t1 - t0} milliseconds.`);
}catch(error){
console.error(`Error while fetching data in StreamMongo`)
}
}
setInterval(GetStreamMongo, 500)

}
}

https://github.com/appsmithorg/appsmith/assets/114161539/fa56a28c-33e2-45d2-a62e-cded3a5cc133