search

Azure / azure-cosmos-dotnet-v2

Contains samples and utilities relating to the Azure Cosmos DB .NET SDK
MIT License
579 stars 837 forks source link

Track TTL-expired deletes in change feed #402

Open SeanFeldman opened 6 years ago

SeanFeldman commented 6 years ago

Sililar to the idea brough up at UserVoice.

From CosmosDB team:

We plan to introduce support for tracking deletes in the change feed. Using this capability, you can integrate with Azure Functions or another event processing system to perform operations like execute procedure or call other services programmatically.

dishakhattri commented 6 years ago

Hi, any update on this feature ?

PhilG112 commented 5 years ago

Any updates to when this would be available?

ramumangu commented 5 years ago

Any update on this?

bonserem commented 5 years ago

Could really use this issue to remove related GDPR Data from other sources. Might have to move this functionality to outside of CosmosDB otherwise

tomkuijsten commented 5 years ago

It's planned according to uservoice, but that comment was already 18 months ago, @ealsur any news?

ealsur commented 5 years ago

This feature is being worked on. It went back to the design board and it's being worked on again, but no ETA at this moment.

wpitallo commented 5 years ago

Any updates, this will be an awesome feature?

ealsur commented 5 years ago

@mkolt Do we have a rough timeline that can be shared?

@wpitallo Meanwhile, one alternative would be to set a soft delete flag, pickup the change from the Change Feed, then delete the document as part of the Change Feed processing?

wpitallo commented 5 years ago

@ealsur thanks, that is an option, it would be much more elegant though tho have a trigger when the TTL expires. Would rather wait a while if its going to be available soon.

wpitallo commented 4 years ago

@mkolt @ealsur Any updates on this?

stap123 commented 4 years ago

Any updates on this, this would be a great feature and would be very useful for GDPR/data retention purposes?

mkolt commented 4 years ago

@wpitallo @stap123 it's coming, check //build conference 2020 for details on availability dates.

wpitallo commented 4 years ago

@mkolt :) awesome do you have a link to the details? Is it being launched at build 2020?

abaltra commented 4 years ago

@wpitallo Hi! was looking forward to some announcement at build2020 around this and didn't see anything. Did I miss it or was this de-prioritized? In any case, any idea if this is coming any time soon?

tomkuijsten commented 4 years ago

You missed it :) https://azure.microsoft.com/en-us/updates/change-feed-with-full-database-operations-for-azure-cosmos-db/

cwoolum commented 4 years ago

It was delayed to 2H 2020 according to UserVoice

rupeshpatel02 commented 3 years ago

Any update on this feature

dsibilio commented 3 years ago

+1, any updates on this?

sdg002 commented 3 years ago

+1 This will be vital for GDPR compliance. Please, any updates on this?

vikash001 commented 3 years ago

+1 any update on this feature.

MhAllan commented 3 years ago

If you are watching this in 2020 hit like! .. Hello, any updates?

RichMercer commented 3 years ago

Over a year after it was announced it's coming and....nothing.

sdg002 commented 3 years ago

Still waiting. The absence of "deletes" in change feed has left our data synchronization functionality incomplete. We are forced to do a full sync periodically - a costly operation.

After nearly 2 years working with Cosmos, I have begun to introspect if there is any merit in chosing Cosmos over MSSQL. It looks like Microsoft created Cosmos as a knee jerk reaction to NOSQL protagonists.

Some 10 years ago - Cosmos/NOSQL did make sense.

When I look at at MSSQL in 2020 - I am forced to resign to the conclusion that there is very little that Cosmos or any NOSQL database accomplishes that MSSQL (or any leading RDBMS) cannot fulfill.

The NOSQL world is trying to solve problems which have already have robust solutions in the RDBMS world.

brunosantos commented 2 years ago

@sdg002 have you tried scaling up or down in a few minutes with a georeplicated azure sql database instances? How long did that take (and how many transactions did you lose?). Compare that with Cosmos' scalability and the flexibility of it's consistency models. It's not true that we need strong consistency all the time and traditional relational dbs lose on availability for the sake of high consistency with their rigid transact model.

I'm not saying that relational dbs don't have their place. I'm saying that there are use cases for NoSQL and for MSSQL. But you always need to make compromises. It's the CAP theorem. We can't have it all.

brunosantos commented 2 years ago

With regards to the announcement if you look in their repo it looks like deletes are available when enabling the "full Fidelity change feed". This is in preview now so it should be GA in the upcoming months.

https://github.com/Azure/azure-cosmos-dotnet-v3/issues/2067

Mazensayed91 commented 2 years ago

Any updates?

keshav7 commented 2 years ago

Amy update on same?

StMarian commented 1 year ago

@mkolt @ealsur Any updates?

mkolt commented 1 year ago

there is a preview feature "All Versions and Deletes" change feed. This exposes deletes on change feed as well as all intermediate replaces, i.e. no change is lost.