Open olitomlinson opened 5 months ago
Hey @olitomlinson
Thanks for raising this issue. We had plans to raise it ourselves but got a bit wrapped up in work and day-to-day.
We would absolutely welcome this change. We were going to raise under dapr/dapr against actors as the usual state management is already base64 encoded while actor state is raw, formatted JSON. Having it as a configurable flag on the component YAML actually feels like a neater solution than what we had planned. We were hoping for an overload on the actor state management API. With it on the YAML, the team will be able to set it once during project bootstrapping as a best practice and ensure costs don't ever grow wildly beyond their expected range. I like it!
Giving it more thought, it makes sense to be an option for all forms of state management and for any consumption-based database, not just CosmosDB. This same behaviour would apply neatly to DynamoDB as well, for example.
Thanks @olitomlinson for detailed information on the request. I raised the issue under dapr/dapr yesterday. [Proposed] Add compression option when saving state
This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions.
Bump
This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions.
Bump
This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions.
Keep alive
This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions.
Keep alive
Do you envision this being something where the developer can specify one of multiple built-in compression algorithms or did you have one in mind that you think would be a better fit than others?
This customer blog-post highlights a very simple and warranted need to compress state before it hits CosmosDb.
The customer rolled their own client side compression solution for this, but I think the CosmosDb component should include its own turn-key compression implementation to achieve the same goal.
This behavior could be enabled via a flag on the spec
enableValueCompression
Pros
Cons
value
would no longer be adocument
but astring
, meaning thequery API
capability would not be possible.Thinking out loud...
Given that Postgres V2 has dropped support for
query API
due to changing the way it stores thevalue
data- I do not see this as a controversial move to make for CosmosDb. Maybe it makes sense to leave the CosmosDb v1 component alone, and implement this compression feature as the standard behavior for CosmosDb V2?@KrylixZA Maybe you could provide some thoughts on if this would feature/change would be welcome to you and how a turnkey solution provided by the component would simplify your client-side concerns.