search

Azure / azure-cosmos-dotnet-v3

.NET SDK for Azure Cosmos DB for the core SQL API
MIT License
723 stars 477 forks source link

Payload Compression #1204

Open jbt00000 opened 4 years ago

jbt00000 commented 4 years ago

Another issue with performance is the payloads (both request and response) in http mode (I did not sniff to verify TCP/IP mode), is the json is uploaded in plain text.

It should be compressed... and possibly delivered in other forms like bson. Both would cut payload size and therefore improve performance.

Compression would be initiated on demand via options or in automatic mode based on payload size.

jbt00000 commented 4 years ago

I just performed a test...

Turned on gateway mode.

Re-ran in fiddler after adding the header

[Accept-Encoding: gzip, deflate]

My upsert worked, but Cosmos server ignored by compression request.

You may still want to add that header when in Gateway mode hoping for the eventual server side support, but then bang away at the server team to add that feature. It would make a big difference when either performing bulk operations or when people use large docs.

jbt00000 commented 4 years ago

As I just wrote a function that triggers off of changes in Cosmos, I'm seeing this as yet another place where compressing the payload will make a HUGE performance difference when compression is turned on and if documents have any size at all OR if there is a large rate of change.

jbt00000 commented 4 years ago

I'm performing a tiny little export of 10K records from Cosmos. Each of my records is ~300KB. data from Cosmos. So, that's 3.5GB. After it got to my hard drive, I zipped it and it was 340MB. That amounts to a significant bandwidth savings and speedups. Imagine what that would do for your performance metrics...

petro2050 commented 3 years ago

Compression should not happen during HTTP calls only. We should also have the ability to store compressed data. I saw https://feedback.azure.com/forums/263030-azure-cosmos-db/suggestions/19164487-compress-stored-data#{toggle_previous_statuses}. Any ETA on this? @j82w

rf-0 commented 1 year ago

Compression should not happen during HTTP calls only. We should also have the ability to store compressed data. I saw https://feedback.azure.com/forums/263030-azure-cosmos-db/suggestions/19164487-compress-stored-data#{toggle_previous_statuses}. Any ETA on this? @j82w

That link is broken. Also interested though.

spk33 commented 10 months ago

its been three years it would be great if this request is taken into consideration