search

Azure / AppConfiguration

Questions, feedback and samples for Azure App Configuration service
MIT License
227 stars 68 forks source link

Error 413 (Request Entity Too Large) when using ConfigurationClient #261

Closed lpunderscore closed 4 years ago

lpunderscore commented 4 years ago

So far I have been updating configuration values directly through API calls. I wanted to start using the new ConfigurationClient but the configuration value that can be set through an API call is throwing an exception when using the ConfigurationClient.SetConfigurationSetting() method.

This is a json configuration of about 8kb.

Status: 413 (Request Entity Too Large)

Content:
<html>
<head><title>413 Request Entity Too Large</title></head>
<body>
<center><h1>413 Request Entity Too Large</h1></center>
<hr><center>openresty/1.15.8.1</center>
</body>
</html>
zhenlan commented 4 years ago

@lpunderscore you probably hit the limit described in https://github.com/Azure/AppConfiguration/issues/174.

lpunderscore commented 4 years ago

Out of curiosity, that limit is not there when using the rest API directly or through the portal? Why is this only a limit when using the ConfigurationClient? the exact same json value can be set in the portal without any issue.

zhenlan commented 4 years ago

@lpunderscore when you use ConfigurationClient, do you use connection string or AAD authentication? Do you have a request-id or client-request-id of the failed request that you can share?

AlexanderSher commented 4 years ago

@lpunderscore , can you tell us the exact size of that json object?

lpunderscore commented 4 years ago

@AlexanderSher The value size is 7044 bytes.

lpunderscore commented 4 years ago

@zhenlan I use a connection string.

Not sure how to get a request-id or client-request-id since an actual exception is thrown and I never get a response object.

AlexanderSher commented 4 years ago

I've created a random json of size 7233 bytes:

[ { "_id": "5e3c5f5e9936ed3761076f1c", "index": 0, "guid": "7338eb7d-e967-4756-ae0c-727d6d04118c", "isActive": true, "balance": "$2,323.46", "picture": "http://placehold.it/32x32", "age": 34, "eyeColor": "blue", "name": "Duran Molina", "gender": "male", "company": "ACCRUEX", "email": "duranmolina@accruex.com", "phone": "+1 (555) 575-3234", "address": "390 Madeline Court, Sidman, Nebraska, 2723", "about": "Sunt sit Lorem consequat consequat incididunt. Dolore minim sint non incididunt. Sunt sit Lorem consequat consequat incididunt. Dolore minim sint non incididunt. Sunt sit Lorem consequat consequat incididunt. Dolore minim sint non incididunt. Sunt sit Lorem consequat consequat incididunt. Dolore minim sint non incididunt. Laboris eiusmod occaecat officia qui deserunt consequat officia dolore. Est labore culpa exercitation reprehenderit quis nostrud anim minim. Ipsum consequat aliqua labore dolore sint laborum enim velit. Dolore minim sint non incididunt. Sunt Lorem consequat consequat incididunt. Dolore minim sint non incididunt. Sunt sit Lorem consequat consequat incididunt. Dolore minim sint non incididunt. Sunt sit Lorem consequat consequat incididunt. Dolore minim sint non incididunt. Laboris eiusmod occaecat officia qui deserunt consequat officia dolore. Est labore culpa exercitation reprehenderit quis nostrud anim minim. Ipsum consequat aliqua labore dolore sint laborum enim velit. Cillum quis ut ut non exercitation esse cillum eiusmod dolor adipisicing excepteur excepteur. Lorem minim esse sit amet enim qui veniam.\r\nSunt sit Lorem consequat consequat incididunt. Dolore minim sint non incididunt. Sunt sit Lorem consequat consequat incididunt. Dolore minim sint non incididunt. Sunt sit Lorem consequat consequat incididunt. Dolore minim sint non incididunt. Sunt sit Lorem consequat consequat incididunt. Dolore minim sint non incididunt. Laboris eiusmod occaecat officia qui deserunt consequat officia dolore. Est labore culpa exercitation reprehenderit quis nostrud anim minim. Ipsum consequat aliqua labore dolore sint laborum enim velit. Cillum quis ut ut non exercitation esse cillum eiusmod dolor adipisicing excepteur excepteur. Lorem minim esse sit amet enim qui veniam.\r\n", "registered": "2018-10-06T04:01:55 +05:00", "latitude": 48.754199, "longitude": 143.314589, "tags": [ "anim", "consectetur", "anim", "ullamco", "consequat", "consectetur", "voluptate" ], "friends": [ { "id": 0, "name": "Mullins Sanders" }, { "id": 1, "name": "Bryant Alston" }, { "id": 2, "name": "Harding Francis" } ], "greeting": "Hello, Duran Molina! You have 4 unread messages.", "favoriteFruit": "strawberry" }, { "_id": "5e3c5f5ec4c45b2fbc210a58", "index": 1, "guid": "6cecc064-1e58-4ec5-8a45-3da153a4fe89", "isActive": true, "balance": "$3,978.11", "picture": "http://placehold.it/32x32", "age": 33, "eyeColor": "brown", "name": "Jody Reed", "gender": "female", "company": "KONGENE", "email": "jodyreed@kongene.com", "phone": "+1 (555) 416-2416", "address": "683 Quentin Road, Katonah, Hawaii, 5549", "about": "Enim in aute fugiat ipsum ex Lorem nisi laboris. Sit minim duis culpa do cupidatat. Excepteur mollit sit consequat eiusmod sint anim labore exercitation nisi incididunt consectetur occaecat occaecat veniam.\r\n", "registered": "2015-09-30T02:08:47 +05:00", "latitude": -73.941771, "longitude": 132.977499, "tags": [ "irure", "do", "aliqua", "eiusmod", "ullamco", "incididunt", "nulla" ], "friends": [ { "id": 0, "name": "Bettie Durham" }, { "id": 1, "name": "Lily Roach" }, { "id": 2, "name": "Francis Benson" } ], "greeting": "Hello, Jody Reed! You have 10 unread messages.", "favoriteFruit": "apple" }, { "_id": "5e3c5f5ede12af0f03f5d71d", "index": 2, "guid": "1e532edb-2c5d-4850-9db1-4c38f84da720", "isActive": true, "balance": "$1,178.32", "picture": "http://placehold.it/32x32", "age": 27, "eyeColor": "blue", "name": "Fletcher Maldonado", "gender": "male", "company": "ZENSOR", "email": "fletchermaldonado@zensor.com", "phone": "+1 (555) 474-2920", "address": "582 Stoddard Place, Homeland, Palau, 1350", "about": "Dolore exercitation cupidatat cillum voluptate voluptate nulla esse irure Lorem eu. Pariatur ex labore tempor excepteur pariatur fugiat irure irure veniam minim et. Voluptate laborum ullamco ex velit aliqua cupidatat ullamco id est pariatur fugiat.\r\n", "registered": "2015-09-13T07:59:47 +05:00", "latitude": 19.654033, "longitude": -142.388314, "tags": [ "est", "cillum", "Lorem", "in", "amet", "laboris", "et" ], "friends": [ { "id": 0, "name": "Lacy Savage" }, { "id": 1, "name": "Ware Carson" }, { "id": 2, "name": "Klein Mccullough" }, { "id": 3, "name": "Bettie Durham" } ], "greeting": "Hello, Fletcher Maldonado! You have 8 unread messages.", "favoriteFruit": "apple" }, { "_id": "5e3c5f5ebcc98b64c6a01468", "index": 3, "guid": "93832bce-52d5-4e5a-af99-38167fd277ec", "isActive": false, "balance": "$2,979.66", "picture": "http://placehold.it/32x32", "age": 37, "eyeColor": "brown", "name": "Burt Mcclain", "gender": "male", "company": "EXTRO", "email": "burtmcclain@extro.com", "phone": "+1 (555) 448-2214", "address": "417 Remsen Street, Haena, Alabama, 8546", "about": "Adipisicing aliqua nisi enim aliqua sunt enim elit cupidatat aute amet irure qui mollit dolor. Qui fugiat et in dolore Lorem ipsum velit ex velit nulla enim non. Sunt velit sint fugiat culpa ea. Irure duis quis sint sint quis ex magna nostrud voluptate eu proident nostrud. Aute dolor veniam culpa duis nulla esse laborum voluptate culpa dolor elit eu amet dolore.\r\n", "registered": "2018-06-08T05:35:46 +05:00", "latitude": -1.630596, "longitude": 139.718867, "tags": [ "cillum", "ullamco", "elit", "consequat", "eiusmod", "aliquip", "incididunt" ], "friends": [ { "id": 0, "name": "Duffy Foreman" }, { "id": 1, "name": "West Waller" }, { "id": 2, "name": "Olivia Vinson" } ], "greeting": "Hello, Burt Mcclain! You have 9 unread messages.", "favoriteFruit": "strawberry" }, { "_id": "5e3c5f5ee64bd067c054fc90", "index": 4, "guid": "c8d126ce-7f08-41c9-9f8f-44cfbd81ab63", "isActive": false, "balance": "$2,690.68", "picture": "http://placehold.it/32x32", "age": 26, "eyeColor": "green", "name": "Bobbi Drake", "gender": "female", "company": "OMNIGOG", "email": "bobbidrake@omnigog.com", "phone": "+1 (555) 521-2884", "address": "512 Bartlett Place, Trona, California, 6087", "about": "Duis tempor ut ullamco adipisicing dolore. Anim mollit pariatur exercitation aliqua consequat ex aliqua. Consectetur eu reprehenderit officia cupidatat veniam aliqua voluptate magna elit ad sit est. Consectetur eu reprehenderit officia cupidatat veniam aliqua voluptate magna elit ad sit est. Consectetur eu reprehenderit officia cupidatat veniam aliqua voluptate magna elit ad sit est. Do minim cupidatat dolore aliqua eiusmod sit. Veniam est veniam nulla nostrud ullamco non est do cupidatat.\r\n", "registered": "2017-10-18T10:09:37 +05:00", "latitude": -39.734003, "longitude": -100.722304, "tags": [ "cillum", "consequat", "ullamco", "commodo", "elit", "aliqua", "occaecat" ], "friends": [ { "id": 0, "name": "Jamie Ward" }, { "id": 1, "name": "Francisca Stark" }, { "id": 2, "name": "Dorsey Moreno" } ], "greeting": "Hello, Bobbi Drake! You have 3 unread messages.", "favoriteFruit": "banana" } ]

@lpunderscore , can you please try it instead of your setting?

lpunderscore commented 4 years ago

@AlexanderSher

The mock json worked. It has the exact same size as my json string. My json string is created using JsonConvert.SerializeObject(resources, Formatting.None);. The json is a simple object array of this:

{"BaseAddress":"https://some-url.azurewebsites.net/","Theme":"Some.String.Here","Environment":"Development","Status":"Online","LastStatusUpdate":"2019-11-13T15:26:52.732+00:00","IsActive":true}

Not sure how a json serialized object could make an api call crash...

AlexanderSher commented 4 years ago

Ok, at least I've found where it happens: image

lpunderscore commented 4 years ago

@AlexanderSher Just to make sure, are you saying this is a bug or am I doing something wrong? Is there a workaround?

Right now i'm gonna stick with calling the api directly (without using the client). Next step I was thinking of compressing the json before sending with the client if need be.

Le me know, thanks.

AlexanderSher commented 4 years ago

The bug is on the server side, they measure the size of the value before unescape, not after it. When you use API directly, you probably don't escape some of the characters that Utf8JsonWriter escapes, e.g. /.

AlexanderSher commented 4 years ago

But in general, there still will be a limit, so if your configuration value can grow above 10K, compression may be an option.

drago-draganov commented 4 years ago

@lpunderscore,

It doesn't seem as optimal use of AppConfiguration. It looks like you are trying to augment configuration primitives, instead of going after native key-value entries. I don't have context here, but that may have long term impact on your configuration management.

lpunderscore commented 4 years ago

@drago-draganov Yeah you might be right, we are having the same discussion internally.

There is also the fact that one of those is a setting for a feature toggle, which might grow to more than 10 kb (not sure if those have the same limitation). Sitting in the feature configuration just make sense in terms of location and change detection etc.

Most of our configuration are simple key value and others are keyvault links. At the time, it made sense to have this one there as well. I guess we will have to revisit this decision.

Thanks.

drago-draganov commented 4 years ago

@lpunderscore

FeatureFlags is also subject to the 10KB payload limit.

If you provide some abstract context and details of your configuration structure and goals, I'll be glad to assist.

zhenlan commented 4 years ago

@lpunderscore please open a new issue if you need further assistance. I'm closing this one.