search

Azure / azure-data-lake-store-net

Azure Data Lake Store .Net SDK
MIT License
18 stars 24 forks source link

Error: Specified value has invalid Control characters #19

Closed lpunderscore closed 6 years ago

lpunderscore commented 6 years ago

Exception occurs when using ConcurrentAppendAsync in an azure function listening to a cosmosdb changefeed.

Error in concurrent append for file files/edited-entity-type/y=2018/m=8/d=6/h=14/m=8/data.json. Operation: CONCURRENTAPPEND failed with Unknown Error: Specified value has invalid Control characters. Parameter name: value Source: System.Net.WebHeaderCollection StackTrace: at System.Net.HttpValidationHelpers.CheckBadHeaderValueChars(String value) at System.Net.WebHeaderCollection.Set(String name, String value) at Microsoft.Azure.DataLake.Store.WebTransport.AssignCommonHttpHeaders(HttpWebRequest webReq, AdlsClient client, RequestOptions req, String token, String opMethod, IDictionary2 customHeaders) at Microsoft.Azure.DataLake.Store.WebTransport.MakeSingleCallAsync(String opCode, String path, ByteBuffer requestData, ByteBuffer responseData, QueryParams qp, AdlsClient client, RequestOptions req, OperationResponse resp, CancellationToken cancelToken, IDictionary2 customHeaders). Last encountered exception thrown after 1 tries. [Specified value has invalid Control characters. Parameter name: value] [ServerRequestId:] ID:: b6c67b50-46e7-45ad-908c-a00ac93db54a

rahuldutta90 commented 6 years ago

@lpunderscore What is the version of sdk you are using?1.1.8? 1.1.8 had this issue. If you are using 1.1.8, can you pelase switch to 1.1.9?

lpunderscore commented 6 years ago

@rahuldutta90 I'm using 1.1.8 yes. Will update to 1.1.9 and monitor the result. Will keep you posted, I get thousands of those every day, I should know quickly.

rahuldutta90 commented 6 years ago

@lpunderscore little context why you are seeing this error: We send a latency details of previous request as a part of a custom header of the current request. Now when there is a exception, the error details are also sent as a part of http header. Some errors which are not handled properly can cause the error message have these invalid characters for http header.

1.1.8 had a bug due to which lot of other people faced same error. Just curious: do you know if you were getting Unauthorized error or BadRequest error.

Now updating to 1.1.9 might not fix it because there might be some dotnet exception that returns control characters as part of exception message. So if it does not fix it, then you can try setting LatencyTracker.Disable() at the begining of code so that the sdk does not pass latency details and we can isolate the actual exception.

lpunderscore commented 6 years ago

The exception I posted is all the data I could find in application insights of the error. It says "Unkonwn error" with the stack trace. I don't know the nature of the underlying exception since the dependency doesn't report it apparently.

lpunderscore commented 6 years ago

So far I have not received this error in the last 12 hours of running (couple million requests to DL). Thanks!

Will update if it happens again.

rahuldutta90 commented 6 years ago

I am closing the issue. Please reopen it, if you encounter this again.