search

Azure / azure-data-lake-store-java

Microsoft Azure Data Lake Store Filesystem Library for Java
Other
20 stars 36 forks source link

HttpTransport swallowing detailed error message. #3

Closed jinhyukchang closed 7 years ago

jinhyukchang commented 7 years ago

Hi,

I am currently implicitly using ADL store client via Hadoop ADLFileSystem.

While testing out in firewalled environment, I found out it's really hard to know the cause of failure due to very short error message. Turning on DEBUG does not help as well as response object has not been updated -- also, it's hard to expect DEBUG mode to be on in production.

Below is error message and link to the code.

https://github.com/Azure/azure-data-lake-store-java/blob/master/src/main/java/com/microsoft/azure/datalake/store/HttpTransport.java#L185

DEBUG AADToken: starting to fetch token using client creds for client ID DEBUG HTTPRequest,Failed,cReqId:.0,lat:47,err:HTTP0(null),Reqlen:0,Resplen:0,token_ns:47224784,sReqId:null,path:/tmp/,qp:op=MKDIRS&permission=750&api-version=2016-11-01 WARN Caught exception. This may be retried. com.microsoft.azure.datalake.store.ADLException: Error fetching access token Last encountered exception thrown after 1 tries [HTTP0(null)] at com.microsoft.azure.datalake.store.ADLStoreClient.getExceptionFromResponse(ADLStoreClient.java:1018) at com.microsoft.azure.datalake.store.ADLStoreClient.createDirectory(ADLStoreClient.java:471) at org.apache.hadoop.fs.adl.AdlFileSystem.mkdirs(AdlFileSystem.java:560) ...

Thanks, Jin

asikaria-msft commented 7 years ago

Indeed, looks like the exception returned by getAccessToken() is getting swallowed, and a generic error message (Error fetching access token) is returned instead. The fix is to bubble up the original exception.

We will make this fix in the next release of the SDK.

Thanks for reporting.

spyk commented 7 years ago

This has already been fixed on master, right? I tried with latest version and it now prints the root cause of the exception.

jinhyukchang commented 7 years ago

I can see the fix has been applied. https://github.com/Azure/azure-data-lake-store-java/commit/1bf9594522d8898c22e8764f65eb72ad45f859d3#diff-91304ddce2ef47f7f9ae6a1c4b6c8ade

Thanks for the fix. Jin