search

JuliaLang / MbedTLS.jl

Wrapper around mbedtls
Other
41 stars 50 forks source link

MbedTLS 1.1.1 causing issues for authentication on Azure #250

Closed samtkaplan closed 2 years ago

samtkaplan commented 2 years ago

Hi,

I'm noticing that MbedTLS 1.1.1 is causing issues when authenticating on Azure using the AzSessions.jl package. In particular, I am noticing that the AzSessions.jl retry logic is being hit more than expected with a Base.IOError thrown. The trace indicates that the problem may be coming from the MbedTLS ssl_write method. I suppose that this may be related to #247?

Here is a simple code to test (Please note that sessions is using device code flow for authentication):

using AzSessions, Dates
session = AzSession(protocal=AzDeviceCodeCredentials, client_id="myclientid", tenant="mytenant")
while true
    sleep(120)
    tkn = token(session)
    @info "tkn=$(tkn[1:5]), $(now())"
end

The error is triggered when AzSessions uses its OAuth2 refresh-token to retrieve a new OAuth2 access-token. Note that the refresh-token logic is only triggered about once per hour (when the access-token expires). Below is a screen-shot where on the left we are using MbedTLS 1.1.1, and on the right we are using MbedTLS 1.1.0. image Notice that on the right, we hit the IOError causing the AzSessions retry logic to be invoked, where-as on the right (with MbedTLS 1.1.0) we don't hit the retry logic.

I have not done much to dig into the possible root cause, but I thought I would share my current observation.

Thanks!

Sam

quinnj commented 2 years ago

I just tagged v1.1.2 (like 5 minutes ago) with a fix for a an issue introduced in v1.1.1. Could you pull and try again to see if the issue persists?

samtkaplan commented 2 years ago

Unfortunetly, it looks like the problem might still be there with MbedTLS 1.1.2.

Here's the Pkg.status() image and here's the output (same test-code as before): image Note that the error message is a little less verbose this time, but I can add back the printing of the trace and run again.

edit Here's a trace from the error with MbedTLS 1.1.2

[ Info: tkn=eyJ0eXAi, 2022-07-30T19:36:38.912
┌ Warning: retry 1, sleeping for 1.6882656734235162 seconds, e=HTTP.Exceptions.RequestError(HTTP.Messages.Request:
│ """
│ POST /fd799da1-bfc1-4234-a91c-72b3a1cb9e26/oauth2/token HTTP/1.1
│ Content-Type: application/x-www-form-urlencoded
│ Host: login.microsoftonline.com
│ Accept: */*
│ User-Agent: HTTP.jl/1.7.3
│ Content-Length: 1098
│ Accept-Encoding: gzip
│ Cookie: fpc=AvfbrmcNDf5JqoLGlOMSmc5AMlg6AQAAAFFud9oOAAAA; stsservicecookie=estsfd; x-ms-gateway-slice=estsfd
│ 
│ client_id=641ea2ce-3859-4a40-9292-679f679d94a6&refresh_token=0.AQ4AoZ15_cG_NEKpHHKzocueJs6iHmRZOEBKkpJnn2edlKYOACw.AgABAAEAAAD--DLA3VO7QrddgJg7WevrAgDs_wQA9P9Qx8_eF--e0HjXC1fyVwy3u-dqS1vEzLLn4Ep4YJw3zuCwTpVO9mPu4f1uYHtjo3UB0Zid4VHv8aznBMFFeZl2xSr5m397dsnJL-H_4MjytXBWaIOr2CmXXimGlY7Hu7D5ugkj--9WJPiYcbClIcnxlvBD643oREnGESv7oguu2H1gmGi7nC6tzlpqXbtREGRwneh_QkfRjo_jsQsWoCUQ0n48wdxEe4GbOWwPOoFJK8WafsVxcSi_73iSGfrR6tMu6du3cxJy2km1yhvFLkZqAmQc--ltkLoKtEX6GjjDnGUYK8LpgLyMPTHsiE7K5grsC4jKdH92fEzDCIgkN3Iw1oacZcCMEtUko0Di8gyTq1z018BYCoe9KsFbH5ZeHsYO1NwqB6NXEt-Hgd_xzq7ZqnMqXEEb3oubL1-cCPfUighPc4UR-FlMyO4XwIk784QJHSAnwHzg3DOMLYB1mwLoTbq3Azmjp1xJBEFr5gI_ceCWRBMGzpMjcs9UxwQmjfw0NVP9DwTVszbhaIk5BDAqTzhsVZJnHEWkbhpV1LOyx01tNQlrDqVV-ILwf9Ob5y7APHGm1h5kcWTKJbVj8rCaP1_AD-k8gWNP6PkJARNDPC56RBREE1flgc_sts8cYYB2lOZNmH1tnwo94R6yxDbXjhDKLAA1lwPNiSI-zNCtCPqcznvSYd9Ce0p9mHfUgMKhIPxDHQS1mwzaJ9iV3i-njeolXiHBpIjXfcG4YbxHxrVTOOQ28hyztHg7ForkvkKwW9Gd9kyh6iwkp9N91VTTtdOHxICuQjn9&grant_type=refresh_token&scope=openid+of
│ ⋮
│ 1098-byte body
│ """, Base.IOError("write: connection reset by peer (ECONNRESET)", -104))
└ @ AzSessions ~/.julia/dev/AzSessions/src/AzSessions.jl:99
┌ Warning: HTTP.Exceptions.RequestError(HTTP.Messages.Request:
│ """
│ POST /fd799da1-bfc1-4234-a91c-72b3a1cb9e26/oauth2/token HTTP/1.1
│ Content-Type: application/x-www-form-urlencoded
│ Host: login.microsoftonline.com
│ Accept: */*
│ User-Agent: HTTP.jl/1.7.3
│ Content-Length: 1098
│ Accept-Encoding: gzip
│ Cookie: fpc=AvfbrmcNDf5JqoLGlOMSmc5AMlg6AQAAAFFud9oOAAAA; stsservicecookie=estsfd; x-ms-gateway-slice=estsfd
│ 
│ client_id=641ea2ce-3859-4a40-9292-679f679d94a6&refresh_token=0.AQ4AoZ15_cG_NEKpHHKzocueJs6iHmRZOEBKkpJnn2edlKYOACw.AgABAAEAAAD--DLA3VO7QrddgJg7WevrAgDs_wQA9P9Qx8_eF--e0HjXC1fyVwy3u-dqS1vEzLLn4Ep4YJw3zuCwTpVO9mPu4f1uYHtjo3UB0Zid4VHv8aznBMFFeZl2xSr5m397dsnJL-H_4MjytXBWaIOr2CmXXimGlY7Hu7D5ugkj--9WJPiYcbClIcnxlvBD643oREnGESv7oguu2H1gmGi7nC6tzlpqXbtREGRwneh_QkfRjo_jsQsWoCUQ0n48wdxEe4GbOWwPOoFJK8WafsVxcSi_73iSGfrR6tMu6du3cxJy2km1yhvFLkZqAmQc--ltkLoKtEX6GjjDnGUYK8LpgLyMPTHsiE7K5grsC4jKdH92fEzDCIgkN3Iw1oacZcCMEtUko0Di8gyTq1z018BYCoe9KsFbH5ZeHsYO1NwqB6NXEt-Hgd_xzq7ZqnMqXEEb3oubL1-cCPfUighPc4UR-FlMyO4XwIk784QJHSAnwHzg3DOMLYB1mwLoTbq3Azmjp1xJBEFr5gI_ceCWRBMGzpMjcs9UxwQmjfw0NVP9DwTVszbhaIk5BDAqTzhsVZJnHEWkbhpV1LOyx01tNQlrDqVV-ILwf9Ob5y7APHGm1h5kcWTKJbVj8rCaP1_AD-k8gWNP6PkJARNDPC56RBREE1flgc_sts8cYYB2lOZNmH1tnwo94R6yxDbXjhDKLAA1lwPNiSI-zNCtCPqcznvSYd9Ce0p9mHfUgMKhIPxDHQS1mwzaJ9iV3i-njeolXiHBpIjXfcG4YbxHxrVTOOQ28hyztHg7ForkvkKwW9Gd9kyh6iwkp9N91VTTtdOHxICuQjn9&grant_type=refresh_token&scope=openid+of
│ ⋮
│ 1098-byte body
│ """, Base.IOError("write: connection reset by peer (ECONNRESET)", -104))
│       error type: HTTP.Exceptions.RequestError
│ IOError: write: connection reset by peer (ECONNRESET)
│ Stacktrace:
│   [1] uv_write(s::Sockets.TCPSocket, p::Ptr{UInt8}, n::UInt64)
│     @ Base ./stream.jl:1010
│   [2] unsafe_write(s::Sockets.TCPSocket, p::Ptr{UInt8}, n::UInt64)
│     @ Base ./stream.jl:1064
│   [3] f_send(c_bio::Ptr{Nothing}, buf::Ptr{UInt8}, nbytes::UInt64)
│     @ MbedTLS ~/.julia/packages/MbedTLS/W4GKm/src/ssl.jl:276
│   [4] ssl_write(ctx::MbedTLS.SSLContext, ptr::Ptr{UInt8}, n::UInt64)
│     @ MbedTLS ~/.julia/packages/MbedTLS/W4GKm/src/ssl.jl:745
│   [5] ssl_unsafe_write(ctx::MbedTLS.SSLContext, buf::Ptr{UInt8}, nbytes::UInt64)
│     @ MbedTLS ~/.julia/packages/MbedTLS/W4GKm/src/ssl.jl:250
│   [6] unsafe_write(ctx::MbedTLS.SSLContext, msg::Ptr{UInt8}, N::UInt64)
│     @ MbedTLS ~/.julia/packages/MbedTLS/W4GKm/src/ssl.jl:430
│   [7] unsafe_write(c::HTTP.ConnectionPool.Connection, p::Ptr{UInt8}, n::UInt64)
│     @ HTTP.ConnectionPool ~/.julia/packages/HTTP/XJG1J/src/ConnectionPool.jl:102
│   [8] unsafe_write
│     @ ./io.jl:648 [inlined]
│   [9] write
│     @ ./io.jl:671 [inlined]
│  [10] startwrite(http::HTTP.Streams.Stream{HTTP.Messages.Response, HTTP.ConnectionPool.Connection})
│     @ HTTP.Streams ~/.julia/packages/HTTP/XJG1J/src/Streams.jl:81
│  [11] streamlayer(stream::HTTP.Streams.Stream{HTTP.Messages.Response, HTTP.ConnectionPool.Connection}; iofunction::Nothing, decompress::Bool, kw::Base.Pairs{Symbol, Int64, Tuple{Symbol}, NamedTuple{(:verbose,), Tuple{Int64}}})
│     @ HTTP.StreamRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/StreamRequest.jl:26
│  [12] (::HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}})(stream::HTTP.Streams.Stream{HTTP.Messages.Response, HTTP.ConnectionPool.Connection}; status_exception::Bool, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose), Tuple{Nothing, Bool, Int64}}})
│     @ HTTP.ExceptionRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/ExceptionRequest.jl:14
│  [13] (::HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}})(stream::HTTP.Streams.Stream{HTTP.Messages.Response, HTTP.ConnectionPool.Connection}; readtimeout::Int64, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose), Tuple{Nothing, Bool, Int64}}})
│     @ HTTP.TimeoutRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/TimeoutRequest.jl:17
│  [14] (::HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}})(req::HTTP.Messages.Request; proxy::Nothing, socket_type::Type, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose), Tuple{Nothing, Bool, Int64}}})
│     @ HTTP.ConnectionRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/ConnectionRequest.jl:103
│  [15] (::HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}})(req::HTTP.Messages.Request; canonicalize_headers::Bool, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose), Tuple{Nothing, Bool, Int64}}})
│     @ HTTP.CanonicalizeRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/CanonicalizeRequest.jl:17
│  [16] (::HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}})(req::HTTP.Messages.Request; retry::Bool, retries::Int64, retry_non_idempotent::Bool, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose), Tuple{Nothing, Bool, Int64}}})
│     @ HTTP.RetryRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/RetryRequest.jl:25
│  [17] (::HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}})(req::HTTP.Messages.Request; cookies::Bool, cookiejar::HTTP.Cookies.CookieJar, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, NTuple{4, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose, :retry), Tuple{Nothing, Bool, Int64, Bool}}})
│     @ HTTP.CookieRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/CookieRequest.jl:42
│  [18] (::HTTP.ContentTypeDetection.var"#1#3"{HTTP.ContentTypeDetection.var"#1#2#4"{HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}}}})(req::HTTP.Messages.Request; detect_content_type::Bool, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, NTuple{4, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose, :retry), Tuple{Nothing, Bool, Int64, Bool}}})
│     @ HTTP.ContentTypeDetection ~/.julia/packages/HTTP/XJG1J/src/clientlayers/ContentTypeRequest.jl:23
│  [19] (::HTTP.BasicAuthRequest.var"#1#3"{HTTP.BasicAuthRequest.var"#1#2#4"{HTTP.ContentTypeDetection.var"#1#3"{HTTP.ContentTypeDetection.var"#1#2#4"{HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}}}}}})(req::HTTP.Messages.Request; basicauth::Bool, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, NTuple{4, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose, :retry), Tuple{Nothing, Bool, Int64, Bool}}})
│     @ HTTP.BasicAuthRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/BasicAuthRequest.jl:22
│  [20] (::HTTP.DefaultHeadersRequest.var"#1#3"{HTTP.DefaultHeadersRequest.var"#1#2#4"{HTTP.BasicAuthRequest.var"#1#3"{HTTP.BasicAuthRequest.var"#1#2#4"{HTTP.ContentTypeDetection.var"#1#3"{HTTP.ContentTypeDetection.var"#1#2#4"{HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}}}}}}}})(req::HTTP.Messages.Request; iofunction::Nothing, decompress::Bool, kw::Base.Pairs{Symbol, Integer, Tuple{Symbol, Symbol}, NamedTuple{(:verbose, :retry), Tuple{Int64, Bool}}})
│     @ HTTP.DefaultHeadersRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/DefaultHeadersRequest.jl:47
│  [21] (::HTTP.RedirectRequest.var"#1#4"{HTTP.RedirectRequest.var"#1#2#5"{HTTP.DefaultHeadersRequest.var"#1#3"{HTTP.DefaultHeadersRequest.var"#1#2#4"{HTTP.BasicAuthRequest.var"#1#3"{HTTP.BasicAuthRequest.var"#1#2#4"{HTTP.ContentTypeDetection.var"#1#3"{HTTP.ContentTypeDetection.var"#1#2#4"{HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}}}}}}}}}})(req::HTTP.Messages.Request; redirect::Bool, redirect_limit::Int64, redirect_method::Nothing, forwardheaders::Bool, response_stream::Nothing, kw::Base.Pairs{Symbol, Integer, Tuple{Symbol, Symbol}, NamedTuple{(:verbose, :retry), Tuple{Int64, Bool}}})
│     @ HTTP.RedirectRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/RedirectRequest.jl:25
│  [22] #1#2
│     @ ~/.julia/packages/HTTP/XJG1J/src/clientlayers/DebugRequest.jl:22 [inlined]
│  [23] (::HTTP.MessageRequest.var"#1#3"{HTTP.MessageRequest.var"#1#2#4"{HTTP.DebugRequest.var"#1#4"{HTTP.DebugRequest.var"#1#2#5"{HTTP.RedirectRequest.var"#1#4"{HTTP.RedirectRequest.var"#1#2#5"{HTTP.DefaultHeadersRequest.var"#1#3"{HTTP.DefaultHeadersRequest.var"#1#2#4"{HTTP.BasicAuthRequest.var"#1#3"{HTTP.BasicAuthRequest.var"#1#2#4"{HTTP.ContentTypeDetection.var"#1#3"{HTTP.ContentTypeDetection.var"#1#2#4"{HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}}}}}}}}}}}}}})(method::String, url::URIs.URI, headers::Vector{Pair{SubString{String}, SubString{String}}}, body::String; response_stream::Nothing, http_version::VersionNumber, kw::Base.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:retry,), Tuple{Bool}}})
│     @ HTTP.MessageRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/MessageRequest.jl:17
│  [24] request(stack::HTTP.MessageRequest.var"#1#3"{HTTP.MessageRequest.var"#1#2#4"{HTTP.DebugRequest.var"#1#4"{HTTP.DebugRequest.var"#1#2#5"{HTTP.RedirectRequest.var"#1#4"{HTTP.RedirectRequest.var"#1#2#5"{HTTP.DefaultHeadersRequest.var"#1#3"{HTTP.DefaultHeadersRequest.var"#1#2#4"{HTTP.BasicAuthRequest.var"#1#3"{HTTP.BasicAuthRequest.var"#1#2#4"{HTTP.ContentTypeDetection.var"#1#3"{HTTP.ContentTypeDetection.var"#1#2#4"{HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}}}}}}}}}}}}}}, method::String, url::String, h::Vector{Pair{String, String}}, b::String, q::Nothing; headers::Vector{Pair{String, String}}, body::String, query::Nothing, kw::Base.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:retry,), Tuple{Bool}}})
│     @ HTTP ~/.julia/packages/HTTP/XJG1J/src/HTTP.jl:412
│  [25] #request#16
│     @ ~/.julia/packages/HTTP/XJG1J/src/HTTP.jl:272 [inlined]
│  [26] refresh_token(session::AzSessions.AzDeviceCodeFlowSession)
│     @ AzSessions ~/.julia/dev/AzSessions/src/AzSessions.jl:108
│  [27] token(session::AzSessions.AzDeviceCodeFlowSession, bootstrap::Bool; offset::Minute)
│     @ AzSessions ~/.julia/dev/AzSessions/src/AzSessions.jl:586
│  [28] token (repeats 2 times)
│     @ ~/.julia/dev/AzSessions/src/AzSessions.jl:574 [inlined]
│  [29] top-level scope
│     @ ./REPL[8]:3
│  [30] eval
│     @ ./boot.jl:373 [inlined]
│  [31] eval_user_input(ast::Any, backend::REPL.REPLBackend)
│     @ REPL /opt/julia/share/julia/stdlib/v1.7/REPL/src/REPL.jl:150
│  [32] repl_backend_loop(backend::REPL.REPLBackend)
│     @ REPL /opt/julia/share/julia/stdlib/v1.7/REPL/src/REPL.jl:246
│  [33] start_repl_backend(backend::REPL.REPLBackend, consumer::Any)
│     @ REPL /opt/julia/share/julia/stdlib/v1.7/REPL/src/REPL.jl:231
│  [34] run_repl(repl::REPL.AbstractREPL, consumer::Any; backend_on_current_task::Bool)
│     @ REPL /opt/julia/share/julia/stdlib/v1.7/REPL/src/REPL.jl:364
│  [35] run_repl(repl::REPL.AbstractREPL, consumer::Any)
│     @ REPL /opt/julia/share/julia/stdlib/v1.7/REPL/src/REPL.jl:351
│  [36] (::Base.var"#936#938"{Bool, Bool, Bool})(REPL::Module)
│     @ Base ./client.jl:394
│  [37] #invokelatest#2
│     @ ./essentials.jl:716 [inlined]
│  [38] invokelatest
│     @ ./essentials.jl:714 [inlined]
│  [39] run_main_repl(interactive::Bool, quiet::Bool, banner::Bool, history_file::Bool, color_set::Bool)
│     @ Base ./client.jl:379
│  [40] exec_options(opts::Base.JLOptions)
│     @ Base ./client.jl:309
│  [41] _start()
│     @ Base ./client.jl:495
│ HTTP.Exceptions.RequestError(HTTP.Messages.Request:
│ """
│ POST /fd799da1-bfc1-4234-a91c-72b3a1cb9e26/oauth2/token HTTP/1.1
│ Content-Type: application/x-www-form-urlencoded
│ Host: login.microsoftonline.com
│ Accept: */*
│ User-Agent: HTTP.jl/1.7.3
│ Content-Length: 1098
│ Accept-Encoding: gzip
│ Cookie: fpc=AvfbrmcNDf5JqoLGlOMSmc5AMlg6AQAAAFFud9oOAAAA; stsservicecookie=estsfd; x-ms-gateway-slice=estsfd
│ 
│ client_id=641ea2ce-3859-4a40-9292-679f679d94a6&refresh_token=0.AQ4AoZ15_cG_NEKpHHKzocueJs6iHmRZOEBKkpJnn2edlKYOACw.AgABAAEAAAD--DLA3VO7QrddgJg7WevrAgDs_wQA9P9Qx8_eF--e0HjXC1fyVwy3u-dqS1vEzLLn4Ep4YJw3zuCwTpVO9mPu4f1uYHtjo3UB0Zid4VHv8aznBMFFeZl2xSr5m397dsnJL-H_4MjytXBWaIOr2CmXXimGlY7Hu7D5ugkj--9WJPiYcbClIcnxlvBD643oREnGESv7oguu2H1gmGi7nC6tzlpqXbtREGRwneh_QkfRjo_jsQsWoCUQ0n48wdxEe4GbOWwPOoFJK8WafsVxcSi_73iSGfrR6tMu6du3cxJy2km1yhvFLkZqAmQc--ltkLoKtEX6GjjDnGUYK8LpgLyMPTHsiE7K5grsC4jKdH92fEzDCIgkN3Iw1oacZcCMEtUko0Di8gyTq1z018BYCoe9KsFbH5ZeHsYO1NwqB6NXEt-Hgd_xzq7ZqnMqXEEb3oubL1-cCPfUighPc4UR-FlMyO4XwIk784QJHSAnwHzg3DOMLYB1mwLoTbq3Azmjp1xJBEFr5gI_ceCWRBMGzpMjcs9UxwQmjfw0NVP9DwTVszbhaIk5BDAqTzhsVZJnHEWkbhpV1LOyx01tNQlrDqVV-ILwf9Ob5y7APHGm1h5kcWTKJbVj8rCaP1_AD-k8gWNP6PkJARNDPC56RBREE1flgc_sts8cYYB2lOZNmH1tnwo94R6yxDbXjhDKLAA1lwPNiSI-zNCtCPqcznvSYd9Ce0p9mHfUgMKhIPxDHQS1mwzaJ9iV3i-njeolXiHBpIjXfcG4YbxHxrVTOOQ28hyztHg7ForkvkKwW9Gd9kyh6iwkp9N91VTTtdOHxICuQjn9&grant_type=refresh_token&scope=openid+of
│ ⋮
│ 1098-byte body
│ """, Base.IOError("write: connection reset by peer (ECONNRESET)", -104))
│ Stacktrace:
│   [1] (::HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}})(req::HTTP.Messages.Request; proxy::Nothing, socket_type::Type, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose), Tuple{Nothing, Bool, Int64}}})
│     @ HTTP.ConnectionRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/ConnectionRequest.jl:107
│   [2] (::HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}})(req::HTTP.Messages.Request; canonicalize_headers::Bool, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose), Tuple{Nothing, Bool, Int64}}})
│     @ HTTP.CanonicalizeRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/CanonicalizeRequest.jl:17
│   [3] (::HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}})(req::HTTP.Messages.Request; retry::Bool, retries::Int64, retry_non_idempotent::Bool, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose), Tuple{Nothing, Bool, Int64}}})
│     @ HTTP.RetryRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/RetryRequest.jl:25
│   [4] (::HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}})(req::HTTP.Messages.Request; cookies::Bool, cookiejar::HTTP.Cookies.CookieJar, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, NTuple{4, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose, :retry), Tuple{Nothing, Bool, Int64, Bool}}})
│     @ HTTP.CookieRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/CookieRequest.jl:42
│   [5] (::HTTP.ContentTypeDetection.var"#1#3"{HTTP.ContentTypeDetection.var"#1#2#4"{HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}}}})(req::HTTP.Messages.Request; detect_content_type::Bool, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, NTuple{4, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose, :retry), Tuple{Nothing, Bool, Int64, Bool}}})
│     @ HTTP.ContentTypeDetection ~/.julia/packages/HTTP/XJG1J/src/clientlayers/ContentTypeRequest.jl:23
│   [6] (::HTTP.BasicAuthRequest.var"#1#3"{HTTP.BasicAuthRequest.var"#1#2#4"{HTTP.ContentTypeDetection.var"#1#3"{HTTP.ContentTypeDetection.var"#1#2#4"{HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}}}}}})(req::HTTP.Messages.Request; basicauth::Bool, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, NTuple{4, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose, :retry), Tuple{Nothing, Bool, Int64, Bool}}})
│     @ HTTP.BasicAuthRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/BasicAuthRequest.jl:22
│   [7] (::HTTP.DefaultHeadersRequest.var"#1#3"{HTTP.DefaultHeadersRequest.var"#1#2#4"{HTTP.BasicAuthRequest.var"#1#3"{HTTP.BasicAuthRequest.var"#1#2#4"{HTTP.ContentTypeDetection.var"#1#3"{HTTP.ContentTypeDetection.var"#1#2#4"{HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}}}}}}}})(req::HTTP.Messages.Request; iofunction::Nothing, decompress::Bool, kw::Base.Pairs{Symbol, Integer, Tuple{Symbol, Symbol}, NamedTuple{(:verbose, :retry), Tuple{Int64, Bool}}})
│     @ HTTP.DefaultHeadersRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/DefaultHeadersRequest.jl:47
│   [8] (::HTTP.RedirectRequest.var"#1#4"{HTTP.RedirectRequest.var"#1#2#5"{HTTP.DefaultHeadersRequest.var"#1#3"{HTTP.DefaultHeadersRequest.var"#1#2#4"{HTTP.BasicAuthRequest.var"#1#3"{HTTP.BasicAuthRequest.var"#1#2#4"{HTTP.ContentTypeDetection.var"#1#3"{HTTP.ContentTypeDetection.var"#1#2#4"{HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}}}}}}}}}})(req::HTTP.Messages.Request; redirect::Bool, redirect_limit::Int64, redirect_method::Nothing, forwardheaders::Bool, response_stream::Nothing, kw::Base.Pairs{Symbol, Integer, Tuple{Symbol, Symbol}, NamedTuple{(:verbose, :retry), Tuple{Int64, Bool}}})
│     @ HTTP.RedirectRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/RedirectRequest.jl:25
│   [9] #1#2
│     @ ~/.julia/packages/HTTP/XJG1J/src/clientlayers/DebugRequest.jl:22 [inlined]
│  [10] (::HTTP.MessageRequest.var"#1#3"{HTTP.MessageRequest.var"#1#2#4"{HTTP.DebugRequest.var"#1#4"{HTTP.DebugRequest.var"#1#2#5"{HTTP.RedirectRequest.var"#1#4"{HTTP.RedirectRequest.var"#1#2#5"{HTTP.DefaultHeadersRequest.var"#1#3"{HTTP.DefaultHeadersRequest.var"#1#2#4"{HTTP.BasicAuthRequest.var"#1#3"{HTTP.BasicAuthRequest.var"#1#2#4"{HTTP.ContentTypeDetection.var"#1#3"{HTTP.ContentTypeDetection.var"#1#2#4"{HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}}}}}}}}}}}}}})(method::String, url::URIs.URI, headers::Vector{Pair{SubString{String}, SubString{String}}}, body::String; response_stream::Nothing, http_version::VersionNumber, kw::Base.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:retry,), Tuple{Bool}}})
│     @ HTTP.MessageRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/MessageRequest.jl:17
│  [11] request(stack::HTTP.MessageRequest.var"#1#3"{HTTP.MessageRequest.var"#1#2#4"{HTTP.DebugRequest.var"#1#4"{HTTP.DebugRequest.var"#1#2#5"{HTTP.RedirectRequest.var"#1#4"{HTTP.RedirectRequest.var"#1#2#5"{HTTP.DefaultHeadersRequest.var"#1#3"{HTTP.DefaultHeadersRequest.var"#1#2#4"{HTTP.BasicAuthRequest.var"#1#3"{HTTP.BasicAuthRequest.var"#1#2#4"{HTTP.ContentTypeDetection.var"#1#3"{HTTP.ContentTypeDetection.var"#1#2#4"{HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}}}}}}}}}}}}}}, method::String, url::String, h::Vector{Pair{String, String}}, b::String, q::Nothing; headers::Vector{Pair{String, String}}, body::String, query::Nothing, kw::Base.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:retry,), Tuple{Bool}}})
│     @ HTTP ~/.julia/packages/HTTP/XJG1J/src/HTTP.jl:412
│  [12] #request#16
│     @ ~/.julia/packages/HTTP/XJG1J/src/HTTP.jl:272 [inlined]
│  [13] refresh_token(session::AzSessions.AzDeviceCodeFlowSession)
│     @ AzSessions ~/.julia/dev/AzSessions/src/AzSessions.jl:108
│  [14] token(session::AzSessions.AzDeviceCodeFlowSession, bootstrap::Bool; offset::Minute)
│     @ AzSessions ~/.julia/dev/AzSessions/src/AzSessions.jl:586
│  [15] token (repeats 2 times)
│     @ ~/.julia/dev/AzSessions/src/AzSessions.jl:574 [inlined]
│  [16] top-level scope
│     @ ./REPL[8]:3
│  [17] eval
│     @ ./boot.jl:373 [inlined]
│  [18] eval_user_input(ast::Any, backend::REPL.REPLBackend)
│     @ REPL /opt/julia/share/julia/stdlib/v1.7/REPL/src/REPL.jl:150
│  [19] repl_backend_loop(backend::REPL.REPLBackend)
│     @ REPL /opt/julia/share/julia/stdlib/v1.7/REPL/src/REPL.jl:246
│  [20] start_repl_backend(backend::REPL.REPLBackend, consumer::Any)
│     @ REPL /opt/julia/share/julia/stdlib/v1.7/REPL/src/REPL.jl:231
│  [21] run_repl(repl::REPL.AbstractREPL, consumer::Any; backend_on_current_task::Bool)
│     @ REPL /opt/julia/share/julia/stdlib/v1.7/REPL/src/REPL.jl:364
│  [22] run_repl(repl::REPL.AbstractREPL, consumer::Any)
│     @ REPL /opt/julia/share/julia/stdlib/v1.7/REPL/src/REPL.jl:351
│  [23] (::Base.var"#936#938"{Bool, Bool, Bool})(REPL::Module)
│     @ Base ./client.jl:394
│  [24] #invokelatest#2
│     @ ./essentials.jl:716 [inlined]
│  [25] invokelatest
│     @ ./essentials.jl:714 [inlined]
│  [26] run_main_repl(interactive::Bool, quiet::Bool, banner::Bool, history_file::Bool, color_set::Bool)
│     @ Base ./client.jl:379
│  [27] exec_options(opts::Base.JLOptions)
│     @ Base ./client.jl:309
│  [28] _start()
│     @ Base ./client.jl:495
└ @ AzSessions ~/.julia/dev/AzSessions/src/AzSessions.jl:21
quinnj commented 2 years ago

ok, pinging @vtjnash here; I'm currently on vacation, but can help dig in and try to figure out what's going on once I get back (next week).

samtkaplan commented 2 years ago

Not a big rush since using 1.1.0 is a simple fix for now.

samtkaplan commented 2 years ago

Hi,

Just a small update given the release of MbedTLS 1.1.3 (Thanks to @mbauman for pointing me to the new release). It looks like we have the same issue. On the left is mbedtls 1.1.0 where we see no errors thrrown, in the middle is with mbedtls 1.1.3 where we see the same IOError, and on the right is mbedtls 1.1.2. image Note that in the case of mbedtls 1.1.3, I also see an mbedtls error which I guess was added between mbedtls 1.1.2 and 1.1.3 releases (see below screen-shot). image

~Apologies for the lack of a trace in these error messages. I can try another run with mbedtls 1.1.3 and where I collect the traces.~ edit Here is a trace:

 Warning: HTTP.Exceptions.RequestError(HTTP.Messages.Request:
│ """
│ POST /fd799da1-bfc1-4234-a91c-72b3a1cb9e26/oauth2/token HTTP/1.1
│ Content-Type: application/x-www-form-urlencoded
│ Host: login.microsoftonline.com
│ Accept: */*
│ User-Agent: HTTP.jl/1.7.3
│ Content-Length: 1058
│ Accept-Encoding: gzip
│ Cookie: fpc=AsmnufhD7wJPm3XODt9rRsNAMlg6AQAAAFyxgNoOAAAA; stsservicecookie=estsfd; x-ms-gateway-slice=estsfd
│ 
│ client_id=641ea2ce-3859-4a40-9292-679f679d94a6&refresh_token=********************&grant_type=refresh_token&scope=openid+offline_access+https://management.azure.co
│ ⋮
│ 1058-byte body
│ """, MbedTLS error code -78: NET - Sending information through the socket failed)
│       error type: HTTP.Exceptions.RequestError
│ MbedTLS error code -78: NET - Sending information through the socket failed
│ Stacktrace:
│   [1] ssl_unsafe_write(ctx::MbedTLS.SSLContext, buf::Ptr{UInt8}, nbytes::UInt64)
│     @ MbedTLS ~/.julia/packages/MbedTLS/F2OjD/src/ssl.jl:258
│   [2] unsafe_write(ctx::MbedTLS.SSLContext, msg::Ptr{UInt8}, N::UInt64)
│     @ MbedTLS ~/.julia/packages/MbedTLS/F2OjD/src/ssl.jl:435
│   [3] unsafe_write(c::HTTP.ConnectionPool.Connection, p::Ptr{UInt8}, n::UInt64)
│     @ HTTP.ConnectionPool ~/.julia/packages/HTTP/XJG1J/src/ConnectionPool.jl:102
│   [4] unsafe_write
│     @ ./io.jl:648 [inlined]
│   [5] write
│     @ ./io.jl:671 [inlined]
│   [6] startwrite(http::HTTP.Streams.Stream{HTTP.Messages.Response, HTTP.ConnectionPool.Connection})
│     @ HTTP.Streams ~/.julia/packages/HTTP/XJG1J/src/Streams.jl:81
│   [7] streamlayer(stream::HTTP.Streams.Stream{HTTP.Messages.Response, HTTP.ConnectionPool.Connection}; iofunction::Nothing, decompress::Bool, kw::Base.Pairs{Symbol, Int64, Tuple{Symbol}, NamedTuple{(:verbose,), Tuple{Int64}}})
│     @ HTTP.StreamRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/StreamRequest.jl:26
│   [8] (::HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}})(stream::HTTP.Streams.Stream{HTTP.Messages.Response, HTTP.ConnectionPool.Connection}; status_exception::Bool, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose), Tuple{Nothing, Bool, Int64}}})
│     @ HTTP.ExceptionRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/ExceptionRequest.jl:14
│   [9] (::HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}})(stream::HTTP.Streams.Stream{HTTP.Messages.Response, HTTP.ConnectionPool.Connection}; readtimeout::Int64, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose), Tuple{Nothing, Bool, Int64}}})
│     @ HTTP.TimeoutRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/TimeoutRequest.jl:17
│  [10] (::HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}})(req::HTTP.Messages.Request; proxy::Nothing, socket_type::Type, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose), Tuple{Nothing, Bool, Int64}}})
│     @ HTTP.ConnectionRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/ConnectionRequest.jl:103
│  [11] (::HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}})(req::HTTP.Messages.Request; canonicalize_headers::Bool, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose), Tuple{Nothing, Bool, Int64}}})
│     @ HTTP.CanonicalizeRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/CanonicalizeRequest.jl:17
│  [12] (::HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}})(req::HTTP.Messages.Request; retry::Bool, retries::Int64, retry_non_idempotent::Bool, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose), Tuple{Nothing, Bool, Int64}}})
│     @ HTTP.RetryRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/RetryRequest.jl:25
│  [13] (::HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}})(req::HTTP.Messages.Request; cookies::Bool, cookiejar::HTTP.Cookies.CookieJar, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, NTuple{4, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose, :retry), Tuple{Nothing, Bool, Int64, Bool}}})
│     @ HTTP.CookieRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/CookieRequest.jl:42
│  [14] (::HTTP.ContentTypeDetection.var"#1#3"{HTTP.ContentTypeDetection.var"#1#2#4"{HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}}}})(req::HTTP.Messages.Request; detect_content_type::Bool, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, NTuple{4, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose, :retry), Tuple{Nothing, Bool, Int64, Bool}}})
│     @ HTTP.ContentTypeDetection ~/.julia/packages/HTTP/XJG1J/src/clientlayers/ContentTypeRequest.jl:23
│  [15] (::HTTP.BasicAuthRequest.var"#1#3"{HTTP.BasicAuthRequest.var"#1#2#4"{HTTP.ContentTypeDetection.var"#1#3"{HTTP.ContentTypeDetection.var"#1#2#4"{HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}}}}}})(req::HTTP.Messages.Request; basicauth::Bool, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, NTuple{4, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose, :retry), Tuple{Nothing, Bool, Int64, Bool}}})
│     @ HTTP.BasicAuthRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/BasicAuthRequest.jl:22
│  [16] (::HTTP.DefaultHeadersRequest.var"#1#3"{HTTP.DefaultHeadersRequest.var"#1#2#4"{HTTP.BasicAuthRequest.var"#1#3"{HTTP.BasicAuthRequest.var"#1#2#4"{HTTP.ContentTypeDetection.var"#1#3"{HTTP.ContentTypeDetection.var"#1#2#4"{HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}}}}}}}})(req::HTTP.Messages.Request; iofunction::Nothing, decompress::Bool, kw::Base.Pairs{Symbol, Integer, Tuple{Symbol, Symbol}, NamedTuple{(:verbose, :retry), Tuple{Int64, Bool}}})
│     @ HTTP.DefaultHeadersRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/DefaultHeadersRequest.jl:47
│  [17] (::HTTP.RedirectRequest.var"#1#4"{HTTP.RedirectRequest.var"#1#2#5"{HTTP.DefaultHeadersRequest.var"#1#3"{HTTP.DefaultHeadersRequest.var"#1#2#4"{HTTP.BasicAuthRequest.var"#1#3"{HTTP.BasicAuthRequest.var"#1#2#4"{HTTP.ContentTypeDetection.var"#1#3"{HTTP.ContentTypeDetection.var"#1#2#4"{HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}}}}}}}}}})(req::HTTP.Messages.Request; redirect::Bool, redirect_limit::Int64, redirect_method::Nothing, forwardheaders::Bool, response_stream::Nothing, kw::Base.Pairs{Symbol, Integer, Tuple{Symbol, Symbol}, NamedTuple{(:verbose, :retry), Tuple{Int64, Bool}}})
│     @ HTTP.RedirectRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/RedirectRequest.jl:25
│  [18] #1#2
│     @ ~/.julia/packages/HTTP/XJG1J/src/clientlayers/DebugRequest.jl:22 [inlined]
│  [19] (::HTTP.MessageRequest.var"#1#3"{HTTP.MessageRequest.var"#1#2#4"{HTTP.DebugRequest.var"#1#4"{HTTP.DebugRequest.var"#1#2#5"{HTTP.RedirectRequest.var"#1#4"{HTTP.RedirectRequest.var"#1#2#5"{HTTP.DefaultHeadersRequest.var"#1#3"{HTTP.DefaultHeadersRequest.var"#1#2#4"{HTTP.BasicAuthRequest.var"#1#3"{HTTP.BasicAuthRequest.var"#1#2#4"{HTTP.ContentTypeDetection.var"#1#3"{HTTP.ContentTypeDetection.var"#1#2#4"{HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}}}}}}}}}}}}}})(method::String, url::URIs.URI, headers::Vector{Pair{SubString{String}, SubString{String}}}, body::String; response_stream::Nothing, http_version::VersionNumber, kw::Base.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:retry,), Tuple{Bool}}})
│     @ HTTP.MessageRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/MessageRequest.jl:17
│  [20] request(stack::HTTP.MessageRequest.var"#1#3"{HTTP.MessageRequest.var"#1#2#4"{HTTP.DebugRequest.var"#1#4"{HTTP.DebugRequest.var"#1#2#5"{HTTP.RedirectRequest.var"#1#4"{HTTP.RedirectRequest.var"#1#2#5"{HTTP.DefaultHeadersRequest.var"#1#3"{HTTP.DefaultHeadersRequest.var"#1#2#4"{HTTP.BasicAuthRequest.var"#1#3"{HTTP.BasicAuthRequest.var"#1#2#4"{HTTP.ContentTypeDetection.var"#1#3"{HTTP.ContentTypeDetection.var"#1#2#4"{HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}}}}}}}}}}}}}}, method::String, url::String, h::Vector{Pair{String, String}}, b::String, q::Nothing; headers::Vector{Pair{String, String}}, body::String, query::Nothing, kw::Base.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:retry,), Tuple{Bool}}})
│     @ HTTP ~/.julia/packages/HTTP/XJG1J/src/HTTP.jl:412
│  [21] #request#16
│     @ ~/.julia/packages/HTTP/XJG1J/src/HTTP.jl:272 [inlined]
│  [22] refresh_token(session::AzSessions.AzDeviceCodeFlowSession)
│     @ AzSessions ~/.julia/dev/AzSessions/src/AzSessions.jl:108
│  [23] token(session::AzSessions.AzDeviceCodeFlowSession, bootstrap::Bool; offset::Minute)
│     @ AzSessions ~/.julia/dev/AzSessions/src/AzSessions.jl:586
│  [24] token (repeats 2 times)
│     @ ~/.julia/dev/AzSessions/src/AzSessions.jl:574 [inlined]
│  [25] top-level scope
│     @ ./REPL[17]:2
│  [26] eval
│     @ ./boot.jl:373 [inlined]
│  [27] eval_user_input(ast::Any, backend::REPL.REPLBackend)
│     @ REPL /opt/julia/share/julia/stdlib/v1.7/REPL/src/REPL.jl:150
│  [28] repl_backend_loop(backend::REPL.REPLBackend)
│     @ REPL /opt/julia/share/julia/stdlib/v1.7/REPL/src/REPL.jl:246
│  [29] start_repl_backend(backend::REPL.REPLBackend, consumer::Any)
│     @ REPL /opt/julia/share/julia/stdlib/v1.7/REPL/src/REPL.jl:231
│  [30] run_repl(repl::REPL.AbstractREPL, consumer::Any; backend_on_current_task::Bool)
│     @ REPL /opt/julia/share/julia/stdlib/v1.7/REPL/src/REPL.jl:364
│  [31] run_repl(repl::REPL.AbstractREPL, consumer::Any)
│     @ REPL /opt/julia/share/julia/stdlib/v1.7/REPL/src/REPL.jl:351
│  [32] (::Base.var"#936#938"{Bool, Bool, Bool})(REPL::Module)
│     @ Base ./client.jl:394
│  [33] #invokelatest#2
│     @ ./essentials.jl:716 [inlined]
│  [34] invokelatest
│     @ ./essentials.jl:714 [inlined]
│  [35] run_main_repl(interactive::Bool, quiet::Bool, banner::Bool, history_file::Bool, color_set::Bool)
│     @ Base ./client.jl:379
│  [36] exec_options(opts::Base.JLOptions)
│     @ Base ./client.jl:309
│  [37] _start()
│     @ Base ./client.jl:495
│ HTTP.Exceptions.RequestError(HTTP.Messages.Request:
│ """
│ POST /fd799da1-bfc1-4234-a91c-72b3a1cb9e26/oauth2/token HTTP/1.1
│ Content-Type: application/x-www-form-urlencoded
│ Host: login.microsoftonline.com
│ Accept: */*
│ User-Agent: HTTP.jl/1.7.3
│ Content-Length: 1058
│ Accept-Encoding: gzip
│ Cookie: fpc=AsmnufhD7wJPm3XODt9rRsNAMlg6AQAAAFyxgNoOAAAA; stsservicecookie=estsfd; x-ms-gateway-slice=estsfd
│ 
│ client_id=641ea2ce-3859-4a40-9292-679f679d94a6&refresh_token=******************&grant_type=refresh_token&scope=openid+offline_access+https://management.azure.co
│ ⋮
│ 1058-byte body
│ """, MbedTLS error code -78: NET - Sending information through the socket failed)
│ Stacktrace:
│   [1] (::HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}})(req::HTTP.Messages.Request; proxy::Nothing, socket_type::Type, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose), Tuple{Nothing, Bool, Int64}}})
│     @ HTTP.ConnectionRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/ConnectionRequest.jl:107
│   [2] (::HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}})(req::HTTP.Messages.Request; canonicalize_headers::Bool, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose), Tuple{Nothing, Bool, Int64}}})
│     @ HTTP.CanonicalizeRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/CanonicalizeRequest.jl:17
│   [3] (::HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}})(req::HTTP.Messages.Request; retry::Bool, retries::Int64, retry_non_idempotent::Bool, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose), Tuple{Nothing, Bool, Int64}}})
│     @ HTTP.RetryRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/RetryRequest.jl:25
│   [4] (::HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}})(req::HTTP.Messages.Request; cookies::Bool, cookiejar::HTTP.Cookies.CookieJar, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, NTuple{4, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose, :retry), Tuple{Nothing, Bool, Int64, Bool}}})
│     @ HTTP.CookieRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/CookieRequest.jl:42
│   [5] (::HTTP.ContentTypeDetection.var"#1#3"{HTTP.ContentTypeDetection.var"#1#2#4"{HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}}}})(req::HTTP.Messages.Request; detect_content_type::Bool, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, NTuple{4, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose, :retry), Tuple{Nothing, Bool, Int64, Bool}}})
│     @ HTTP.ContentTypeDetection ~/.julia/packages/HTTP/XJG1J/src/clientlayers/ContentTypeRequest.jl:23
│   [6] (::HTTP.BasicAuthRequest.var"#1#3"{HTTP.BasicAuthRequest.var"#1#2#4"{HTTP.ContentTypeDetection.var"#1#3"{HTTP.ContentTypeDetection.var"#1#2#4"{HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}}}}}})(req::HTTP.Messages.Request; basicauth::Bool, kw::Base.Pairs{Symbol, Union{Nothing, Integer}, NTuple{4, Symbol}, NamedTuple{(:iofunction, :decompress, :verbose, :retry), Tuple{Nothing, Bool, Int64, Bool}}})
│     @ HTTP.BasicAuthRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/BasicAuthRequest.jl:22
│   [7] (::HTTP.DefaultHeadersRequest.var"#1#3"{HTTP.DefaultHeadersRequest.var"#1#2#4"{HTTP.BasicAuthRequest.var"#1#3"{HTTP.BasicAuthRequest.var"#1#2#4"{HTTP.ContentTypeDetection.var"#1#3"{HTTP.ContentTypeDetection.var"#1#2#4"{HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}}}}}}}})(req::HTTP.Messages.Request; iofunction::Nothing, decompress::Bool, kw::Base.Pairs{Symbol, Integer, Tuple{Symbol, Symbol}, NamedTuple{(:verbose, :retry), Tuple{Int64, Bool}}})
│     @ HTTP.DefaultHeadersRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/DefaultHeadersRequest.jl:47
│   [8] (::HTTP.RedirectRequest.var"#1#4"{HTTP.RedirectRequest.var"#1#2#5"{HTTP.DefaultHeadersRequest.var"#1#3"{HTTP.DefaultHeadersRequest.var"#1#2#4"{HTTP.BasicAuthRequest.var"#1#3"{HTTP.BasicAuthRequest.var"#1#2#4"{HTTP.ContentTypeDetection.var"#1#3"{HTTP.ContentTypeDetection.var"#1#2#4"{HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}}}}}}}}}})(req::HTTP.Messages.Request; redirect::Bool, redirect_limit::Int64, redirect_method::Nothing, forwardheaders::Bool, response_stream::Nothing, kw::Base.Pairs{Symbol, Integer, Tuple{Symbol, Symbol}, NamedTuple{(:verbose, :retry), Tuple{Int64, Bool}}})
│     @ HTTP.RedirectRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/RedirectRequest.jl:25
│   [9] #1#2
│     @ ~/.julia/packages/HTTP/XJG1J/src/clientlayers/DebugRequest.jl:22 [inlined]
│  [10] (::HTTP.MessageRequest.var"#1#3"{HTTP.MessageRequest.var"#1#2#4"{HTTP.DebugRequest.var"#1#4"{HTTP.DebugRequest.var"#1#2#5"{HTTP.RedirectRequest.var"#1#4"{HTTP.RedirectRequest.var"#1#2#5"{HTTP.DefaultHeadersRequest.var"#1#3"{HTTP.DefaultHeadersRequest.var"#1#2#4"{HTTP.BasicAuthRequest.var"#1#3"{HTTP.BasicAuthRequest.var"#1#2#4"{HTTP.ContentTypeDetection.var"#1#3"{HTTP.ContentTypeDetection.var"#1#2#4"{HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}}}}}}}}}}}}}})(method::String, url::URIs.URI, headers::Vector{Pair{SubString{String}, SubString{String}}}, body::String; response_stream::Nothing, http_version::VersionNumber, kw::Base.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:retry,), Tuple{Bool}}})
│     @ HTTP.MessageRequest ~/.julia/packages/HTTP/XJG1J/src/clientlayers/MessageRequest.jl:17
│  [11] request(stack::HTTP.MessageRequest.var"#1#3"{HTTP.MessageRequest.var"#1#2#4"{HTTP.DebugRequest.var"#1#4"{HTTP.DebugRequest.var"#1#2#5"{HTTP.RedirectRequest.var"#1#4"{HTTP.RedirectRequest.var"#1#2#5"{HTTP.DefaultHeadersRequest.var"#1#3"{HTTP.DefaultHeadersRequest.var"#1#2#4"{HTTP.BasicAuthRequest.var"#1#3"{HTTP.BasicAuthRequest.var"#1#2#4"{HTTP.ContentTypeDetection.var"#1#3"{HTTP.ContentTypeDetection.var"#1#2#4"{HTTP.CookieRequest.var"#1#5"{HTTP.CookieRequest.var"#1#2#6"{HTTP.RetryRequest.var"#1#4"{HTTP.RetryRequest.var"#1#2#5"{HTTP.CanonicalizeRequest.var"#1#3"{HTTP.CanonicalizeRequest.var"#1#2#4"{HTTP.ConnectionRequest.var"#1#4"{HTTP.ConnectionRequest.var"#1#2#5"{HTTP.TimeoutRequest.var"#1#4"{HTTP.TimeoutRequest.var"#1#2#5"{HTTP.ExceptionRequest.var"#1#3"{HTTP.ExceptionRequest.var"#1#2#4"{typeof(HTTP.StreamRequest.streamlayer)}}}}}}}}}}}}}}}}}}}}}}}}, method::String, url::String, h::Vector{Pair{String, String}}, b::String, q::Nothing; headers::Vector{Pair{String, String}}, body::String, query::Nothing, kw::Base.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:retry,), Tuple{Bool}}})
│     @ HTTP ~/.julia/packages/HTTP/XJG1J/src/HTTP.jl:412
│  [12] #request#16
│     @ ~/.julia/packages/HTTP/XJG1J/src/HTTP.jl:272 [inlined]
│  [13] refresh_token(session::AzSessions.AzDeviceCodeFlowSession)
│     @ AzSessions ~/.julia/dev/AzSessions/src/AzSessions.jl:108
│  [14] token(session::AzSessions.AzDeviceCodeFlowSession, bootstrap::Bool; offset::Minute)
│     @ AzSessions ~/.julia/dev/AzSessions/src/AzSessions.jl:586
│  [15] token (repeats 2 times)
│     @ ~/.julia/dev/AzSessions/src/AzSessions.jl:574 [inlined]
│  [16] top-level scope
│     @ ./REPL[17]:2
│  [17] eval
│     @ ./boot.jl:373 [inlined]
│  [18] eval_user_input(ast::Any, backend::REPL.REPLBackend)
│     @ REPL /opt/julia/share/julia/stdlib/v1.7/REPL/src/REPL.jl:150
│  [19] repl_backend_loop(backend::REPL.REPLBackend)
│     @ REPL /opt/julia/share/julia/stdlib/v1.7/REPL/src/REPL.jl:246
│  [20] start_repl_backend(backend::REPL.REPLBackend, consumer::Any)
│     @ REPL /opt/julia/share/julia/stdlib/v1.7/REPL/src/REPL.jl:231
│  [21] run_repl(repl::REPL.AbstractREPL, consumer::Any; backend_on_current_task::Bool)
│     @ REPL /opt/julia/share/julia/stdlib/v1.7/REPL/src/REPL.jl:364
│  [22] run_repl(repl::REPL.AbstractREPL, consumer::Any)
│     @ REPL /opt/julia/share/julia/stdlib/v1.7/REPL/src/REPL.jl:351
│  [23] (::Base.var"#936#938"{Bool, Bool, Bool})(REPL::Module)
│     @ Base ./client.jl:394
│  [24] #invokelatest#2
│     @ ./essentials.jl:716 [inlined]
│  [25] invokelatest
│     @ ./essentials.jl:714 [inlined]
│  [26] run_main_repl(interactive::Bool, quiet::Bool, banner::Bool, history_file::Bool, color_set::Bool)
│     @ Base ./client.jl:379
│  [27] exec_options(opts::Base.JLOptions)
│     @ Base ./client.jl:309
│  [28] _start()
│     @ Base ./client.jl:495
└ @ AzSessions ~/.julia/dev/AzSessions/src/AzSessions.jl:21
vtjnash commented 2 years ago

Do you have local changes to AzSessions? I tried your example above with my own account, and just got back generic "invalid_client" messages due to the lack of a client_secret, so it seemed that this would not be able to get a token with that authentication flow.

samtkaplan commented 2 years ago

@vtjnash I do not have any local changes to AzSessions other than changing the Project.toml file to allow for a newer version of MbedTLS. With respect to the error that you are seeing, I'm not sure. The page here does not seem to indicate that the client_secret is needed. Perhaps there is some setting in the Azure application that needs to be changed. I can look into it, and see if I can find an answer.

edit I think you might have to add a "Mobile and desktop applications" section to your application settings. See the docs here. I think you can do this on an existing application by going to the portal and selecting "App registrations" , "yourApplicationName", and "Authentication". Then, click the "add a platform" button and select "mobile and desktop applications". You may also need to select "Access tokens" under the "Implict grant and hybrid flows". Sigh. Sorry this is so complicated :(

vtjnash commented 2 years ago

Okay, I found there is a checkbox for this (https://stackoverflow.com/questions/45609432/how-do-i-resolve-the-error-aadsts7000218-the-request-body-must-contain-the-foll)

It looks like this is an unhandled race condition in HTTP.jl with the ConnectionPool, but we can also reduce the duration of the race window so that it is harder to hit that bug, which helps to make this unlikely: https://github.com/JuliaWeb/HTTP.jl/pull/901

quinnj commented 2 years ago

@samtkaplan, can you confirm that upgrading HTTP.jl (v1.2.1) fixes the issue here? I realize this got auto-closed, but just wanted to follow up with you.

samtkaplan commented 2 years ago

@quinnj Thanks so much for checking in. I really appreciate all of the work. Unfortunately, it looks like there is still an issue. Perhaps, Interesting, the issue seems to be slightly different now in that it manifests more quickly than it did before. image

quinnj commented 2 years ago

Hmmm, yeah, so this error should be benign; i.e. your requests should keep working. This is more reporting that a connection we were thinking of reusing ended up getting closed.

@vtjnash, what do you think of changing montior_idle_connections in HTTP.ConnectionPools to:

function monitor_idle_connection(c::Connection)
    try
        if eof(c.io)                                  ;@debugv 3 "💀  Closed:     $c"
            close(c.io)
        end
    catch ex
        @try Base.IOError close(c.io)
        ex isa Base.IOError || rethrow()
    end
    nothing
end
vtjnash commented 2 years ago

That sounds good.

Additionally, another reliability feature I think is missing here is checking whether it is still in the right state before calling eof or close. If the object already is being reused or closed, it should not do anything here. A simple age counter that increments on every use, and passed as an argument here, would help do that. (And would allow implementing it more accurately to the comment, which says it would close if any data gets received)

samtkaplan commented 2 years ago

Hi,

I just wanted to give a small update from my side. With MbedTLS 1.1.3 and HTTP master, things are working great. I guess we can close this. Do you know when the next version of HTTP.jl will be released? I'm guessing that this commit that is not yet in a release was the final one required to close this issue (although as you noted that particular "error" message should be benign) . No rush, just curious.

Many thanks!!

quinnj commented 2 years ago

Just made a new HTTP.jl 1.3.0 release today that includes the commit. Thanks for the help tracking these issues down!