search

OxygenFramework / Oxygen.jl

💨 A breath of fresh air for programming web apps in Julia
https://oxygenframework.github.io/Oxygen.jl/
MIT License
383 stars 25 forks source link

NUL Byte on Request.body when processing post on Julia 1.11-beta-1 #186

Closed Westat-Transportation closed 2 months ago

Westat-Transportation commented 2 months ago

I tried to test an API we are developing on the latest Julia 1.11 beta-1 version and ran into an issue. What's happening is that the first byte that comes through in the body of post requests is 0x00. Here's some sample code that sets up the server:

using Oxygen
using HTTP
using StructTypes

struct Login
    user_name::String
end

@post "/login" function(req::HTTP.Request)
    @info req.body
    login_data = json(req, Login)
    @info login_data
    return "hello world!"
end

# start the web server
serve()

And here's the output on the REPL when we make a post request to this server for the /login endpoint:

➜ C:\Users\simas_m\AppData\Local\Programs\Julia-1.11.0-beta1\bin\julia.exe --project=.
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.11.0-beta1 (2024-04-10)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> include("src/API.jl")
   ____
  / __ \_  ____  ______ ____  ____
 / / / / |/_/ / / / __ `/ _ \/ __ \
/ /_/ />  </ /_/ / /_/ /  __/ / / /
\____/_/|_|\__, /\__, /\___/_/ /_/
          /____//____/

[ Info: 📦 Version 1.5.5 (2024-04-11)
[ Info: ✅ Started server: http://127.0.0.1:8080
[ Info: 📖 Documentation: http://127.0.0.1:8080/docs
[ Info: 📊 Metrics: http://127.0.0.1:8080/docs/metrics
[ Info: Listening on: 127.0.0.1:8080, thread id: 1
[ Info: UInt8[0x00, 0x22, 0x75, 0x73, 0x65, 0x72, 0x5f, 0x6e, 0x61, 0x6d, 0x65, 0x22, 0x3a, 0x22, 0x74, 0x65, 0x73, 0x74, 0x22, 0x7d]
┌ Error: ERROR:
│   exception =
│    ArgumentError: embedded NULs are not allowed in C strings: "\0\"user_name\":\"test\"}"
│    Stacktrace:
│      [1] unsafe_convert
│        @ .\strings\cstring.jl:85 [inlined]
│      [2] stat(path::String)
│        @ Base.Filesystem .\stat.jl:174
│      [3] isfile
│        @ .\stat.jl:494 [inlined]
│      [4] read_json_str(json::String)
│        @ JSON3 C:\Users\simas_m\.julia\packages\JSON3\jSAdy\src\utils.jl:240
│      [5] _prepare_read
│        @ C:\Users\simas_m\.julia\packages\JSON3\jSAdy\src\structs.jl:17 [inlined]
│      [6] read(str::String, ::Type{Login}; jsonlines::Bool, kw::@Kwargs{})
│        @ JSON3 C:\Users\simas_m\.julia\packages\JSON3\jSAdy\src\structs.jl:34
│      [7] read
│        @ C:\Users\simas_m\.julia\packages\JSON3\jSAdy\src\structs.jl:33 [inlined]
│      [8] read
│        @ C:\Users\simas_m\.julia\packages\JSON3\jSAdy\src\structs.jl:13 [inlined]
│      [9] json(req::Request, classtype::Type{Login}; kwargs::@Kwargs{})
│        @ Oxygen.Core.Util C:\Users\simas_m\.julia\packages\Oxygen\SbPrA\src\utilities\bodyparsers.jl:58
│     [10] json
│        @ C:\Users\simas_m\.julia\packages\Oxygen\SbPrA\src\utilities\bodyparsers.jl:56 [inlined]
│     [11] (::var"#7#8")(req::Request)
│        @ Main .\REPL[27]:3
│     [12] #6
│        @ C:\Users\simas_m\.julia\packages\Oxygen\SbPrA\src\handlers.jl:40 [inlined]
│     [13] #14#15
│        @ C:\Users\simas_m\.julia\packages\Oxygen\SbPrA\src\handlers.jl:54 [inlined]
│     [14] #14
│        @ C:\Users\simas_m\.julia\packages\Oxygen\SbPrA\src\handlers.jl:53 [inlined]
│     [15] (::Oxygen.Core.var"#54#58"{var"#7#8", Oxygen.Core.Handlers.var"#14#16"{Oxygen.Core.Handlers.var"#14#15#17"{Oxygen.Core.Handlers.var"#6#12"}}})(req::Request)
│        @ Oxygen.Core C:\Users\simas_m\.julia\packages\Oxygen\SbPrA\src\core.jl:541
│     [16] (::HTTP.Handlers.Router{typeof(HTTP.Handlers.default404), typeof(HTTP.Handlers.default405), Nothing})(req::Request)
│        @ HTTP.Handlers C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\Handlers.jl:439
│     [17] (::Oxygen.Core.var"#35#38"{Request, HTTP.Handlers.Router{typeof(HTTP.Handlers.default404), typeof(HTTP.Handlers.default405), Nothing}})()
│        @ Oxygen.Core C:\Users\simas_m\.julia\packages\Oxygen\SbPrA\src\core.jl:344
│     [18] handlerequest(getresponse::Oxygen.Core.var"#35#38"{Request, HTTP.Handlers.Router{typeof(HTTP.Handlers.default404), typeof(HTTP.Handlers.default405), Nothing}}, catch_errors::Bool; show_errors::Bool)
│        @ Oxygen.Core.Util C:\Users\simas_m\.julia\packages\Oxygen\SbPrA\src\utilities\misc.jl:37
│     [19] handlerequest
│        @ C:\Users\simas_m\.julia\packages\Oxygen\SbPrA\src\utilities\misc.jl:32 [inlined]
│     [20] #34
│        @ C:\Users\simas_m\.julia\packages\Oxygen\SbPrA\src\core.jl:343 [inlined]
│     [21] #41
│        @ C:\Users\simas_m\.julia\packages\Oxygen\SbPrA\src\core.jl:358 [inlined]
│     [22] handlerequest(getresponse::Oxygen.Core.var"#41#45"{Request, Oxygen.Core.var"#34#37"{HTTP.Handlers.Router{typeof(HTTP.Handlers.default404), typeof(HTTP.Handlers.default405), Nothing}, Bool, Bool}, Oxygen.Core.AppContext.Service}, catch_errors::Bool; show_errors::Bool)
│        @ Oxygen.Core.Util C:\Users\simas_m\.julia\packages\Oxygen\SbPrA\src\utilities\misc.jl:37
│     [23] handlerequest
│        @ C:\Users\simas_m\.julia\packages\Oxygen\SbPrA\src\utilities\misc.jl:32 [inlined]
│     [24] #40
│        @ C:\Users\simas_m\.julia\packages\Oxygen\SbPrA\src\core.jl:355 [inlined]
│     [25] (::Oxygen.Core.var"#29#31"{Oxygen.Core.var"#40#44"{Oxygen.Core.var"#34#37"{HTTP.Handlers.Router{typeof(HTTP.Handlers.default404), typeof(HTTP.Handlers.default405), Nothing}, Bool, Bool}, Oxygen.Core.AppContext.Service, Bool}, HTTP.Handlers.Router{typeof(HTTP.Handlers.default404), typeof(HTTP.Handlers.default405), Nothing}, String})(req::Request)
│        @ Oxygen.Core C:\Users\simas_m\.julia\packages\Oxygen\SbPrA\src\core.jl:328
│     [26] #12
│        @ C:\Users\simas_m\.julia\packages\Oxygen\SbPrA\src\core.jl:188 [inlined]
│     [27] (::HTTP.Handlers.var"#1#2"{Oxygen.Core.var"#12#14"{Oxygen.Core.var"#29#31"{Oxygen.Core.var"#40#44"{Oxygen.Core.var"#34#37"{HTTP.Handlers.Router{typeof(HTTP.Handlers.default404), typeof(HTTP.Handlers.default405), Nothing}, Bool, Bool}, Oxygen.Core.AppContext.Service, Bool}, HTTP.Handlers.Router{typeof(HTTP.Handlers.default404), typeof(HTTP.Handlers.default405), Nothing}, String}, Sockets.IPv4, HTTP.Streams.Stream{Request, HTTP.Connections.Connection{Sockets.TCPSocket}}}})(stream::HTTP.Streams.Stream{Request, HTTP.Connections.Connection{Sockets.TCPSocket}})
│        @ HTTP.Handlers C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\Handlers.jl:58
│     [28] (::Oxygen.Core.var"#15#16"{Oxygen.Core.var"#29#31"{Oxygen.Core.var"#40#44"{Oxygen.Core.var"#34#37"{HTTP.Handlers.Router{typeof(HTTP.Handlers.default404), typeof(HTTP.Handlers.default405), Nothing}, Bool, Bool}, Oxygen.Core.AppContext.Service, Bool}, HTTP.Handlers.Router{typeof(HTTP.Handlers.default404), typeof(HTTP.Handlers.default405), Nothing}, String}})(stream::HTTP.Streams.Stream{Request, HTTP.Connections.Connection{Sockets.TCPSocket}})
│        @ Oxygen.Core C:\Users\simas_m\.julia\packages\Oxygen\SbPrA\src\core.jl:205
│     [29] #invokelatest#2
│        @ .\essentials.jl:1030 [inlined]
│     [30] invokelatest
│        @ .\essentials.jl:1027 [inlined]
│     [31] handle_connection(f::Function, c::HTTP.Connections.Connection{Sockets.TCPSocket}, listener::HTTP.Servers.Listener{Nothing, Sockets.TCPServer}, readtimeout::Int64, access_log::Function)
│        @ HTTP.Servers C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\Servers.jl:469
│     [32] (::HTTP.Servers.var"#16#17"{Oxygen.Core.var"#15#16"{Oxygen.Core.var"#29#31"{Oxygen.Core.var"#40#44"{Oxygen.Core.var"#34#37"{HTTP.Handlers.Router{typeof(HTTP.Handlers.default404), typeof(HTTP.Handlers.default405), Nothing}, Bool, Bool}, Oxygen.Core.AppContext.Service, Bool}, HTTP.Handlers.Router{typeof(HTTP.Handlers.default404), typeof(HTTP.Handlers.default405), Nothing}, String}}, HTTP.Servers.Listener{Nothing, Sockets.TCPServer}, Set{HTTP.Connections.Connection}, Int64, Oxygen.Core.var"#25#26", ReentrantLock, Base.Semaphore, HTTP.Connections.Connection{Sockets.TCPSocket}})()
│        @ HTTP.Servers C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\Servers.jl:401
â”” @ Oxygen.Core.Util C:\Users\simas_m\.julia\packages\Oxygen\SbPrA\src\utilities\misc.jl:40
Westat-Transportation commented 2 months ago

I think this may be an issue with HTTP.jl. I tried running the example on their website (https://juliaweb.github.io/HTTP.jl/stable/examples/#Simple-Server) and got a similar error:

➜ C:\Users\simas_m\AppData\Local\Programs\Julia-1.11.0-beta1\bin\julia.exe --project=.
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.11.0-beta1 (2024-04-10)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> include("src/HTTP.jl")
[ Info: Listening on: 127.0.0.1:8080, thread id: 1
[ Info: UInt8[0x00, 0x22, 0x69, 0x64, 0x22, 0x3a, 0x32, 0x2c, 0x22, 0x75, 0x73, 0x65, 0x72, 0x49, 0x64, 0x22, 0x3a, 0x22, 0x30, 0x30, 0x30, 0x30, 0x37, 0x66, 0x66, 0x63, 0x2d, 0x33, 0x38, 0x38, 0x33, 0x2d, 0x31, 0x61, 0x37, 0x30, 0x2d, 0x30, 0x30, 0x30, 0x30, 0x2d, 0x30, 0x32, 0x32, 0x32, 0x30, 0x33, 0x61, 0x63, 0x64, 0x32, 0x35, 0x30, 0x22, 0x2c, 0x22, 0x74, 0x79, 0x70, 0x65, 0x22, 0x3a, 0x22, 0x63, 0x61, 0x74, 0x22, 0x2c, 0x22, 0x6e, 0x61, 0x6d, 0x65, 0x22, 0x3a, 0x22, 0x70, 0x65, 0x74, 0x65, 0x22, 0x7d]
┌ Error: handle_connection handler error.
│
│ ===========================
│ HTTP Error message:
│
│ ERROR: ArgumentError: invalid JSON at byte position 1 while parsing type Animal: ExpectedOpeningObjectChar
│ �"id":2,"userId":"00007ffc
│
│ Stacktrace:
│   [1] invalid(error::JSON3.Error, buf::Vector{UInt8}, pos::Int64, T::Type)
│     @ JSON3 C:\Users\simas_m\.julia\packages\JSON3\jSAdy\src\JSON3.jl:30
│   [2] #read!#48
│     @ C:\Users\simas_m\.julia\packages\JSON3\jSAdy\src\structs.jl:541 [inlined]
│   [3] read!
│     @ C:\Users\simas_m\.julia\packages\JSON3\jSAdy\src\structs.jl:463 [inlined]
│   [4] #read#46
│     @ C:\Users\simas_m\.julia\packages\JSON3\jSAdy\src\structs.jl:455 [inlined]
│   [5] read
│     @ C:\Users\simas_m\.julia\packages\JSON3\jSAdy\src\structs.jl:453 [inlined]
│   [6] read(str::JSON3.VectorString{Vector{UInt8}}, ::Type{Animal}; jsonlines::Bool, kw::@Kwargs{})
│     @ JSON3 C:\Users\simas_m\.julia\packages\JSON3\jSAdy\src\structs.jl:41
│   [7] read
│     @ C:\Users\simas_m\.julia\packages\JSON3\jSAdy\src\structs.jl:33 [inlined]
│   [8] read(bytes::Vector{UInt8}, ::Type{Animal})
│     @ JSON3 C:\Users\simas_m\.julia\packages\JSON3\jSAdy\src\structs.jl:14
│   [9] createAnimal(req::HTTP.Messages.Request)
│     @ Main C:\Users\simas_m\Git\oxygen_mvp\src\HTTP.jl:26
│  [10] (::HTTP.Handlers.Router{typeof(HTTP.Handlers.default404), typeof(HTTP.Handlers.default405), Nothing})(req::HTTP.Messages.Request)
│     @ HTTP.Handlers C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\Handlers.jl:439
│  [11] (::HTTP.Handlers.var"#1#2"{HTTP.Handlers.Router{typeof(HTTP.Handlers.default404), typeof(HTTP.Handlers.default405), Nothing}})(stream::HTTP.Streams.Stream{HTTP.Messages.Request, HTTP.Connections.Connection{TCPSocket}})
│     @ HTTP.Handlers C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\Handlers.jl:58
│  [12] #invokelatest#2
│     @ .\essentials.jl:1030 [inlined]
│  [13] invokelatest
│     @ .\essentials.jl:1027 [inlined]
│  [14] handle_connection(f::Function, c::HTTP.Connections.Connection{TCPSocket}, listener::HTTP.Servers.Listener{Nothing, Sockets.TCPServer}, readtimeout::Int64, access_log::Nothing)
│     @ HTTP.Servers C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\Servers.jl:469
│  [15] (::HTTP.Servers.var"#16#17"{HTTP.Handlers.var"#1#2"{HTTP.Handlers.Router{typeof(HTTP.Handlers.default404), typeof(HTTP.Handlers.default405), Nothing}}, HTTP.Servers.Listener{Nothing, Sockets.TCPServer}, Set{HTTP.Connections.Connection}, Int64, Nothing, ReentrantLock, Base.Semaphore, HTTP.Connections.Connection{TCPSocket}})()
│     @ HTTP.Servers C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\Servers.jl:401
â”” @ HTTP.Servers C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\Servers.jl:483
ERROR: LoadError: HTTP.RequestError:
HTTP.Request:
HTTP.Messages.Request:
"""
POST /api/zoo/v1/animals HTTP/1.1
Host: localhost:8080
Accept: */*
User-Agent: HTTP.jl/1.11.0-beta1
Content-Length: 83
Accept-Encoding: gzip

{"id":2,"userId":"00007ffc-3883-1a70-0000-022203acd250","type":"cat","name":"pete"}"""Underlying error:
BoundsError: attempt to access 0-element Memory{UInt8} at index [67:71]
Stacktrace:
  [1] (::HTTP.ConnectionRequest.var"#connections#4"{…})(req::HTTP.Messages.Request; proxy::Nothing, socket_type::Type, socket_type_tls::Type, readtimeout::Int64, connect_timeout::Int64, logerrors::Bool, logtag::Nothing, kw::@Kwargs{…})
    @ HTTP.ConnectionRequest C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\ConnectionRequest.jl:140
  [2] (::Base.var"#104#106"{Base.var"#104#105#107"{…}})(args::HTTP.Messages.Request; kwargs::@Kwargs{iofunction::Nothing, decompress::Nothing, verbose::Int64})
    @ Base .\error.jl:298
  [3] (::HTTP.RetryRequest.var"#manageretries#3"{…})(req::HTTP.Messages.Request; retry::Bool, retries::Int64, retry_delays::ExponentialBackOff, retry_check::Function, retry_non_idempotent::Bool, kw::@Kwargs{…})
    @ HTTP.RetryRequest C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\RetryRequest.jl:75
  [4] manageretries
    @ C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\RetryRequest.jl:30 [inlined]
  [5] (::HTTP.CookieRequest.var"#managecookies#4"{…})(req::HTTP.Messages.Request; cookies::Bool, cookiejar::HTTP.Cookies.CookieJar, kw::@Kwargs{…})
    @ HTTP.CookieRequest C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\CookieRequest.jl:42
  [6] managecookies
    @ C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\CookieRequest.jl:19 [inlined]
  [7] (::HTTP.HeadersRequest.var"#defaultheaders#2"{…})(req::HTTP.Messages.Request; iofunction::Nothing, decompress::Nothing, basicauth::Bool, detect_content_type::Bool, canonicalize_headers::Bool, kw::@Kwargs{…})
    @ HTTP.HeadersRequest C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\HeadersRequest.jl:71
  [8] defaultheaders
    @ C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\HeadersRequest.jl:14 [inlined]
  [9] (::HTTP.RedirectRequest.var"#redirects#3"{…})(req::HTTP.Messages.Request; redirect::Bool, redirect_limit::Int64, redirect_method::Nothing, forwardheaders::Bool, response_stream::Nothing, kw::@Kwargs{…})
    @ HTTP.RedirectRequest C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\RedirectRequest.jl:25
 [10] redirects
    @ C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\RedirectRequest.jl:14 [inlined]
 [11] (::HTTP.MessageRequest.var"#makerequest#3"{…})(method::String, url::URIs.URI, headers::Vector{…}, body::String; copyheaders::Bool, response_stream::Nothing, http_version::HTTP.Strings.HTTPVersion, verbose::Int64, kw::@Kwargs{})
    @ HTTP.MessageRequest C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\MessageRequest.jl:35
 [12] makerequest
    @ C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\MessageRequest.jl:24 [inlined]
 [13] request(stack::HTTP.MessageRequest.var"#makerequest#3"{…}, method::String, url::String, h::Vector{…}, b::String, q::Nothing; headers::Vector{…}, body::String, query::Nothing, kw::@Kwargs{})
    @ HTTP C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\HTTP.jl:457
 [14] request(stack::Function, method::String, url::String, h::Vector{Any}, b::String, q::Nothing)
    @ HTTP C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\HTTP.jl:455
 [15] #request#20
    @ C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\HTTP.jl:315 [inlined]
 [16] request(method::String, url::String, h::Vector{Any}, b::String)
    @ HTTP C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\HTTP.jl:313
 [17] post(::String, ::Vararg{Any}; kw::@Kwargs{})
    @ HTTP C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\HTTP.jl:532
 [18] top-level scope
    @ C:\Users\simas_m\Git\oxygen_mvp\src\HTTP.jl:65
 [19] include(fname::String)
    @ Main .\sysimg.jl:38
 [20] top-level scope
    @ REPL[1]:1
in expression starting at C:\Users\simas_m\Git\oxygen_mvp\src\HTTP.jl:65

caused by: TaskFailedException

    nested task error: BoundsError: attempt to access 0-element Memory{UInt8} at index [67:71]
    Stacktrace:
      [1] throw_boundserror(A::Memory{UInt8}, I::Tuple{UnitRange{Int64}})
        @ Base .\essentials.jl:14
      [2] checkbounds
        @ .\abstractarray.jl:699 [inlined]
      [3] view
        @ .\genericmemory.jl:309 [inlined]
      [4] readuntil
        @ C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\IOExtras.jl:118 [inlined]
      [5] readuntil(c::HTTP.Connections.Connection{TCPSocket}, f::typeof(HTTP.Parsers.find_end_of_chunk_size), sizehint::Int64)
        @ HTTP.Connections C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\Connections.jl:242
      [6] readuntil
        @ C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\Connections.jl:238 [inlined]
      [7] readchunksize(io::HTTP.Connections.Connection{TCPSocket}, message::HTTP.Messages.Response)
        @ HTTP.Messages C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\Messages.jl:558
      [8] ntoread
        @ C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\Streams.jl:202 [inlined]
      [9] readavailable(http::HTTP.Streams.Stream{HTTP.Messages.Response, HTTP.Connections.Connection{TCPSocket}}, n::Int64)
        @ HTTP.Streams C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\Streams.jl:225
     [10] readavailable
        @ C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\Streams.jl:225 [inlined]
     [11] macro expansion
        @ C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\Streams.jl:372 [inlined]
     [12] macro expansion
        @ C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\Exceptions.jl:19 [inlined]
     [13] closeread(http::HTTP.Streams.Stream{HTTP.Messages.Response, HTTP.Connections.Connection{TCPSocket}})
        @ HTTP.Streams C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\Streams.jl:371
     [14] macro expansion
        @ C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\StreamRequest.jl:61 [inlined]
     [15] (::HTTP.StreamRequest.var"#3#5"{…})()
        @ HTTP.StreamRequest C:\Users\simas_m\.julia\packages\ConcurrentUtilities\J6iMP\src\ConcurrentUtilities.jl:9

    caused by: BoundsError: attempt to access 0-element Memory{UInt8} at index [67:71]
    Stacktrace:
      [1] throw_boundserror(A::Memory{UInt8}, I::Tuple{UnitRange{Int64}})
        @ Base .\essentials.jl:14
      [2] checkbounds
        @ .\abstractarray.jl:699 [inlined]
      [3] view
        @ .\genericmemory.jl:309 [inlined]
      [4] readuntil
        @ C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\IOExtras.jl:118 [inlined]
      [5] readuntil(c::HTTP.Connections.Connection{TCPSocket}, f::typeof(HTTP.Parsers.find_end_of_chunk_size), sizehint::Int64)
        @ HTTP.Connections C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\Connections.jl:242
      [6] readuntil
        @ C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\Connections.jl:238 [inlined]
      [7] readchunksize(io::HTTP.Connections.Connection{TCPSocket}, message::HTTP.Messages.Response)
        @ HTTP.Messages C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\Messages.jl:558
      [8] ntoread
        @ C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\Streams.jl:202 [inlined]
      [9] readall!(http::HTTP.Streams.Stream{HTTP.Messages.Response, HTTP.Connections.Connection{TCPSocket}}, buf::IOBuffer)
        @ HTTP.Streams C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\Streams.jl:303
     [10] read (repeats 2 times)
        @ C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\Streams.jl:297 [inlined]
     [11] macro expansion
        @ C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\StreamRequest.jl:183 [inlined]
     [12] macro expansion
        @ .\lock.jl:273 [inlined]
     [13] readbody!(stream::HTTP.Streams.Stream{…}, res::HTTP.Messages.Response, buf_or_stream::HTTP.Streams.Stream{…}, lock::ReentrantLock)
        @ HTTP.StreamRequest C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\StreamRequest.jl:182
     [14] readbody(stream::HTTP.Streams.Stream{…}, res::HTTP.Messages.Response, decompress::Nothing, lock::ReentrantLock)
        @ HTTP.StreamRequest C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\StreamRequest.jl:142
     [15] macro expansion
        @ C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\StreamRequest.jl:54 [inlined]
     [16] (::HTTP.StreamRequest.var"#3#5"{…})()
        @ HTTP.StreamRequest C:\Users\simas_m\.julia\packages\ConcurrentUtilities\J6iMP\src\ConcurrentUtilities.jl:9
Stacktrace:
  [1] sync_end(c::Channel{Any})
    @ Base .\task.jl:459
  [2] macro expansion
    @ .\task.jl:492 [inlined]
  [3] streamlayer(stream::HTTP.Streams.Stream{…}; iofunction::Nothing, decompress::Nothing, logerrors::Bool, logtag::Nothing, timedout::Nothing, kw::@Kwargs{…})
    @ HTTP.StreamRequest C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\StreamRequest.jl:35
  [4] streamlayer
    @ C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\StreamRequest.jl:21 [inlined]
  [5] (::HTTP.ExceptionRequest.var"#exceptions#2"{…})(stream::HTTP.Streams.Stream{…}; status_exception::Bool, timedout::Nothing, logerrors::Bool, logtag::Nothing, kw::@Kwargs{…})
    @ HTTP.ExceptionRequest C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\ExceptionRequest.jl:14
  [6] exceptions
    @ C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\ExceptionRequest.jl:13 [inlined]
  [7] (::HTTP.TimeoutRequest.var"#timeouts#3"{…})(stream::HTTP.Streams.Stream{…}; readtimeout::Int64, logerrors::Bool, logtag::Nothing, kw::@Kwargs{…})
    @ HTTP.TimeoutRequest C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\TimeoutRequest.jl:18
  [8] (::HTTP.ConnectionRequest.var"#connections#4"{…})(req::HTTP.Messages.Request; proxy::Nothing, socket_type::Type, socket_type_tls::Type, readtimeout::Int64, connect_timeout::Int64, logerrors::Bool, logtag::Nothing, kw::@Kwargs{…})
    @ HTTP.ConnectionRequest C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\ConnectionRequest.jl:119
  [9] (::Base.var"#104#106"{Base.var"#104#105#107"{…}})(args::HTTP.Messages.Request; kwargs::@Kwargs{iofunction::Nothing, decompress::Nothing, verbose::Int64})
    @ Base .\error.jl:298
 [10] (::HTTP.RetryRequest.var"#manageretries#3"{…})(req::HTTP.Messages.Request; retry::Bool, retries::Int64, retry_delays::ExponentialBackOff, retry_check::Function, retry_non_idempotent::Bool, kw::@Kwargs{…})
    @ HTTP.RetryRequest C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\RetryRequest.jl:75
 [11] manageretries
    @ C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\RetryRequest.jl:30 [inlined]
 [12] (::HTTP.CookieRequest.var"#managecookies#4"{…})(req::HTTP.Messages.Request; cookies::Bool, cookiejar::HTTP.Cookies.CookieJar, kw::@Kwargs{…})
    @ HTTP.CookieRequest C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\CookieRequest.jl:42
 [13] managecookies
    @ C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\CookieRequest.jl:19 [inlined]
 [14] (::HTTP.HeadersRequest.var"#defaultheaders#2"{…})(req::HTTP.Messages.Request; iofunction::Nothing, decompress::Nothing, basicauth::Bool, detect_content_type::Bool, canonicalize_headers::Bool, kw::@Kwargs{…})
    @ HTTP.HeadersRequest C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\HeadersRequest.jl:71
 [15] defaultheaders
    @ C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\HeadersRequest.jl:14 [inlined]
 [16] (::HTTP.RedirectRequest.var"#redirects#3"{…})(req::HTTP.Messages.Request; redirect::Bool, redirect_limit::Int64, redirect_method::Nothing, forwardheaders::Bool, response_stream::Nothing, kw::@Kwargs{…})
    @ HTTP.RedirectRequest C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\RedirectRequest.jl:25
 [17] redirects
    @ C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\RedirectRequest.jl:14 [inlined]
 [18] (::HTTP.MessageRequest.var"#makerequest#3"{…})(method::String, url::URIs.URI, headers::Vector{…}, body::String; copyheaders::Bool, response_stream::Nothing, http_version::HTTP.Strings.HTTPVersion, verbose::Int64, kw::@Kwargs{})
    @ HTTP.MessageRequest C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\MessageRequest.jl:35
 [19] makerequest
    @ C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\clientlayers\MessageRequest.jl:24 [inlined]
 [20] request(stack::HTTP.MessageRequest.var"#makerequest#3"{…}, method::String, url::String, h::Vector{…}, b::String, q::Nothing; headers::Vector{…}, body::String, query::Nothing, kw::@Kwargs{})
    @ HTTP C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\HTTP.jl:457
 [21] request(stack::Function, method::String, url::String, h::Vector{Any}, b::String, q::Nothing)
    @ HTTP C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\HTTP.jl:455
 [22] #request#20
    @ C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\HTTP.jl:315 [inlined]
 [23] request(method::String, url::String, h::Vector{Any}, b::String)
    @ HTTP C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\HTTP.jl:313
 [24] post(::String, ::Vararg{Any}; kw::@Kwargs{})
    @ HTTP C:\Users\simas_m\.julia\packages\HTTP\vnQzp\src\HTTP.jl:532
 [25] top-level scope
    @ C:\Users\simas_m\Git\oxygen_mvp\src\HTTP.jl:65
 [26] include(fname::String)
    @ Main .\sysimg.jl:38
 [27] top-level scope
    @ REPL[1]:1
Some type information was truncated. Use `show(err)` to see complete types.

Please note that I added the @info call to the method handling new animal creations:

# "service" functions to actually do the work
function createAnimal(req::HTTP.Request)
    @info req.body
    animal = JSON3.read(req.body, Animal)
    animal.id = getNextId()
    ANIMALS[animal.id] = animal
    return HTTP.Response(200, JSON3.write(animal))
end
Westat-Transportation commented 2 months ago

Yep, these two issues seem related:

ndortega commented 2 months ago

Hi @Westat-Transportation,

From a quick glance, this looks more like a json serialization issue from the JSON3.jl library which doesn't allow those embedded nulls in json strings.

I'd recommend taking a look at how you're sending data to see if it's accidently getting added to the body of the request. Something is probably getting added to your body when making the request. If there's nothing you change from your end, i'd recommend raising this issue on the JSON3.jl page with your examples.

If you want to get things working NOW, you could always create a middleware to filter out the "0x00" from the first place in all request bodies.

Westat-Transportation commented 2 months ago

The same requests work with Julia 1.10, the error only happens on 1.11.

The problem happens before we call JSON3, if you look at the output of the @info statement you can see that the contents of the body are corrupt. So, I don't think this is a de-serialization issue.

Whatever the problem is here, it's replacing the first character with "0x00", so middleware would not be correct as we would have no way of knowing what that first character should have been...

ndortega commented 2 months ago

Hi @Westat-Transportation,

Sorry about that, you're absolutely right. I skimmed over that detail when I first read it. Like you mentioned, this appears to be a HTTP.jl issue which will need to be patched from their end

Westat-Transportation commented 2 months ago

Yep, I agree and cross tagged issues from HTTP.jl to make them more aware. Thanks for responding.