Tests fail on nightly build Julia

0x005c commented 5 years ago

Tests on v0.21.0 and master branch fail on nightly build Julia.

How to reproduce: Replace Pkg.add("JSON") with Pkg.add(Pkg.PackageSpec(url="https://github.com/JuliaIO/JSON.jl")) to test master branch

$ PATH="path/to/nightly-julia/bin:$PATH"
$ julia -e 'import Pkg; Pkg.add("JSON"); Pkg.test("JSON")'

Expected result: All tests pass

Current result:

  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
 Resolving package versions...
  Updating `~/.julia/environments/v1.4/Project.toml`
  [682c06a0] ~ JSON v0.21.0 #master (https://github.com/JuliaIO/JSON.jl) ⇒ v0.21.0
  Updating `~/.julia/environments/v1.4/Manifest.toml`
  [682c06a0] ~ JSON v0.21.0 #master (https://github.com/JuliaIO/JSON.jl) ⇒ v0.21.0
   Testing JSON
 Resolving package versions...
Status `/tmp/jl_czKTmK/Manifest.toml`
  [864edb3b] DataStructures v0.17.3
  [53c48c17] FixedPointNumbers v0.6.1
  [682c06a0] JSON v0.21.0
  [6fe1bfb0] OffsetArrays v0.11.1
  [bac558e1] OrderedCollections v1.1.0
  [69de0a69] Parsers v0.3.7
  [2a0f44e3] Base64  [`@stdlib/Base64`]
  [ade2ca70] Dates  [`@stdlib/Dates`]
  [8ba89e20] Distributed  [`@stdlib/Distributed`]
  [b77e0a4c] InteractiveUtils  [`@stdlib/InteractiveUtils`]
  [56ddb016] Logging  [`@stdlib/Logging`]
  [d6f4376e] Markdown  [`@stdlib/Markdown`]
  [a63ad114] Mmap  [`@stdlib/Mmap`]
  [de0858da] Printf  [`@stdlib/Printf`]
  [9a3f8284] Random  [`@stdlib/Random`]
  [9e88b42a] Serialization  [`@stdlib/Serialization`]
  [6462fe0b] Sockets  [`@stdlib/Sockets`]
  [8dfed614] Test  [`@stdlib/Test`]
  [4ec0a83e] Unicode  [`@stdlib/Unicode`]
Test Summary: | Pass  Total
Parser        |   86     86
Test Summary: | Pass  Total
Serializer    |   48     48
Test Summary: | Pass  Total
Integration   |  108    108
for issue #163: Error During Test at /home/myusername/.julia/packages/JSON/d89fA/test/regression/issue163.jl:1
  Test threw exception
  Expression: Float32(JSON.parse(json(2.1f-8))) == 2.1f-8
  Expected end of input
  Line: 0
  Around: ...2.1f-8...
                ^

  Stacktrace:
   [1] error(::String) at ./error.jl:33
   [2] _error(::String, ::JSON.Parser.MemoryParserState) at /home/myusername/.julia/packages/JSON/d89fA/src/Parser.jl:142
   [3] parse(::String; dicttype::Type, inttype::Type{Int64}, allownan::Bool, null::Nothing) at /home/myusername/.julia/packages/JSON/d89fA/src/Parser.jl:466
   [4] parse(::String) at /home/myusername/.julia/packages/JSON/d89fA/src/Parser.jl:461
   [5] top-level scope at /home/myusername/.julia/packages/JSON/d89fA/test/regression/issue163.jl:1
   [6] include(::String) at ./client.jl:447
   [7] top-level scope at /home/myusername/.julia/packages/JSON/d89fA/test/runtests.jl:84
   [8] top-level scope at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Test/src/Test.jl:1180
   [9] top-level scope at /home/myusername/.julia/packages/JSON/d89fA/test/runtests.jl:83
   [10] top-level scope at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Test/src/Test.jl:1107
   [11] top-level scope at /home/myusername/.julia/packages/JSON/d89fA/test/runtests.jl:83

Test Summary:    | Pass  Error  Total
Regression       |    7      1      8
  for issue #21  |    2             2
  for issue #26  |    1             1
  for issue #57  |    1             1
  for issue #109 |    1             1
  for issue #152 |    2             2
  for issue #163 |           1      1
ERROR: LoadError: Some tests did not pass: 7 passed, 0 failed, 1 errored, 0 broken.
in expression starting at /home/myusername/.julia/packages/JSON/d89fA/test/runtests.jl:82
ERROR: Package JSON errored during testing
Stacktrace:
 [1] pkgerror(::String, ::Vararg{String,N} where N) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/Types.jl:114
 [2] test(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}; coverage::Bool, test_fn::Nothing, julia_args::Cmd, test_args::Cmd) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/Operations.jl:1410
 [3] test(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}; coverage::Bool, test_fn::Nothing, julia_args::Cmd, test_args::Cmd, kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:284
 [4] test(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:271
 [5] #test#60 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:265 [inlined]
 [6] test at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:265 [inlined]
 [7] #test#59 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:264 [inlined]
 [8] test at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:264 [inlined]
 [9] test(::String; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:263
 [10] test(::String) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:263
 [11] top-level scope at none:1

Julia version: julia version 1.4.0-DEV (commit 1a6549a8708bd4a3e6425daa3f8cf04df955f1e2)

quinnj commented 5 years ago

This probably has to do w/ the switch in Base Julia to using the Ryu algorithm for float writing/printing, though I don't quite know the details of what JSON is doing under the hood here. But on nightly, json(2.1f-8) produces "2.1f-8", whereas Julia 1.2 has

julia> json(2.1f-8)
"2.1e-8"

0x005c commented 4 years ago

fixed by JuliaLang/julia#33909

JuliaIO / JSON.jl

Tests fail on nightly build Julia #294