search

TuringLang / Bijectors.jl

Implementation of normalising flows and constrained random variable transformations
https://turinglang.org/Bijectors.jl/
MIT License
200 stars 33 forks source link

Unit tests for PlanarLayer failed in Julia 1.5 #124

Closed yiyuezhuo closed 4 years ago

yiyuezhuo commented 4 years ago 
] test

raise:

PlanarLayer: Test Failed at /home/yiyuezhuo/.julia/packages/Bijectors/66bcj/test/norm_flows.jl:30
  Expression: ≈((inv(flow))(flow(z)), z, rtol = 0.2)
   Evaluated: [2.2075135426840555 -0.18182425891670018 … 1.3994120484164432 0.9248803865772497; -1.5719067963233282 -0.1836889984451685 … 0.7022959819078269 0.27282017311337303] ≈ [1.9716204287413468 -0.15707698104964826 … 1.3606109850208474 0.4787377009574211; -1.3961992590989196 -0.20212227601885754 … 0.7311973736313007 0.6051343777576946] (rtol=0.2)
Stacktrace:
 [1] top-level scope at /home/yiyuezhuo/.julia/packages/Bijectors/66bcj/test/norm_flows.jl:30
 [2] top-level scope at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Test/src/Test.jl:1115
 [3] top-level scope at /home/yiyuezhuo/.julia/packages/Bijectors/66bcj/test/norm_flows.jl:23
PlanarLayer: Test Failed at /home/yiyuezhuo/.julia/packages/Bijectors/66bcj/test/norm_flows.jl:31
  Expression: ≈((inv(flow) ∘ flow)(z), z, rtol = 0.2)
   Evaluated: [2.2075135426840555 -0.18182425891670018 … 1.3994120484164432 0.9248803865772497; -1.5719067963233282 -0.1836889984451685 … 0.7022959819078269 0.27282017311337303] ≈ [1.9716204287413468 -0.15707698104964826 … 1.3606109850208474 0.4787377009574211; -1.3961992590989196 -0.20212227601885754 … 0.7311973736313007 0.6051343777576946] (rtol=0.2)
Stacktrace:
 [1] top-level scope at /home/yiyuezhuo/.julia/packages/Bijectors/66bcj/test/norm_flows.jl:31
 [2] top-level scope at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Test/src/Test.jl:1115
 [3] top-level scope at /home/yiyuezhuo/.julia/packages/Bijectors/66bcj/test/norm_flows.jl:23
Test Summary: | Pass  Fail  Total
PlanarLayer   |   11     2     13

Strangely, this test passed in Julia 1.4.1.

I guess it should be attributed to seed (seed!(1) before tests), since even in Julia 1.4.1, which passed the test, will raise similar errors when run corresponding code multiple times or change seed (ex: seed!(3)). Is it valid to specify a new seed to make this test pass? But this will lead to subtle version compatible problems.

torfjelde commented 4 years ago

So the issue with the inverse of PlanarLayer is that it requires an iterated algorithm to compute. As a result, it's not guaranteed to be 100% accurate. Not sure what the best approach to testing these is. Maybe just lowering the rtol requirement?