Open wenkokke opened 4 years ago
Ouch sorry about this! Is there some way you can point us to your repo so we can try to replicate? Specifically I’m wondering what the definition of the various VectorX aliases look like...
On Mon, Feb 3, 2020 at 8:39 AM Wen Kokke notifications@github.com wrote:
This is a bit of a frustrating bug report to write.
There is a problem with reflection that I've encountered several times, but it's unclear to me how to minimize my code to isolate the issue. I am writing a tool for verifying neural networks using Liquid Haskell. At the moment, my project https://github.com/wenkokke/leadbeater fails to compile with the following error:
:1:1-1:1: Error
elaborate PLE1 195 failed on:
... && Leadbeater.Network.norm == Leadbeater.LinearAlgebra.map lam lam_arg##1 : real . Leadbeater.Prelude.rdiv lam_arg##1 (Leadbeater.LinearAlgebra.sumPos xs) ...
with error
Unbound symbol xs --- perhaps you meant: fst ?
in environment
...
I've omitted a huge number of lines, basically containing the reflection of every single line of Haskell in my current codebase.
The problem seemingly lies with the following snippet https://github.com/wenkokke/leadbeater/blob/master/src/Leadbeater/Network.hs#L60-L63 :
{-@ reflect norm @-}
{-@ norm :: xs:VectorNE Rpos -> VectorX R xs @-} norm :: Vector R -> Vector R
norm xs = let s = sumPos xs in map (
rdiv
s) xsIt seems that Liquid Haskell somehow reflects the xs in the let binding as a literal xs. This problem remains if I rewrite the norm function to inline the definition for s, or to factor the lambda out into a top-level function:
{-@ reflect norm @-}
{-@ norm :: xs:VectorNE Rpos -> VectorX R xs @-} norm2 :: Vector R -> Vector R
norm2 xs = map (
rdiv
sumPos xs) xs{-@ reflect norm @-}
{-@ norm :: s:Rpos -> xs:VectorNE Rpos -> VectorX R xs @-} norm3' :: R -> Vector R -> Vector R
norm3' s xs = map (
rdiv
s) xs{-@ reflect norm3 @-}
{-@ norm3 :: xs:VectorNE Rpos -> VectorX R xs @-} norm3 :: Vector R -> Vector R
norm3 xs = norm3' (sumPos xs) xs
This is where things get weird. If I only build and verify my library, which includes the norm function, everything checks out. The error above only happens if I also verify my tests. The error itself is the failure to elaborate the following definition https://github.com/wenkokke/leadbeater/blob/master/test/perceptron.hs#L26-L37, which doesn't even use norm:
{-@ reflect example @-}
{-@ example :: NetworkN 2 1 @-} example :: Network
example = NLast
( Layer { bias = 0.184 , weights = (1 >< 2) [ 0.194 , 0.195 ] , activation = Sigmoid } )
(The only function in the above definition is >< https://github.com/wenkokke/leadbeater/blob/master/src/Leadbeater/LinearAlgebra.hs#L137-L140, the rest are all constructors and constants.)
Similarly, if I extract the code involved in defining norm, as I've done here https://gist.github.com/wenkokke/8e46debb9d6e8177e007f1e545110f47, everything checks just fine.
If you have any idea what may be causing this, please let me know, I'd be happy to investigate more and provide further evidence.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ucsd-progsys/liquidhaskell/issues/1601?email_source=notifications&email_token=AAMS4OEHIW7ZXM5MI62DS7LRBBCEXA5CNFSM4KPIKKC2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IKUWXXA, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMS4OGTQAAEHQQHXCJJAY3RBBCEXANCNFSM4KPIKKCQ .
There's a bunch of links into my repository in the bug report itself. The aliases for Vector
are defined here. I've been trying to follow this tutorial fairly closely.
Ah just saw the “my project” link will take a look ...
Btw I’m guessing you’ve tried renaming the “xs” in both the signature and code of norm?
On Mon, Feb 3, 2020 at 8:44 AM Ranjit Jhala jhala@cs.ucsd.edu wrote:
Ouch sorry about this! Is there some way you can point us to your repo so we can try to replicate? Specifically I’m wondering what the definition of the various VectorX aliases look like...
On Mon, Feb 3, 2020 at 8:39 AM Wen Kokke notifications@github.com wrote:
This is a bit of a frustrating bug report to write.
There is a problem with reflection that I've encountered several times, but it's unclear to me how to minimize my code to isolate the issue. I am writing a tool for verifying neural networks using Liquid Haskell. At the moment, my project https://github.com/wenkokke/leadbeater fails to compile with the following error:
:1:1-1:1: Error
elaborate PLE1 195 failed on:
... && Leadbeater.Network.norm == Leadbeater.LinearAlgebra.map lam lam_arg##1 : real . Leadbeater.Prelude.rdiv lam_arg##1 (Leadbeater.LinearAlgebra.sumPos xs) ...
with error
Unbound symbol xs --- perhaps you meant: fst ?
in environment
...
I've omitted a huge number of lines, basically containing the reflection of every single line of Haskell in my current codebase.
The problem seemingly lies with the following snippet https://github.com/wenkokke/leadbeater/blob/master/src/Leadbeater/Network.hs#L60-L63 :
{-@ reflect norm @-}
{-@ norm :: xs:VectorNE Rpos -> VectorX R xs @-} norm :: Vector R -> Vector R
norm xs = let s = sumPos xs in map (
rdiv
s) xsIt seems that Liquid Haskell somehow reflects the xs in the let binding as a literal xs. This problem remains if I rewrite the norm function to inline the definition for s, or to factor the lambda out into a top-level function:
{-@ reflect norm @-}
{-@ norm :: xs:VectorNE Rpos -> VectorX R xs @-} norm2 :: Vector R -> Vector R
norm2 xs = map (
rdiv
sumPos xs) xs{-@ reflect norm @-}
{-@ norm :: s:Rpos -> xs:VectorNE Rpos -> VectorX R xs @-} norm3' :: R -> Vector R -> Vector R
norm3' s xs = map (
rdiv
s) xs{-@ reflect norm3 @-}
{-@ norm3 :: xs:VectorNE Rpos -> VectorX R xs @-} norm3 :: Vector R -> Vector R
norm3 xs = norm3' (sumPos xs) xs
This is where things get weird. If I only build and verify my library, which includes the norm function, everything checks out. The error above only happens if I also verify my tests. The error itself is the failure to elaborate the following definition https://github.com/wenkokke/leadbeater/blob/master/test/perceptron.hs#L26-L37, which doesn't even use norm:
{-@ reflect example @-}
{-@ example :: NetworkN 2 1 @-} example :: Network
example = NLast
( Layer { bias = 0.184 , weights = (1 >< 2) [ 0.194 , 0.195 ] , activation = Sigmoid } )
(The only function in the above definition is >< https://github.com/wenkokke/leadbeater/blob/master/src/Leadbeater/LinearAlgebra.hs#L137-L140, the rest are all constructors and constants.)
Similarly, if I extract the code involved in defining norm, as I've done here https://gist.github.com/wenkokke/8e46debb9d6e8177e007f1e545110f47, everything checks just fine.
If you have any idea what may be causing this, please let me know, I'd be happy to investigate more and provide further evidence.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ucsd-progsys/liquidhaskell/issues/1601?email_source=notifications&email_token=AAMS4OEHIW7ZXM5MI62DS7LRBBCEXA5CNFSM4KPIKKC2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IKUWXXA, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMS4OGTQAAEHQQHXCJJAY3RBBCEXANCNFSM4KPIKKCQ .
Btw I’m guessing you’ve tried renaming the “xs” in both the signature and code of norm?
I hadn't tried that, but I just have. It seems the variable xs
from the type signature is ending up in the reflection somehow:
{-@ reflect norm @-}
{-@ norm :: bar:VectorNE Rpos -> VectorX R bar @-}
norm :: Vector R -> Vector R
norm foo = let s = sumPos foo in map (`rdiv` s) foo
:1:1-1:1: Error
elaborate PLE1 195 failed on:
...
&& Leadbeater.Network.norm == Leadbeater.LinearAlgebra.map lam lam_arg##1 : real . Leadbeater.Prelude.rdiv lam_arg##1 (Leadbeater.LinearAlgebra.sumPos bar)
...
with error
Unbound symbol bar --- perhaps you meant: bvor ?
in environment
...
If I comment out the type signature, the error message changes to:
:1:1-1:1: Error
elaborate PLE1 195 failed on:
...
&& Leadbeater.Network.norm == Leadbeater.LinearAlgebra.map lam lam_arg##1 : real . Leadbeater.Prelude.rdiv lam_arg##1 (Leadbeater.LinearAlgebra.sumPos lq1)
...
with error
Unbound symbol lq1 --- perhaps you meant: len ?
in environment
...
Which looks like some Liquid Haskell internals to me?
Yes, I worry this is some gross PLE related bug...
oh yikes looking again, I see there are "lam" in the refinements?
In general, the interaction of PLE and lambdas in refinements is very suspect and I'd advise against using lambdas in refinements at all...
Even more puzzling, I thought the lam
comes from the partially
applied rdiv
in norm
but the following works fine:
{-@ LIQUID "--reflection" @-}
{-@ LIQUID "--ple" @-}
{-@ reflect mmap @-}
mmap :: (a -> b) -> [a] -> [b]
mmap _ [] = []
mmap f (x:xs) = f x : mmap f xs
{-@ reflect add @-}
add :: Int -> Int -> Int
add x y = x + y
{-@ reflect bumpOK @-}
bumpOK :: [Int] -> [Int]
bumpOK xs = mmap (add 1) xs
{-@ testOK :: _ -> {b:_ | b} @-}
testOK :: () -> Bool
testOK () = (bumpOK [0, 1]) == [1, 2]
Even more mysteriously if we use (
add1)
LH grumbles... I expect the core is different? Any idea @nikivazou ?
{-@ reflect bumpFail @-}
bumpFail :: [Int] -> [Int]
bumpFail xs = mmap (`add` 1) xs
{-@ testFail :: _ -> {b:_ | b} @-}
testFail :: () -> Bool
testFail () = (bumpFail [0, 1]) == [1, 2]
OTOH, @wenkokke -- can you see if you have any luck replacing the (
rdivs)
with (rdiv' s)
defined below?
{-@ reflect rdiv' @-}
rdiv' s x = rdiv x s
clearly undesirable and will be fixed, but may provide some progress in the short term?
@wenkokke can you see if you have any luck replacing the
(rdiv s)
with(rdiv' s)
defined below?{-@ reflect rdiv' @-} rdiv' s x = rdiv x s
clearly undesirable and will be fixed, but may provide some progress in the short term?
I've tried replacing rdiv
with rdiv'
, both defined as above and using (/)
directly. Unfortunately, neither solves the problem.
Hmm can you remind me how you’re running LH to reproduce this? Will try to take a look today. Thanks!
On Tue, Feb 4, 2020 at 6:31 AM Wen Kokke notifications@github.com wrote:
@wenkokke https://github.com/wenkokke -- can you see if you have any luck replacing the (rdiv s) with (rdiv' s) defined below?
{-@ reflect rdiv' @-} rdiv' s x = rdiv x s
clearly undesirable and will be fixed, but may provide some progress in the short term?
I've tried replacing rdiv with rdiv', defined as you mentioned, and defined using (/) directly. Unfortunately, this does not solve the problem.
— You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/ucsd-progsys/liquidhaskell/issues/1601?email_source=notifications&email_token=AAMS4OBWXARQBQWCIJGNVADRBF327A5CNFSM4KPIKKC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKX2LWQ#issuecomment-581936602, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMS4OH63TVJDYICO5KO3FLRBF327ANCNFSM4KPIKKCQ .
I'm running make test
, which runs:
stack exec -- liquid -isrc test/*.hs
stack test --flag leadbeater:liquidhaskell
It already fails on the first command, so my cabal setup isn't super relevant.
Hi @wenkokke -- when I do a fresh clone of your repo and run make test
things seem to work out fine (see attached screenshot) -- maybe you've edited it to sidestep the issue? Could you point me to a commit that shows the problem? Thanks!
Ah, darn, I'm sorry. My master branch has the offending reflect pragmas commented out, since I'm continuing work on that.
I've created the issue-1601
branch for you, with a version which reproduces the reported error. Thanks so much for looking into this!
Thanks @wenkokke !
Hmm.. @nikivazou -- I think there is something fishy going on here that may have to do with your recent changes for function equality? Specifically, given the code:
{-@ reflect norm @-}
{-@ norm :: bar:VectorNE Rpos -> VectorX R bar @-}
norm :: Vector R -> Vector R
norm foo = map (plus (sumPos foo)) foo
PLE generates the odd equality:
&& Leadbeater.Network.norm == Leadbeater.LinearAlgebra.map (Leadbeater.Prelude.plus (Leadbeater.LinearAlgebra.sumPos bar))
where indeed bar
is out of scope. The above looks like some malformed version of the term:
Leadbeater.Network.norm bar == Leadbeater.LinearAlgebra.map (Leadbeater.Prelude.plus (Leadbeater.LinearAlgebra.sumPos bar)) bar
but where you have somehow eta-contracted (is that a word?) by removing the bar
(which is the name for the parameter in the reflected function.)
Does this sound like it may have to do with your recent fun-extensionality related edits?
Indeed, looking at the .fq I see:
define Leadbeater.Network.norm (bar : (Leadbeater.LinearAlgebra.Vector real)) = (&& [((Leadbeater.Network.norm bar) = (Leadbeater.LinearAlgebra.map (Leadbeater.Prelude.plus (Leadbea
ter.LinearAlgebra.sumPos bar)) bar))])
So I'm wondering where this mysterious Leadbeater.Network.norm == Leadbeater.LinearAlgebra.map (Leadbeater.Prelude.plus (Leadbeater.LinearAlgebra.sumPos bar))
term is coming from ...???
Aha. @nikivazou -- it is most certainly related to that because when I comment this line out
@wenkokke's code works fine...
I am reverting your change right now, to get Wen unstuck I think you may want to use their example to debug your eta-elim?
Hi @wenkokke -- I made a branch
https://github.com/ucsd-progsys/liquidhaskell/tree/T1601
where your particular problem appears solved; will merge into develop
soon pending CI and perhaps some deprecating of tests that depend on the suppressed feature...
Can you LMK if it works?
Best!
Hi @wenkokke -- this should be fixed in develop
please LMK if not!
@nikivazou -- see the above bug in the elimEta
the diff in the above PR #1602 should be pretty explanatory?
Thanks!
I've just tried, and everything works now! Thank you so much for all your effort!
Actually, elimEta on PLE is not mine ;)
@yiyunliu if you have time, can you take a look? In short, the elimEta of fixpoint crashes the code above. It is not urgent since it is flagged now, but if you can take a look at this before me, it would be great!
Let me open the issue again, in hope of providing a proper fix to elimEta.
I can see what's going on here. elimEta takes the function:
norm2 xs = map (`rdiv` sumPos xs) xs
then naively turns it into:
norm2 = map (`rdiv` sumPos xs)
I'll add a check to see if the binder is free so it won't mistakenly get rid of the binder when it's needed.
oh thanks!
It's is possible to build this repo with the latest LH by
git clone https://github.com/wenkokke/lazuli.gitgit clone -b issue-1601
cabal exec ghc -- -ilazuli/src -package liquidhaskell lazuli/src/Leadbeater/Network.hs -fplugin=LiquidHaskell
I'm still getting strange failures when reflecting.
lazuli/src/Leadbeater/LinearAlgebra.hs:210:13: error:
Illegal type specification for `Leadbeater.LinearAlgebra.mXv`
Leadbeater.LinearAlgebra.mXv :: xss:(Leadbeater.LinearAlgebra.Matrix GHC.Types.Double) -> ys:{VV##0 : (Leadbeater.LinearAlgebra.Vector GHC.Types.Double) | Leadbeater.LinearAlgebra.size VV##0 == Leadbeater.LinearAlgebra.cols xss} -> {VV##0 : (Leadbeater.LinearAlgebra.Vector GHC.Types.Double) | VV##0 == Leadbeater.LinearAlgebra.flatten (xss < Leadbeater.LinearAlgebra.asColumn ys)
&& VV##0 == Leadbeater.LinearAlgebra.mXv xss ys
&& Leadbeater.LinearAlgebra.size VV##0 == Leadbeater.LinearAlgebra.rows xss}
Sort Error in Refinement: {VV##0 : (Leadbeater.LinearAlgebra.Vector real) | VV##0 == Leadbeater.LinearAlgebra.flatten (xss < Leadbeater.LinearAlgebra.asColumn ys)
&& VV##0 == Leadbeater.LinearAlgebra.mXv xss ys
&& Leadbeater.LinearAlgebra.size VV##0 == Leadbeater.LinearAlgebra.rows xss}
Cannot unify (Leadbeater.LinearAlgebra.Matrix @(11572)) with bool in expression: Leadbeater.LinearAlgebra.flatten (xss < Leadbeater.LinearAlgebra.asColumn ys)
Just reflect
|
210 | {-@ reflect mXv @-}
| ^^^^
lazuli/src/Leadbeater/LinearAlgebra.hs:220:13: error:
Illegal type specification for `Leadbeater.LinearAlgebra.outer`
Leadbeater.LinearAlgebra.outer :: xs:(Leadbeater.LinearAlgebra.Vector GHC.Types.Double) -> ys:(Leadbeater.LinearAlgebra.Vector GHC.Types.Double) -> {VV##0 : (Leadbeater.LinearAlgebra.Matrix GHC.Types.Double) | VV##0 == Leadbeater.LinearAlgebra.outer xs ys
&& VV##0 == (Leadbeater.LinearAlgebra.asColumn xs < Leadbeater.LinearAlgebra.asRow ys)
&& Leadbeater.LinearAlgebra.cols VV##0 == Leadbeater.LinearAlgebra.size ys
&& Leadbeater.LinearAlgebra.rows VV##0 == Leadbeater.LinearAlgebra.size xs}
Sort Error in Refinement: {VV##0 : (Leadbeater.LinearAlgebra.Matrix real) | VV##0 == Leadbeater.LinearAlgebra.outer xs ys
&& VV##0 == (Leadbeater.LinearAlgebra.asColumn xs < Leadbeater.LinearAlgebra.asRow ys)
&& Leadbeater.LinearAlgebra.cols VV##0 == Leadbeater.LinearAlgebra.size ys
&& Leadbeater.LinearAlgebra.rows VV##0 == Leadbeater.LinearAlgebra.size xs}
Cannot unify (Leadbeater.LinearAlgebra.Matrix real) with bool in expression: VV##0 == (Leadbeater.LinearAlgebra.asColumn xs < Leadbeater.LinearAlgebra.asRow ys)
because
Invalid Relation VV##0 == (Leadbeater.LinearAlgebra.asColumn xs < Leadbeater.LinearAlgebra.asRow ys) with operand types (Leadbeater.LinearAlgebra.Matrix real) and bool
Just reflect
|
220 | {-@ reflect outer @-}
| ^^^^^^
This is a bit of a frustrating bug report to write.
There is a problem with reflection that I've encountered several times, but it's unclear to me how to minimize my code to isolate the issue. I am writing a tool for verifying neural networks using Liquid Haskell. At the moment, my project fails to compile with the following error:
I've omitted a huge number of lines, basically containing the reflection of every single line of Haskell in my current codebase.
The problem seemingly lies with the following snippet:
It seems that Liquid Haskell somehow reflects the
xs
in the let binding as a literalxs
. This problem remains if I rewrite thenorm
function to inline the definition fors
, or to factor the lambda out into a top-level function:This is where things get weird. If I only compile and verify the library, which includes the
norm
function, everything checks just fine. The error above only happens if I also compile and verify my test. The error itself is the failure to elaborate the following definition, which doesn't even usenorm
:(The only function in the above definition is
><
, the rest are all constructors and constants.)Similarly, if I extract the code involved in defining
norm
, as I've done here, everything checks just fine...If you have any idea what may be causing this, please let me know, I'd be happy to investigate more and provide further evidence.