Closed ehildenb closed 10 months ago
The following are the results from kevm prove tests/specs/erc20/ds/allowance-spec.k VERIFICATION --backend haskell --format-failures --concrete-rules-file tests/specs/erc20/ds/concrete-rules.txt --depth $DEPTH
for difference values of DEPTH
.
DEPTH=400
b"```@R`\x046\x10a\x01\x1dW`\x005|\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x90\x04c\xff\xff\xff\xff\x16\x80c\x06\xfd\xde\x03\x14a\x01\"W\x80c\x07\xdah\xf5\x14a\x01SW\x80c\t^\xa7\xb3\x14a\x01hW\x80c\x13\xaf@5\x14a\x01\xc2W\x80c\x18\x16\r\xdd\x14a\x01\xfbW\x80c#\xb8r\xdd\x14a\x02$W\x80c1<\xe5g\x14a\x02\x9dW\x80c4R\xf5\x1d\x14a\x02\xc6W\x80cZ\xc8\x01\xfe\x14a\x032W\x80ci\xd3\xe2\x0e\x14a\x03YW\x80cp\xa0\x821\x14a\x03\x8eW\x80cu\xf1+!\x14a\x03\xdbW\x80cz\x9e^K\x14a\x04\x08W\x80c\x84\x02\x18\x1f\x14a\x04AW\x80c\x8d\xa5\xcb[\x14a\x04\xadW\x80c\x90\xbc\x16\x93\x14a\x05\x02W\x80c\x95\xd8\x9bA\x14a\x057W\x80c\xa9\x05\x9c\xbb\x14a\x05hW\x80c\xbe\x9aeU\x14a\x05\xc2W\x80c\xbf~!O\x14a\x05\xd7W\x80c\xddb\xed>\x14a\x06,W[`\x00\x80\xfd[4\x15a\x01-W`\x00\x80\xfd[a\x015a\x06\x98V[`@Q\x80\x82`\x00\x19\x16`\x00\x19\x16\x81R` \x01\x91PP`@Q\x80\x91\x03\x90\xf3[4\x15a\x01^W`\x00\x80\xfd[a\x01fa\x06\x9eV[\x00[4\x15a\x01sW`\x00\x80\xfd[a\x01\xa8`\x04\x80\x805s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16\x90` \x01\x90\x91\x90\x805\x90` \x01\x90\x91\x90PPa\x07\x9eV[`@Q\x80\x82\x15\x15\x15\x15\x81R` \x01\x91PP`@Q\x80\x91\x03\x90\xf3[4\x15a\x01\xcdW`\x00\x80\xfd[a\x01\xf9`\x04\x80\x805s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16\x90` \x01\x90\x91\x90PPa\x08xV[\x00[4\x15a\x02\x06W`\x00\x80\xfd[a\x02\x0ea\tWV[`@Q\x80\x82\x81R` \x01\x91PP`@Q\x80\x91\x03\x90\xf3[4\x15a\x02/W`\x00\x80\xfd[a\x02\x83`\x04\x80\x805s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16\x90` \x01\x90\x91\x90\x805s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16\x90` \x01\x90\x91\x90\x805\x90` \x01\x90\x91\x90PPa\t`V[`@Q\x80\x82\x15\x15\x15\x15\x81R` \x01\x91PP`@Q\x80\x91\x03\x90\xf3[4\x15a\x02\xa8W`\x00\x80\xfd[a\x02\xb0a\n$_\xad\xbc\x00)"
DEPTH=600
b"```@R`\x046\x10a\x01\x1dW`\x005|\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x90\x04c\xff\xff\xff\xff\x16\x80c\x06\xfd\xde\x03\x14a\x01\"W\x80c\x07\xdah\xf5\x14a\x01SW\x80c\t^\xa7\xb3\x14a\x01hW\x80c\x13\xaf@5\x14a\x01\xc2W\x80c\x18\x16\r\xdd\x14a\x01\xfbW\x80c#\xb8r\xdd\x14a\x02$W\x80c1<\xe5g\x14a\x02\x9dW\x80c4R\xf5\x1d\x14a\x02\xc6W\x80cZ\xc8\x01\xfe\x14a\x032W\x80ci\xd3\xe2\x0e\x14a\x03YW\x80cp\xa0\x821\x14a\x03\x8eW\x80cu\xf1+!\x14a\x03\xdbW\x80cz\x9e^K\x14a\x04\x08W\x80c\x84\x02\x18\x1f\x14a\x04AW\x80c\x8d\xa5\xcb[\x14a\x04\xadW\x80c\x90\xbc\x16\x93\x14a\x05\x02W\x80c\x95\xd8\x9bA\x14a\x057W\x80c\xa9\x05\x9c\xbb\x14a\x05hW\x80c\xbe\x9aeU\x14a\x05\xc2W\x80c\xbf~!O\x14a\x05\xd7W\x80c\xddb\xed>\x14a\x06,W[`\x00\x80\xfd[4\x15a\x01-W`\x00\x80\xfd[a\x015a\x06\x98V[`@Q\x80\x82`\x00\x19\x16`\x00\x19\x16\x81R` \x01\x91PP`@Q\x80\x91\x03\x90\xf3[4\x15a\x01^W`\x00\x80\xfd[a\x01fa\x06\x9eV[\x00[4\x15a\x01sW`\x00\x80\xfd[a\x01\xa8`\x04\x80\x805s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16\x90` \x01\x90\x91\x90\x805\x90` \x01\x90\x91\x90PPa\x07\x9eV[`@Q\x80\x82\x15\x15\x15\x15\x81R` \x01\x91PP`@Q\x80\x91\x03\x90\xf3[4\x15a\x01\xcdW`\x00\x80\xfd[a\x01\xf9`\x04\x80\x805s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16\x90` \x01\x90\x91\x90PPa\x08xV[\x00[4\x15a\x02\x06W`\x00\x80\xfd[a\x02\x0ea\tWV[`@Q\x80\x82\x81R` \x01\x91PP`@Q\x80\x91\x03\x90\xf3[4\x15a\x02/W`\x00\x80\xfd[a\x02\x83`\x04\x80\x805s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16\x90` \x01\x90\x91\x90\x805s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16\x90` \x01\x90\x91\x90\x805\x90` \x01\x90\x91\x90PPa\t`V[`@Q\x80\x82\x15\x15\x15\x15\x81R` \x01\x91PP`@Q\x80\x91\x03\x90\xf3[4\x15a\x02\xa8W`\x00\x80\xfd[a\x02\xb0a\n$_\xad\xbc\x00)"
I also ran it with DEPTH=300,500,700, but for some reason those didn't produce any output.
The outputs after 400 and 600 steps are exactly the same. Perhaps something went wrong?
Let's run this in the kore-repl
, and see if we can even take 1 step, then print out the configuration. If it's the same as the initial configuration again, then it's a problem in the backend. If it's different, then we need to figure out what went wrong with this experiment ^^^ and re-run.
Something probably did go wrong on my end, but I'm not sure what. In the kore-repl, I found the usual branching from node 0, where one branch immediately halts and the other branch seems to continue computation normally. I'll step through the kore-repl and re-run the test without a timeout to see what happens.
Below is the last node that the proof reaches (node 639), before reaching the "end of proof".
b"```@R`\x046\x10a\x01\x1dW`\x005|\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x90\x04c\xff\xff\xff\xff\x16\x80c\x06\xfd\xde\x03\x14a\x01\"W\x80c\x07\xdah\xf5\x14a\x01SW\x80c\t^\xa7\xb3\x14a\x01hW\x80c\x13\xaf@5\x14a\x01\xc2W\x80c\x18\x16\r\xdd\x14a\x01\xfbW\x80c#\xb8r\xdd\x14a\x02$W\x80c1<\xe5g\x14a\x02\x9dW\x80c4R\xf5\x1d\x14a\x02\xc6W\x80cZ\xc8\x01\xfe\x14a\x032W\x80ci\xd3\xe2\x0e\x14a\x03YW\x80cp\xa0\x821\x14a\x03\x8eW\x80cu\xf1+!\x14a\x03\xdbW\x80cz\x9e^K\x14a\x04\x08W\x80c\x84\x02\x18\x1f\x14a\x04AW\x80c\x8d\xa5\xcb[\x14a\x04\xadW\x80c\x90\xbc\x16\x93\x14a\x05\x02W\x80c\x95\xd8\x9bA\x14a\x057W\x80c\xa9\x05\x9c\xbb\x14a\x05hW\x80c\xbe\x9aeU\x14a\x05\xc2W\x80c\xbf~!O\x14a\x05\xd7W\x80c\xddb\xed>\x14a\x06,W[`\x00\x80\xfd[4\x15a\x01-W`\x00\x80\xfd[a\x015a\x06\x98V[`@Q\x80\x82`\x00\x19\x16`\x00\x19\x16\x81R` \x01\x91PP`@Q\x80\x91\x03\x90\xf3[4\x15a\x01^W`\x00\x80\xfd[a\x01fa\x06\x9eV[\x00[4\x15a\x01sW`\x00\x80\xfd[a\x01\xa8`\x04\x80\x805s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16\x90` \x01\x90\x91\x90\x805\x90` \x01\x90\x91\x90PPa\x07\x9eV[`@Q\x80\x82\x15\x15\x15\x15\x81R` \x01\x91PP`@Q\x80\x91\x03\x90\xf3[4\x15a\x01\xcdW`\x00\x80\xfd[a\x01\xf9`\x04\x80\x805s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16\x90` \x01\x90\x91\x90PPa\x08xV[\x00[4\x15a\x02\x06W`\x00\x80\xfd[a\x02\x0ea\tWV[`@Q\x80\x82\x81R` \x01\x91PP`@Q\x80\x91\x03\x90\xf3[4\x15a\x02/W`\x00\x80\xfd[a\x02\x83`\x04\x80\x805s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16\x90` \x01\x90\x91\x90\x805s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16\x90` \x01\x90\x91\x90\x805\x90` \x01\x90\x91\x90PPa\t`V[`@Q\x80\x82\x15\x15\x15\x15\x81R` \x01\x91PP`@Q\x80\x91\x03\x90\xf3[4\x15a\x02\xa8W`\x00\x80\xfd[a\x02\xb0a\n$_\xad\xbc\x00)"
The <k>
cell is
<k> JUMP 1591 ~> #pc [ JUMPI ] ~> #execute ~> . </k>
I think there might be an issue with this rule on in evm.md
:
rule <k> JUMP DEST => #endBasicBlock... </k>
There is no space between #endBasicBlock
and ...
, so perhaps that's messing something up? I added a space there and am re-testing to see if anything new happens.
I think these two are duplicates:
{
false
#Equals
#hashedLocation ( "Solidity" , #hashedLocation ( "Solidity" , 2 , OWNER .IntList ) , SPENDER .IntList ) in_keys ( _39 )
}
#And
{
false
#Equals
keccak ( #buf ( 32 , SPENDER ) ++ #buf ( 32 , keccak ( #buf ( 32 , OWNER ) ++ b"\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02" ) ) ) in_keys ( _39 )
}
I would expect that this #hashedLocation
should expand out to the keccak(...)
in the second one, and it's not, so the Haskell backend is not able to de-dupe them. It should be able to. Make a claim runLemma(#hashedLocation(...)) => doneLemma(keccak(...))
to make sure it can prove tihs.
Also, inspect the JUMPI
that precedes this JUMP
, to see what the side-condition it's giving is.
I think there might be an issue with this rule on in
evm.md
:rule <k> JUMP DEST => #endBasicBlock... </k>
There is no space between
#endBasicBlock
and...
, so perhaps that's messing something up? I added a space there and am re-testing to see if anything new happens.
Against all odds, this seems to have fixed it. I'm not sure why, but I've just double checked and now it does work. Although the proof seems to take quite a while and would probably time out in CI.
This is quite strange to me. If this is causing failure I think it's a bug. I would like to investigate this.
Please make a checkout of master
branch, and do rm -rf .build ; make deps RELEASE=true ; make build -j8 RELEASE=true
. Then save the entire directory .build/usr/lib/kevm/haskell
as a tarball before.tar
. Then make your change above ^^^, do rm -rf .build ; make deps RELEASE=true ; make build -j8 RELEASE=true
, and again save the resulting .build/usr/lib/kevm/haskell
as after.tar
, and upload both to this issue.
Specifically, we're looking for differences between the files compiled.txt
and definition.kore
which show up in the resulting tarballs.
Huh, there doesn't appear to be any difference. So I must have done something wrong before, I will continue to investigate this.
So, I'm not sure why it didn't pass the first time I ran it, but now allowance-spec.k
does pass on the master branch actually! However it is very slow -- it takes 125 minutes.
I shall try to investigate what the bottleneck is in this proof, since its way too long.
I think this will be much easier once @JKTKops's --profile
option is added via #1060 (this seems to be coming soon). So, I am prioritizing other tasks for now. My impression is that many of the currently-failing Haskell tests are only failing because of a timeout (including this tests, erc20/ds/allowance-spec.k`). So, having profiles of individual steps will allow me to see exactly which rules are taking so long and then focus on optimizing those rules specifically. Without the tool, I'd have to painfully steps through hundreds of kore REPL steps to observe which steps take so long.
Actually, I will try using this as a use case for the --profile
-like tool
I was able to generate the log JSON using the profile workflow. The resulting log file is 113Mb. Here are what the results look like on speedscope: allowance-spec.k.log.timeorder.pdf
What's going on in those long simplify transitions?
The issue appears to be a slow interaction between #computeValidJumpDests
and #hashedLocations
, perhaps at the Kore level. So, an issue has been posted there with a simplified example: https://github.com/kframework/kore/issues/2761.
Now passes on master with global kore master install in ~13min due to solution to kframework/kore#2761.
Let's see how long this one takes to run, and see if there are unsimplified functions which we can work on partway through the execution.
You can run with
--depth
on several different arguments and save output to separate files and look for uninterpreted functions.