Open ehildenb opened 3 years ago
What I would like to do is discuss how to automate this more. The frontend can continue to be handled by pyk
, but I'd like to avoid having to mangle the definition to make this whole process go through.
The biggest changes I have to make to the definition are:
smtlib
and simplification
attribute.smt-lemma
and simplification
attribute.The backend has to make the following changes:
unknown
To simplify the process, I would propose adding a way to send all function definitions as smt-lemma
, without having to manuallly annotate each one. The backend can hide it's new behavior behind a flag. Then pyk (and addition to --emit-json
) can take care of the rest for a prototype.
The branch pyk-prove-lemmas
on KEVM now has all the infrastructure in place for doing this, with just the submodule pegged version of K there. It does not succeed very often, because of the "turning off Z3 unknown
errors" issue in the haskell backend.
We will wait and see what other needs for this arise (other than KEVM) to have a better idea of what is needed.
Motivation
When doing proofs, it's necessary to write a (usually large) body of lemmas for simplifying out functional terms. These lemmas are fed to the backend as axioms, without checking their validity. This increases our trust base (sometimes dramatically) for formal verification engagements).
If we can mechanically prove these lemmas are consistent with the definition (and the other lemmas), then our trust base gets smaller and we can have more confidence when reviewing PRs for verification efforts that the lemmas supplied are sound.
Existing Approach
@ttuegel has helped me develop a way to automatically discharge these lemmas using the haskell backend's ability to encode lemmas to SMT. The procedure is as follows:
smt-lemma
andsimplification
attributes to any rules in the original definition you want to be able to use to discharge the lemmas. Kompile the new definition.smt-lemma
attribute to all simplification lemmas used for proving (usually in a module namedVERIFICATION
).kprove --emit-json --dry-run --def-module VERIFICATION
on a dummy proof module, to produce the filedef-module.json
.pyk-prove-lemmas.sh
, supplying the lemma module names you're interested in and what k files should be required. This generates a single file which contains all the claims needed to feed into Haskell backend.Example Run
Build modified KEVM:
Dummy proof to generate the required
def-module.json
file:Generate the proof obligations (in this case, we are trying to prove every lemma in
tests/specs/lemmas.k
sound):This produces a file
tests/specs/functional/lemmas-verification-spec.k
with 59VERIFICATION-i
modules and 59CLAIM-i
modules. TheVERIFICATION-i
modules contain all the lemmas fromLEMMAS
andLEMMAS-HASKELL
except thei
th lemma, and theCLAIM-i
module contains thei
th lemma.So we prove each
CLAIM-i
module in sequence:This results in:
smt-lemma
attributes in the original semantics, or something else).