zhezhouzz / underapproximation_type

MIT License
5 stars 2 forks source link

Confusing Subtying Query #36

Closed Pat-Lafon closed 6 months ago

Pat-Lafon commented 6 months ago

I discussed this with you on Tuesday. I wanted to post the queries here to get your input and clarify if there is an issue on my end or if your improved typechecker better handles this.

As a reminder, I have inferred path conditions of the form:

[v:int list | (∃ b, (∃ b!26, (∃ _x10!0, (∃ _x9!0, (∃ a!23, (∃ b!27, (∃ x!16, (∃ b1!8, (∃ a!24, (∃ b!28, (∃ b!29, ((b <=> (size!0 == 0)) ∧ b))))))))))))]
[v:int list | (∃ b, (∃ b!38, (∃ b!39, (∃ x!17, (∃ b1!10, (∃ a!30, (∃ b!40, (∃ _x12!1, (∃ _x11!1, (∃ a!31, (∃ b!41, ((b <=> (size!0 == 0)) ∧ ((¬ b) ∧ b1!10)))))))))))))]
[v:int list | (∃ b, (∃ b!50, (∃ b!51, (∃ x!18, (∃ b1!12, (∃ a!37, (∃ b!52, (∃ b!53, (∃ _x14!1, (∃ _x13!1, (∃ a!38, ((b <=> (size!0 == 0)) ∧ ((¬ b) ∧ (¬ b1!12))))))))))))))]

For the sizedlist example. Most of the existentials there are unused and I believe could be safely simplified away.

I have a target type of [v:int list | ((∀ u, ((len v u) => ((0 <= u) ∧ (u <= size!0)))))] as provided in the specification with (0 <= size!0) in my context.

The path conditions are negated using the provided negate function to give versions like:

(∀ b, (∀ b!50, (∀ b!51, (∀ x!18, (∀ b1!12, (∀ a!37, (∀ b!52, (∀ b!53, (∀ _x14!1, (∀ _x13!1, (∀ a!38, ((b <=> (¬ (size!0 == 0))) ∨ (b ∨ b1!12)))))))))))))

I am testing the subtyping checks when I conjunct this negated path condition onto the target type. The goal is to use this to limit the input space that the final coverage needs to consider(rule out certain branches). The following is the outputted query using the meta-config.json show queries, the result of the subtyping check(true or false), and the coverage type I was using in relation to the target type.

Query:

((0 <= size!0) ∧ ((∀ u, ((len v u) => ((0 <= u) ∧ (u <= size!0)))) ∧ (∀ b, (∀ b1!12, ((b <=> (¬ (0 == size!0))) ∨ b ∨ b1!12))))) =>
(∀ u, ((len v u) => ((0 <= u) ∧ (u <= size!0))))
to_Z3: 0.0001s
false : 
[v:int list | ((∀ u, ((len v u) => ((0 <= u) ∧ (u <= size!0)))) ∧ (∀ b, (∀ b!50, (∀ b!51, (∀ x!18, (∀ b1!12, (∀ a!37, (∀ b!52, (∀ b!53, (∀ _x14!1, (∀ _x13!1, (∀ a!38, ((b <=> (¬ (size!0 == 0))) ∨ (b ∨ b1!12))))))))))))))]
Query:

((0 <= size!0) ∧ (∀ u, ((len v u) => ((0 <= u) ∧ (u <= size!0))))) =>
((∀ u, ((len v u) => ((0 <= u) ∧ (u <= size!0)))) ∧ (∀ b, (∀ b1!12, ((b <=> (¬ (0 == size!0))) ∨ b ∨ b1!12))))
to_Z3: 0.0001s
false : 
[v:int list | ((∀ u, ((len v u) => ((0 <= u) ∧ (u <= size!0)))) ∧ (∀ b, (∀ b!50, (∀ b!51, (∀ x!18, (∀ b1!12, (∀ a!37, (∀ b!52, (∀ b!53, (∀ _x14!1, (∀ _x13!1, (∀ a!38, ((b <=> (¬ (size!0 == 0))) ∨ (b ∨ b1!12))))))))))))))]

Not that it is false in both directions. I have checked that this is not the result of a timeout in either direction. I have also checked that this has errored in some other fashion but I could be wrong. At the very least, I am looking to see whether you can replicate this on your end. Most of my other subtyping checks that I run work as expected, even using some of the other path constraints... this one just seems unusual.

zhezhouzz commented 6 months ago

It seems that there is free variable b1!12 in your showed path condition. If there is a free variable, it means the SMT solver can assign any assignment to it and can easily find a "counterexample" which fails subtyping check. I am not sure if that it the reason, however, you can try to use functions has prefix fv (e.g., Languagez.fv_prop function lives in file syntax/prop.ml in meta branch) to check if the property has free variable. I have add new interface ..._infer (see file typing/termsyn.ml in meta branch) which can partially infer the type of a function (you still need to provide a function refinement type, since the implementation may be a recursive function, however, the result would be the inferred type instead of the provided one). You can try it via:

dune exec -- bin/main.exe type-infer meta-config.json data/abduction/prog.ml

which will return

Type Infer:
∅
⊢ fun (s : int) ->
  (let ((x!0 : bool)) = (((s : in
...
     int list) : int list) : int list) : int list) ⇨ (s:{v:int | 0 <= v}) → [v:int list | (s == 0 ∧ (lenF v) == 0) ∨ (¬s == 0 ∧ (∃x!1. ((x!1 ∧ (lenF v) == 0) ∨ (¬x!1 ∧ (∃s!1. ((0 <= s!1 ∧ s!1 == (s - 1)) ∧ (∃x!3. (((s!1 < s ∧ s!1 >= 0) ∧ 0 <= (lenF x!3) ∧ (lenF x!3) <= s!1) ∧ (∃x!4. v == (consF x!4 x!3))))))))))]

that contains three types

(s:{v:int | 0 <= v}) → [v:int list | (s == 0 ∧ (lenF v) == 0)]
(s:{v:int | 0 <= v}) → [v:int list | (¬s == 0 ∧ (∃x!1. ((x!1 ∧ (lenF v) == 0)]
(s:{v:int | 0 <= v}) → [v:int list | ∃x!1. (¬x!1 ∧ (∃s!1. ((0 <= s!1 ∧ s!1 == (s - 1)) ∧ (∃x!3. (((s!1 < s ∧ s!1 >= 0) ∧ 0 <= (lenF x!3) ∧ (lenF x!3) <= s!1) ∧ (∃x!4. v == (consF x!4 x!3))))))))))]
Pat-Lafon commented 6 months ago

By free variable do you mean that it's otherwise unconstrained? It is introduced in the query as a universal quantifier.

If there is a free variable, it means the SMT solver can assign any assignment to it and can easily find a "counterexample" which fails subtyping check.

I understand this as a general principle but not with respect to the query I have included above. Because it is universally quantified, I necessarily want the smt solver to consider all possible assignments of which there are 2, true and false. In the following:

... ∧ (∀ b, (∀ b1!12, ((b <=> (¬ (0 == size!0))) ∨ b ∨ b1!12)))))

if b1!12 is true then this whole part becomes true, but because it's conjuncted on to the target type, this should then just simplify to target type implies target type. Alternatively, b1!12 can be false, in which case this simplifies into exactly what I am interested in, that being the target type conjuncted with the constraint that size!0 is not 0. Given that, the query should be true in one of the two directions. I will look into your inference function changes. It's not clear to me that there is much different in terms of the branch condition that is inferred. Both have existentials that are introduced and otherwise unconstrained.

zhezhouzz commented 6 months ago

In general, the SMT solver is trying to find satisfied model for the negation of a query, it means that it treats all unbound variable as existential quantified. For example, in order to check forall a. a > 3 ==> a > 0, it will check if there is an assignment of a such that exists a. a > 3 /\ not (a > 0). If there is a free boolean variable bbb, for example, it checks forall a. a > 3 ==> (a > 0 /\ bbb), it will check if there is an assignment of a and bbb such that exists bbb. exists a. a > 3 /\ not (a > 0 /\ bbb), in this case, it can find a solution a := 4; bbb := false. For your case, I don't exactly know what happens without your code. we can discuss on Monday.

Pat-Lafon commented 6 months ago

Closing this issue as resolved