epfl-lara / stainless

Verification framework and tool for higher-order Scala programs
https://epfl-lara.github.io/stainless/
Apache License 2.0
355 stars 51 forks source link

Soundness issue with non-strict positive types and impredicativity of Boolean (or of choose realizability) #167

Open jad-hamza opened 6 years ago

jad-hamza commented 6 years ago

Stainless is able to derive false in that example adapted from http://vilhelms.github.io/posts/why-must-inductive-types-be-strictly-positive/ (original source: Inductively Defined Types (COLOG-88), also discussed in [Coq-Club] Positivity and Elimination Principle: https://sympa.inria.fr/sympa/arc/coq-club/2012-01/msg00087.html)

I don't really understand the example, but the ingredients needed in Stainless are: non-strict positive types for A, and quantifiers/impredicativity of Boolean in p0

import stainless.lang._

object Strict {
  def exists[T](p: T => Boolean) = {
    !forall((t: T) => !p(t))
  }

  case class A(magic: (A => Boolean) => Boolean)

  // i(a) = \b. a == b
  def i[T](a: T): T => Boolean = (b: T) => a == b

  def f(p: A => Boolean): A = A(i(p))

  // notContained is the predicate used for the existential formula p0
  def notContained(x: A, p: A => Boolean) = f(p) == x && !p(x)

  // paradoxical p0 and x0; p0(x0) can be neither true nor false
  def p0(x: A): Boolean = exists((p: A => Boolean) => notContained(x,p))
  val x0: A = f(p0)

  // simple lemma about the predicate `notContained`
  def lemma(x: A, p: A => Boolean) = {
    require(!notContained(x,p) && f(p) == x)

    p(x)
  } holds

  // Stainless can prove lemma1 without the require, is that normal?

  // we can then prove that p0(x0) implies !p0(x0)
  def lemma1() = {
    require(p0(x0)) 
    !p0(x0)
  } holds

  // and vice versa
  def lemma2() = {
    require(!p0(x0))
    assert(forall((p: A => Boolean) => !notContained(x0,p)))
    assert(!notContained(x0,p0))
    assert(lemma(x0,p0))

    p0(x0)
  } holds

  def theorem() = {
    // since lemma1 is clear for Stainless, only lemma2 is required here
    assert( 
      if (p0(x0)) lemma1()
      else lemma2()
    )
    assert(false)
  }
}

PS: related to https://github.com/epfl-lara/stainless/issues/140

Edit: added an assert in theorem to prevent the if then else from being simplified away.

vkuncak commented 5 years ago

This should be resolved in #452 ?

jad-hamza commented 5 years ago

Yes, at the moment #452 rejects all non strictly positive types. This can be relaxed by allowing them and then forcing the user to use explicit indices when folding/unfolding an A (and only allow the generalization rules, without indices, for types that are strictly positive).

jad-hamza commented 5 years ago

Here is another example, which doesn't use non-strict positive types but uses forall and choose:

import stainless.lang._
import stainless.proof._

object ForallBool {
  def isEmpty[T]: Boolean = forall((x: T) => false)

  case class A() {
    require(isEmpty[A])
  }

  def isEmptyImpliesFalse(): Unit = {
    require(isEmpty[A])
    val a = A()
    check(false)
    ()
  } ensuring(_ => false)

  def nonEmptyImpliesFalse(): Unit = {
    require(!isEmpty[A])
    val a = choose((x: A) => true)
    assert(isEmpty[A])
    check(false)
    ()
  } ensuring(_ => false)

  def proveFalse() = {
    if (isEmpty[A]) isEmptyImpliesFalse()
    else nonEmptyImpliesFalse()
    assert(false)
  }
}
vkuncak commented 1 year ago

The above snippet from @jad-hamza currently crashes in error: Well-formedness check failed after extraction.

vkuncak commented 1 year ago

The fact that the above snipped failed well-formedness check seemed due to a type checker bug. The unsoundess with choose from false can instead be seen in this slight variation of the code above, with refactored choose:

import stainless.lang._
import stainless.proof._

object ForallBool {
  def isEmpty[T]: Boolean = forall((x: T) => false)

  case class A() {
    require(isEmpty[A])
  }

  def isEmptyImpliesFalse(): Unit = {
    require(isEmpty[A])
    val a = A()
    assert(false)
    ()
  } ensuring((res:Unit) => false)

  def getA: A = {
    choose((x: A) => true)
  }

  def nonEmptyImpliesFalse(): Unit = {
    require(!isEmpty[A])
    val a = getA
    assert(isEmpty[A])
    assert(false)
    ()
  }.ensuring((res:Unit) => false)

  def proveFalse(): Unit = {
    isEmptyImpliesFalse()
    nonEmptyImpliesFalse()
    assert(false)
    ()
  }
}

with Version: 0.9.7-23-g98626b4 giving:

Forall.scala:19:5: warning: `choose` expressions may be unsafe due to difficulty in checking their realizability automatically
               choose((x: A) => true)
               ^^^^^^^^^^^^^^^^^^^^^^
Starting verification...
Verified: 11 / 11

 stainless summary 

Forall.scala:7:14:    ARequireForDefault    class invariant                        valid from cache     0.1 
Forall.scala:19:5:    getA                  body assertion: Choose satisfiability  valid from cache     0.0 
                      isEmptyImpliesFalse   body assertion                         valid from cache     0.0 
Forall.scala:13:13:   isEmptyImpliesFalse   class invariant                        valid from cache     0.0 
Forall.scala:14:5:    isEmptyImpliesFalse   postcondition                          trivial              0.0 
                      nonEmptyImpliesFalse  body assertion                         valid from cache     0.0 
Forall.scala:25:5:    nonEmptyImpliesFalse  postcondition                          valid from cache     0.0 
Forall.scala:25:12:   nonEmptyImpliesFalse  body assertion                         valid from cache     0.0 
                      proveFalse            body assertion                         valid from cache     0.0 
Forall.scala:31:5:    proveFalse            precond. (call isEmptyImpliesFalse)    valid from cache     0.0 
Forall.scala:32:5:    proveFalse            precond. (call nonEmptyImpliesFalse)   valid from cache     0.0 
.............................................................................................................
total: 11   valid: 11   (10 from cache, 1 trivial) invalid: 0    unknown: 0    time:    0.08                
vkuncak commented 1 year ago

We should make sure this is reportedly severely enough, check if the internal typechecking bug is desired or not, and then see if we emit warnings for quantifiers.

vkuncak commented 1 year ago

Just to make it clear: since contravariant types are not allowed currently, the soundness issue only that our report for realizability check is not severe enough.

vkuncak commented 1 year ago

Just to make it clear: since contravariant types are not allowed currently, the soundness issue only that our report for realizability check is not severe enough.

mario-bucev commented 1 year ago

The above ForallBool and the tests in false-valid seem to be all related to missing a realizability check.