YosysHQ / sby

SymbiYosys (sby) -- Front-end for Yosys-based formal verification flows
Other
389 stars 73 forks source link

SBY produces different results for 'verific -formal' and 'read_verilog -formal' #117

Closed christian-krieg closed 2 years ago

christian-krieg commented 3 years ago

I have a simple counter implementation along with some simple properties on the counter value (counter.v):

module counter (clk, rst, mode, cnt);

    input  wire clk;
    input  wire rst;
    input  wire mode;
    output reg signed [9:0] cnt = -28;

    always @(posedge clk) begin
        if (rst)
            cnt <= -28;
        else begin
            if (mode) begin
                if (cnt != -162) begin
                    if (cnt <= 277) begin
                        cnt <= cnt + 5;
                    end
                end else if (cnt <= 272) begin
                    cnt <= cnt + 10;
                end
            // mode = 0
            end else begin
                if (cnt != -148) begin
                    if (cnt >= -334 ) begin
                        cnt <= cnt + -9;
                    end
                end else if (cnt >= -325) begin
                    cnt <= cnt + -18;
                end
            end
        end
    end

    `ifdef FORMAL
        reg init = 1;
        always @(posedge clk) begin
            if (init)    assume(rst);
            else assume(!rst);
            init <= 0;
        end

        always @(posedge clk) begin

            if (rst) begin
                assert (cnt == -28);
            end

            if (!rst) begin

                //
                // Check if counter value is never lower than MIN, larger than MAX, or
                // equal to INV
                //
                assert (cnt <= 282);
                assert (cnt >= -343);
                assert (cnt != -157);

                //
                // Check if the counter value is correctly incremented and decremented
                //

                // Counting up
                if ($past(mode)) begin
                    if (!$past(rst)) begin
                        if ($past(cnt) == -162) // INV - INC, counter jumped over INV
                            assert ((cnt - $past(cnt)) == 10);
                        else if ($past(cnt) <= 282 && $past(cnt) > 277) // cnt was near MAX
                            assert ((cnt - $past(cnt)) == 0);
                        else
                            assert ((cnt - $past(cnt)) == 5);
                    end
                end

                // Counting down
                if (!$past(mode)) begin
                    if (!$past(rst)) begin
                        if ($past(cnt) == -148) // INV + DEC, counter jumped over INV
                            assert (($past(cnt) - cnt) == 18);
                        else if ($past(cnt) >= -343 && $past(cnt) < -334) // cnt was near MIN
                            assert (($past(cnt) - cnt) == 0);
                        else
                            assert (($past(cnt) - cnt) == 9);
                    end
                end

            end
        end
    `endif
endmodule

The following SBY configuration file (vrf.sby) specifies two tasks yosys and verific, each using either the Yosys or the Verific Verilog frontend.

[tasks]
verific
yosys

[options]
mode prove
expect pass
depth 100

[engines]
smtbmc

[script]
yosys:   read_verilog -sv -formal counter.v
verific: verific -formal counter.v
verific: verific -import -all
prep -top counter

[files]
counter.v

A verification run in prove mode succeeds for the case where I read the design with Verific, but it fails when I read it with Yosys. So,

sby -f vrf.sby verific

succeeds, whereas

sby -f vrf.sby yosys

fails with the following report:

SBY 11:29:36 [vrf_yosys] Removing directory 'vrf_yosys'.
SBY 11:29:36 [vrf_yosys] Copy 'counter.v' to 'vrf_yosys/src/counter.v'.
SBY 11:29:36 [vrf_yosys] engine_0: smtbmc
SBY 11:29:36 [vrf_yosys] base: starting process "cd vrf_yosys/src; yosys -ql ../model/design.log ../model/design.ys"
SBY 11:29:36 [vrf_yosys] base: finished (returncode=0)
SBY 11:29:36 [vrf_yosys] smt2: starting process "cd vrf_yosys/model; yosys -ql design_smt2.log design_smt2.ys"
SBY 11:29:36 [vrf_yosys] smt2: finished (returncode=0)
SBY 11:29:36 [vrf_yosys] engine_0.basecase: starting process "cd vrf_yosys; yosys-smtbmc --presat --unroll --noprogress -t 100  --append 0 --dump-vcd engine_0/trace.vcd --dump-vlogtb engine_0/trace_tb.v --dump-smtc engine_0/trace.smtc model/design_smt2.smt2"
SBY 11:29:36 [vrf_yosys] engine_0.induction: starting process "cd vrf_yosys; yosys-smtbmc --presat --unroll -i --noprogress -t 100  --append 0 --dump-vcd engine_0/trace_induct.vcd --dump-vlogtb engine_0/trace_induct_tb.v --dump-smtc engine_0/trace_induct.smtc model/design_smt2.smt2"
SBY 11:29:36 [vrf_yosys] engine_0.basecase: ##   0:00:00  Solver: yices
SBY 11:29:36 [vrf_yosys] engine_0.induction: ##   0:00:00  Solver: yices
SBY 11:29:36 [vrf_yosys] engine_0.basecase: ##   0:00:00  Checking assumptions in step 0..
SBY 11:29:36 [vrf_yosys] engine_0.basecase: ##   0:00:00  Checking assertions in step 0..
SBY 11:29:36 [vrf_yosys] engine_0.induction: ##   0:00:00  Trying induction in step 100..
SBY 11:29:36 [vrf_yosys] engine_0.induction: ##   0:00:00  Trying induction in step 99..
SBY 11:29:36 [vrf_yosys] engine_0.basecase: ##   0:00:00  Checking assumptions in step 1..
SBY 11:29:36 [vrf_yosys] engine_0.basecase: ##   0:00:00  Checking assertions in step 1..
SBY 11:29:36 [vrf_yosys] engine_0.induction: ##   0:00:00  Trying induction in step 98..
SBY 11:29:36 [vrf_yosys] engine_0.basecase: ##   0:00:00  Checking assumptions in step 2..
SBY 11:29:36 [vrf_yosys] engine_0.basecase: ##   0:00:00  Checking assertions in step 2..
SBY 11:29:36 [vrf_yosys] engine_0.induction: ##   0:00:00  Trying induction in step 97..
SBY 11:29:36 [vrf_yosys] engine_0.induction: ##   0:00:00  Trying induction in step 96..
SBY 11:29:36 [vrf_yosys] engine_0.basecase: ##   0:00:00  Checking assumptions in step 3..
SBY 11:29:36 [vrf_yosys] engine_0.basecase: ##   0:00:00  Checking assertions in step 3..
SBY 11:29:36 [vrf_yosys] engine_0.induction: ##   0:00:00  Trying induction in step 95..
SBY 11:29:36 [vrf_yosys] engine_0.basecase: ##   0:00:00  Checking assumptions in step 4..
SBY 11:29:36 [vrf_yosys] engine_0.basecase: ##   0:00:00  Checking assertions in step 4..
SBY 11:29:36 [vrf_yosys] engine_0.induction: ##   0:00:00  Trying induction in step 94..
SBY 11:29:36 [vrf_yosys] engine_0.induction: ##   0:00:00  Trying induction in step 93..
SBY 11:29:36 [vrf_yosys] engine_0.basecase: ##   0:00:00  Checking assumptions in step 5..
SBY 11:29:36 [vrf_yosys] engine_0.basecase: ##   0:00:00  Checking assertions in step 5..
SBY 11:29:36 [vrf_yosys] engine_0.induction: ##   0:00:00  Trying induction in step 92..
SBY 11:29:36 [vrf_yosys] engine_0.induction: ##   0:00:00  Trying induction in step 91..
SBY 11:29:36 [vrf_yosys] engine_0.basecase: ##   0:00:00  Checking assumptions in step 6..
SBY 11:29:36 [vrf_yosys] engine_0.basecase: ##   0:00:00  Checking assertions in step 6..
SBY 11:29:36 [vrf_yosys] engine_0.induction: ##   0:00:00  Trying induction in step 90..
SBY 11:29:36 [vrf_yosys] engine_0.induction: ##   0:00:00  Trying induction in step 89..
SBY 11:29:36 [vrf_yosys] engine_0.basecase: ##   0:00:00  Checking assumptions in step 7..
SBY 11:29:36 [vrf_yosys] engine_0.basecase: ##   0:00:00  Checking assertions in step 7..
SBY 11:29:36 [vrf_yosys] engine_0.induction: ##   0:00:00  Trying induction in step 88..
SBY 11:29:36 [vrf_yosys] engine_0.induction: ##   0:00:00  Trying induction in step 87..
SBY 11:29:36 [vrf_yosys] engine_0.basecase: ##   0:00:00  Checking assumptions in step 8..
SBY 11:29:36 [vrf_yosys] engine_0.basecase: ##   0:00:00  Checking assertions in step 8..
SBY 11:29:36 [vrf_yosys] engine_0.induction: ##   0:00:00  Trying induction in step 86..
SBY 11:29:36 [vrf_yosys] engine_0.basecase: ##   0:00:00  BMC failed!
SBY 11:29:36 [vrf_yosys] engine_0.basecase: ##   0:00:00  Assert failed in counter: counter.v:68.29-69.61
SBY 11:29:36 [vrf_yosys] engine_0.basecase: ##   0:00:00  Writing trace to VCD file: engine_0/trace.vcd
SBY 11:29:36 [vrf_yosys] engine_0.basecase: ##   0:00:00  Writing trace to Verilog testbench: engine_0/trace_tb.v
SBY 11:29:36 [vrf_yosys] engine_0.basecase: ##   0:00:00  Writing trace to constraints file: engine_0/trace.smtc
SBY 11:29:36 [vrf_yosys] engine_0.induction: ##   0:00:00  Trying induction in step 85..
SBY 11:29:36 [vrf_yosys] engine_0.basecase: ##   0:00:00  Status: failed
SBY 11:29:36 [vrf_yosys] engine_0.basecase: finished (returncode=1)
SBY 11:29:36 [vrf_yosys] engine_0: Status returned by engine for basecase: FAIL
SBY 11:29:36 [vrf_yosys] engine_0.induction: terminating process
SBY 11:29:36 [vrf_yosys] summary: Elapsed clock time [H:MM:SS (secs)]: 0:00:00 (0)
SBY 11:29:36 [vrf_yosys] summary: Elapsed process time [H:MM:SS (secs)]: 0:00:00 (0)
SBY 11:29:36 [vrf_yosys] summary: engine_0 (smtbmc) returned FAIL for basecase
SBY 11:29:36 [vrf_yosys] summary: counterexample trace: vrf_yosys/engine_0/trace.vcd
SBY 11:29:36 [vrf_yosys] DONE (FAIL, rc=2)
SBY 11:29:36 One or more tasks produced a non-zero return code.

The failing asserttion is given here (counter.v:68.29-69.61):

assert ((cnt - $past(cnt)) == 5);

I cannot figure out what's the problem here. What am I missing?

ZipCPU commented 3 years ago

Looks like an overflow issue. For a temporary solution, assign a combinatorial register to (cnt - $past(cnt)):

reg [9:0] f_tmp;
always @(posedge clk)
begin
  f_tmp = 0;
  if (!rst) begin
    // ...
    f_tmp = cnt - $past(cnt); // Note the *blocking* assignment
    if ($past(mode))
    begin
      if (!$past(rst))
      begin
        if ($past(cnt) == -162)
          assert(f_tmp == 10);
        else if // ...

etc.

Dan

christian-krieg commented 3 years ago

Hi Dan,

Thanks for your workaround!

It turned out to be an issue with the constants, which are by default 32 bits wide. The Verific frontend trims these values to fit in the 10-bit wide cnt register, while the Yosys frontend doesn't. Explicitly specifying the number format as 10-bit wide signed decimal solves the issue.

Here is an updated version of counter.v:

module counter (clk, rst, mode, cnt);

    input  wire clk;
    input  wire rst;
    input  wire mode;
    output reg signed [9:0] cnt = -10'sd28;

    always @(posedge clk) begin
        if (rst)
            cnt <= -10'sd28;
        else begin
            if (mode) begin
                if (cnt != -10'sd162) begin
                    if (cnt <= 10'sd277) begin
                        cnt <= cnt + 10'sd5;
                    end
                end else if (cnt <= 10'sd272) begin
                    cnt <= cnt + 10'sd10;
                end
            // mode = 0
            end else begin
                if (cnt != -10'sd148) begin
                    if (cnt >= -10'sd334 ) begin
                        cnt <= cnt + -10'sd9;
                    end
                end else if (cnt >= -10'sd325) begin
                    cnt <= cnt + -10'sd18;
                end
            end
        end
    end

    `ifdef FORMAL
        reg init = 1;
        always @(posedge clk) begin
            if (init)    assume(rst);
            else assume(!rst);
            init <= 0;
        end

        always @(posedge clk) begin

            if (rst) begin
                assert (cnt == -10'sd28);
            end

            if (!rst) begin

                //
                // Check if counter value is never lower than MIN, larger than MAX, or
                // equal to INV
                //
                assert (cnt <= 10'sd282);
                assert (cnt >= -10'sd343);
                assert (cnt != -10'sd157);

                //
                // Check if the counter value is correctly incremented and decremented
                //

                // Counting up
                if ($past(mode)) begin
                    if (!$past(rst)) begin
                        if ($past(cnt) == -10'sd162) // INV - INC, counter jumped over INV
                            assert ((cnt - $past(cnt)) == 10'sd10);
                        else if ($past(cnt) <= 10'sd282 && $past(cnt) > 10'sd277) // cnt was near MAX
                            assert ((cnt - $past(cnt)) == 10'sd0);
                        else
                            assert ((cnt - $past(cnt)) == 10'sd5);
                    end
                end

                // Counting down
                if (!$past(mode)) begin
                    if (!$past(rst)) begin
                        if ($past(cnt) == -10'sd148) // INV + DEC, counter jumped over INV
                            assert (($past(cnt) - cnt) == 10'sd18);
                        else if ($past(cnt) >= -10'sd343 && $past(cnt) < -10'sd334) // cnt was near MIN
                            assert (($past(cnt) - cnt) == 10'sd0);
                        else
                            assert (($past(cnt) - cnt) == 10'sd9);
                    end
                end
            end
        end
    `endif
endmodule
christian-krieg commented 2 years ago

Reopening this issue because it would be nice if read_verilog would behave as read. Leaving it like it is potentially makes the Verilog code less readable, and IMO less usable.