tlaplus / tlaplus

TLC is a model checker for specifications written in TLA+. The TLA+Toolbox is an IDE for TLA+.
https://lamport.azurewebsites.net/tla/tla.html
MIT License
2.31k stars 194 forks source link

Ability to save TLC error trace in user-defined formats #640

Closed will62794 closed 2 years ago

will62794 commented 3 years ago

In some cases it would be helpful to have the ability to save the error trace generated by TLC in an easily parseable format (e.g. JSON) for use by other tools, scripts, applications, etc. I have implemented a minimal prototype of this, using the previously developed functionality from the Json community module. The prototype usage is:

java -cp tla2tools.jar tlc2.TLC -jsonTrace <specname>

which saves a JSON error trace file to trace.json. Of course, the exact CLI interface could be worked out more carefully, but that is just one, basic idea. See an example trace.json file for a broken run of Peterson's algorithm.

lemmy commented 3 years ago

The -jsonTrace parameter should be combined with -generateSpecTE into -trace tla, -trace json, and -trace tla,json. As to the implementation, would it be possible to build it with the new-ish tlc2.output.IMessagePrinterRecorder functionality?

Here is an example how -generateSpecTE work as of now:

---- MODULE DieHard_TTrace_1630686553 ----
EXTENDS Sequences, TLCExt, Toolbox, Naturals, TLC, DieHard

_expression ==
    LET DieHard_TEExpression == INSTANCE DieHard_TEExpression
    IN DieHard_TEExpression!expression
----

_trace ==
    LET DieHard_TETrace == INSTANCE DieHard_TETrace
    IN DieHard_TETrace!trace
----

_inv ==
    ~(
        TLCGet("level") = Len(_TETrace)
        /\
        small = (3)
        /\
        big = (4)
    )
----

_init ==
    /\ big = _TETrace[1].big
    /\ small = _TETrace[1].small
----

_next ==
    /\ \E i,j \in DOMAIN _TETrace:
        /\ \/ /\ j = i + 1
              /\ i = TLCGet("level")
        /\ big  = _TETrace[i].big
        /\ big' = _TETrace[j].big
        /\ small  = _TETrace[i].small
        /\ small' = _TETrace[j].small

\* Uncomment the ASSUME below to write the states of the error trace
\* to the given file in Json format. Note that you can pass any tuple
\* to `JsonSerialize`. For example, a sub-sequence of _TETrace.
    \* ASSUME
    \*     LET J == INSTANCE Json
    \*         IN J!JsonSerialize("DieHard_TTrace_1630686553.json", _TETrace)

=============================================================================

 Note that you can extract this module `DieHard_TEExpression`
  to a dedicated file to reuse `expression` (the module in the 
  dedicated `DieHard_TEExpression.tla` file takes precedence 
  over the module `BlockingQueue_TEExpression` below).

---- MODULE DieHard_TEExpression ----
EXTENDS Sequences, TLCExt, Toolbox, Naturals, TLC, DieHard

expression == 
    [
        \* To hide variables of the `DieHard` spec from the error trace,
        \* remove the variables below.  The trace will be written in the order
        \* of the fields of this record.
        big |-> big
        ,small |-> small

        \* Put additional constant-, state-, and action-level expressions here:
        \* ,_stateNumber |-> _TEPosition
        \* ,_bigUnchanged |-> big = big'

        \* Format the `big` variable as Json value.
        \* ,_bigJson |->
        \*     LET J == INSTANCE Json
        \*     IN J!ToJson(big)

        \* Lastly, you may build expressions over arbitrary sets of states by
        \* leveraging the _TETrace operator.  For example, this is how to
        \* count the number of times a spec variable changed up to the current
        \* state in the trace.
        \* ,_bigModCount |->
        \*     LET F[s \in DOMAIN _TETrace] ==
        \*         IF s = 1 THEN 0
        \*         ELSE IF _TETrace[s].big # _TETrace[s-1].big
        \*             THEN 1 + F[s-1] ELSE F[s-1]
        \*     IN F[_TEPosition - 1]
    ]

=============================================================================

Parsing and semantic processing can take forever if the trace below is long.
 In this case, it is advised to deserialize the trace from a binary file.
 To create the file, replace your spec's invariant F with:
  Inv == IF F THEN TRUE ELSE ~IOSerialize(Trace, "file.bin", TRUE)
 (IOUtils module is from https://modules.tlapl.us/)
\*
\*---- MODULE DieHard_TETrace ----
\*EXTENDS IOUtils, TLC, DieHard
\*
\*trace == IODeserialize("file.bin", TRUE)
\*
\*=============================================================================
\*

---- MODULE DieHard_TETrace ----
EXTENDS TLC, DieHard

trace == 
    <<
    ([small |-> 0,big |-> 0]),
    ([small |-> 0,big |-> 5]),
    ([small |-> 3,big |-> 2]),
    ([small |-> 0,big |-> 2]),
    ([small |-> 2,big |-> 0]),
    ([small |-> 2,big |-> 5]),
    ([small |-> 3,big |-> 4])
    >>
----

=============================================================================

---- CONFIG DieHard_TTrace_1630686553 ----

INVARIANT
    _inv

CHECK_DEADLOCK
    \* CHECK_DEADLOCK off because of PROPERTY or INVARIANT above.
    FALSE

INIT
    _init

NEXT
    _next

CONSTANT
    _TETrace <- _trace

ALIAS
    _expression
=============================================================================
\* Generated on Fri Sep 03 09:29:15 PDT 2021
markus@banana:~/examples/specifications/DieHard(master)$ alias tlc='java -Dtlc2.value.Values.width=1000 -Dutil.FileUtil.milliseconds=true -XX:+UseParallelGC -cp /opt/toolbox/tla2tools.jar tlc2.TLC'

markus@banana:~/examples/specifications/DieHard(master)$ tlc DieHard
TLC2 Version 2.16 of Day Month 20?? (rev: c824a8a)
Running breadth-first search Model-Checking with fp 59 and seed -6705806797176730866 with 1 worker on 16 cores with 7054MB heap and 64MB offheap memory [pid: 245017] (Linux 5.11.0-27-generic amd64, Ubuntu 11.0.11 x86_64, MSBDiskFPSet, DiskStateQueue).
Parsing file /home/markus/src/TLA/_specs/examples/specifications/DieHard/DieHard.tla
Parsing file /tmp/Naturals.tla (jar:file:/opt/toolbox/tla2tools.jar!/tla2sany/StandardModules/Naturals.tla)
Semantic processing of module Naturals
Semantic processing of module DieHard
Starting... (2021-09-03 09:29:14)
Computing initial states...
Finished computing initial states: 1 distinct state generated at 2021-09-03 09:29:15.
Error: Invariant NotSolved is violated.
Error: The behavior up to this point is:
State 1: <Initial predicate>
/\ big = 0
/\ small = 0

State 2: <FillBigJug line 68, col 18 to line 69, col 34 of module DieHard>
/\ big = 5
/\ small = 0

State 3: <BigToSmall line 97, col 15 to line 98, col 48 of module DieHard>
/\ big = 2
/\ small = 3

State 4: <EmptySmallJug line 71, col 18 to line 72, col 30 of module DieHard>
/\ big = 2
/\ small = 0

State 5: <BigToSmall line 97, col 15 to line 98, col 48 of module DieHard>
/\ big = 0
/\ small = 2

State 6: <FillBigJug line 68, col 18 to line 69, col 34 of module DieHard>
/\ big = 5
/\ small = 2

State 7: <BigToSmall line 97, col 15 to line 98, col 48 of module DieHard>
/\ big = 4
/\ small = 3

73 states generated, 14 distinct states found, 1 states left on queue.
The depth of the complete state graph search is 7.
Finished in 01s at (2021-09-03 09:29:15)
Trace exploration spec path: ./DieHard_TTrace_1630686553.tla

markus@banana:~/examples/specifications/DieHard(master)$ tlc -config DieHard_TTrace_1630686553.tla DieHard_TTrace_1630686553.tla 
TLC2 Version 2.16 of Day Month 20?? (rev: c824a8a)
Running breadth-first search Model-Checking with fp 60 and seed 7508979940518507983 with 1 worker on 16 cores with 7054MB heap and 64MB offheap memory [pid: 245140] (Linux 5.11.0-27-generic amd64, Ubuntu 11.0.11 x86_64, MSBDiskFPSet, DiskStateQueue).
Parsing file /home/markus/src/TLA/_specs/examples/specifications/DieHard/DieHard_TTrace_1630686553.tla
Parsing file /tmp/Sequences.tla (jar:file:/opt/toolbox/tla2tools.jar!/tla2sany/StandardModules/Sequences.tla)
Parsing file /tmp/TLCExt.tla (jar:file:/opt/toolbox/tla2tools.jar!/tla2sany/StandardModules/TLCExt.tla)
Parsing file /tmp/Toolbox.tla (jar:file:/opt/toolbox/tla2tools.jar!/tla2sany/StandardModules/Toolbox.tla)
Parsing file /tmp/Naturals.tla (jar:file:/opt/toolbox/tla2tools.jar!/tla2sany/StandardModules/Naturals.tla)
Parsing file /tmp/TLC.tla (jar:file:/opt/toolbox/tla2tools.jar!/tla2sany/StandardModules/TLC.tla)
Parsing file /home/markus/src/TLA/_specs/examples/specifications/DieHard/DieHard.tla
Parsing file /tmp/DieHard_TEExpression.tla (/home/markus/src/TLA/_specs/examples/specifications/DieHard/DieHard_TTrace_1630686553.tla)
Parsing file /tmp/DieHard_TETrace.tla (/home/markus/src/TLA/_specs/examples/specifications/DieHard/DieHard_TTrace_1630686553.tla)
Parsing file /tmp/FiniteSets.tla (jar:file:/opt/toolbox/tla2tools.jar!/tla2sany/StandardModules/FiniteSets.tla)
Semantic processing of module Naturals
Semantic processing of module Sequences
Semantic processing of module FiniteSets
Semantic processing of module TLC
Semantic processing of module TLCExt
Semantic processing of module Toolbox
Semantic processing of module DieHard
Semantic processing of module DieHard_TEExpression
Semantic processing of module DieHard_TETrace
Semantic processing of module DieHard_TTrace_1630686553
Starting... (2021-09-03 09:29:38)
Computing initial states...
Finished computing initial states: 1 distinct state generated at 2021-09-03 09:29:39.
Error: Invariant _inv is violated.
Error: The behavior up to this point is:
State 1: <Initial predicate>
/\ big = 0
/\ small = 0

State 2: <_next line 30, col 5 to line 36, col 37 of module DieHard_TTrace_1630686553>
/\ big = 5
/\ small = 0

State 3: <_next line 30, col 5 to line 36, col 37 of module DieHard_TTrace_1630686553>
/\ big = 2
/\ small = 3

State 4: <_next line 30, col 5 to line 36, col 37 of module DieHard_TTrace_1630686553>
/\ big = 2
/\ small = 0

State 5: <_next line 30, col 5 to line 36, col 37 of module DieHard_TTrace_1630686553>
/\ big = 0
/\ small = 2

State 6: <_next line 30, col 5 to line 36, col 37 of module DieHard_TTrace_1630686553>
/\ big = 5
/\ small = 2

State 7: <_next line 30, col 5 to line 36, col 37 of module DieHard_TTrace_1630686553>
/\ big = 4
/\ small = 3

7 states generated, 7 distinct states found, 0 states left on queue.
The depth of the complete state graph search is 7.
Finished in 01s at (2021-09-03 09:29:39)
pfeodrippe commented 3 years ago

I have been using EDN to get the trace (see https://github.com/pfeodrippe/recife/blob/master/test/hillel_test.clj#L45 for usage in a test).

Also another POC for JSON output is at https://youtu.be/vzdSQNAZt-Q (different approach as I am using IMessagePrinterRecorder to get the output like mentioned by Markus), although I would not recommend using this format as you cannot represent e.g. sets in JSON natively, maybe XML (if not EDN) would be a better fit.

lemmy commented 2 years ago

/cc @konnov @andrey-kuprianov @danwt related to https://apalache.informal.systems/docs/adr/015adr-trace.html

lemmy commented 2 years ago

@will62794 What's the deal with https://github.com/will62794/tlaplus/commit/6c6d64976fad18a459e548af9684516b5b4134f4 ? :-)

will62794 commented 2 years ago

@will62794 What's the deal with will62794@6c6d649 ? :-)

@lemmy I've just been using that branch for some personal work, but haven't made any further efforts to productionize it.

lemmy commented 2 years ago

After a cursory look, a large portion of your work (e.g. tlatools/org.lamport.tlatools/src/tlc2/tool/TLCJson.java) could now use https://github.com/tlaplus/tlaplus/blob/master/tlatools/org.lamport.tlatools/src/tlc2/module/Json.java.

That said, I'd rather introduce a generic hook (similar to ALIAS or an extension thereof) with which users define a (constant-level) conversion operator whose parameter would be the TLA+ trace.

WithIOUtils!IOEnv, users could easily choose the conversion on the command line.

lemmy commented 2 years ago

I've just realized that we only have make TLCExt!Trace work in the scope of POSTCONDITION. => More easily done than said for safety (https://github.com/tlaplus/tlaplus/commit/b9ab320da557646d174528b5c92b995611378e6e)!

---- MODULE MC ----
EXTENDS SomeSpec, Json, TLCExt

...

PostCondition ==
    JsonSerialize("trace.json", Trace)

====
SPECIFICATION Spec
CONSTANT N = 42
...
POSTCONDITION PostCondition
lemmy commented 2 years ago

For TLC to support the Apalache trace format, somebody now has to define a TLA+ operator that converts TLCExt!Trace to the format described in https://apalache.informal.systems/docs/adr/015adr-trace.html.

will62794 commented 2 years ago

From a user perspective, I would prefer to have a dedicated command line flag that dumps the error trace to a specified JSON file. This seems the most straightforward approach. I would prefer to not have to write extra commands in TLA+ just to dump an error trace to a file. Using the POSTCONDITION feature/syntax just feels too esoteric.

lemmy commented 2 years ago

I'm happy to review a PR that adds a command-line flag as described in https://github.com/tlaplus/tlaplus/issues/640#issuecomment-869070134 that is based on the infrastructure provided in https://github.com/tlaplus/tlaplus/commit/b9ab320da557646d174528b5c92b995611378e6e.

konnov commented 2 years ago

IMO supporting a particular trace format like ours is less important than just having a non-ambiguous and easy-to-parse output format. We can always write a json-to-json convertor. The current solution with the Json module and IOUtils requires some manipulations on the user side, which is harder to automate.

danwt commented 2 years ago

I agree with @will62794, for what it's worth. We should place fewer demands on the model and model writer.

lemmy commented 2 years ago

Nobody should ever parse TLC's output (see the TLA+ Toolbox paper where this is listed as a major Toolbox flaw)!

Also, I'm sure there are users who will write json->ndjson->yaml->tomorrow's favorite format converters. However, there is no reason to make it impossible for users, who prefer other formats such as EDN (see above), to use the same infrastructure that converts TLA+ traces--the one format that is guaranteed to represent all of TLA+ is TLA+.

Convenience for popular formats such as json can be added by:

0) Wherever TLC prints traces, the code has to call a hook (we could call it jsonConverter, but I like doPostCondition better). 1) The definition of TLCExt!Trace gets extended to handle lassos (I'm leaning towards something inspired by Graphs.tla). 2) A -postcondition flag is added to TLC.java that takes the full-qualified name of a (constant-level) operator. In other words, we expose the POSTCONDITION config keyword on the command-line. 3) On the command-line, -postcondition accepts (basic) operator parameters such as string and integer.
4) TLC (or CommunityModules) ships TLA+ modules such as JsonTrace, ITFTrace that all define a Convert operator. 5) For niche formats, users manually add corresponding modules via the TLA-Library path.

=> Users who wish to convert a trace to e.g. Json run TLC with -postcondition \"JsonTrace!Convert("trace.json")\".

Advantages:

Who is going to provide a patch for 2) and 3)? 4) will happen over time.

lemmy commented 2 years ago

The commits above make it easy to get a counter-example from TLC:

            [ action |->
                  { << << 1,
                          [ Nodes |-> (0 :> [Load |-> 0] @@ 1 :> [Load |-> 0]),
                            Agent |->
                                [ Loc |-> 0,
                                  LastLoad |-> 0,
                                  ReadyToMove |-> TRUE,
                                  Task |-> 0 ],
                            CanCreate |-> TRUE ] >>,
                       [ name |-> "CreateTasks",
                         location |->
                             [ beginLine |-> 82,
                               beginColumn |-> 3,
                               endLine |-> 84,
                               endColumn |-> 35,
                               module |-> "AgentRing" ] ],
                       << 2,
                          [ Nodes |-> (0 :> [Load |-> 2] @@ 1 :> [Load |-> 0]),
                            Agent |->
                                [ Loc |-> 0,
                                  LastLoad |-> 0,
                                  ReadyToMove |-> TRUE,
                                  Task |-> 0 ],
                            CanCreate |-> TRUE ] >> >>,
                    << << 2,
                          [ Nodes |-> (0 :> [Load |-> 2] @@ 1 :> [Load |-> 0]),
                            Agent |->
                                [ Loc |-> 0,
                                  LastLoad |-> 0,
                                  ReadyToMove |-> TRUE,
                                  Task |-> 0 ],
                            CanCreate |-> TRUE ] >>,
                       [ name |-> "Move",
                         location |->
                             [ beginLine |-> 58,
                               beginColumn |-> 3,
                               endLine |-> 60,
                               endColumn |-> 35,
                               module |-> "AgentRing" ] ],
                       << 3,
                          [ Nodes |-> (0 :> [Load |-> 2] @@ 1 :> [Load |-> 0]),
                            Agent |->
                                [ Loc |-> 1,
                                  LastLoad |-> 0,
                                  ReadyToMove |-> FALSE,
                                  Task |-> 0 ],
                            CanCreate |-> TRUE ] >> >>,
                    << << 3,
                          [ Nodes |-> (0 :> [Load |-> 2] @@ 1 :> [Load |-> 0]),
                            Agent |->
                                [ Loc |-> 1,
                                  LastLoad |-> 0,
                                  ReadyToMove |-> FALSE,
                                  Task |-> 0 ],
                            CanCreate |-> TRUE ] >>,
                       [ name |-> "CreateTasks",
                         location |->
                             [ beginLine |-> 82,
                               beginColumn |-> 3,
                               endLine |-> 84,
                               endColumn |-> 35,
                               module |-> "AgentRing" ] ],
                       << 4,
                          [ Nodes |-> (0 :> [Load |-> 2] @@ 1 :> [Load |-> 2]),
                            Agent |->
                                [ Loc |-> 1,
                                  LastLoad |-> 0,
                                  ReadyToMove |-> FALSE,
                                  Task |-> 0 ],
                            CanCreate |-> TRUE ] >> >>,
                    << << 4,
                          [ Nodes |-> (0 :> [Load |-> 2] @@ 1 :> [Load |-> 2]),
                            Agent |->
                                [ Loc |-> 1,
                                  LastLoad |-> 0,
                                  ReadyToMove |-> FALSE,
                                  Task |-> 0 ],
                            CanCreate |-> TRUE ] >>,
                       [ name |-> "Stop",
                         location |->
                             [ beginLine |-> 78,
                               beginColumn |-> 3,
                               endLine |-> 79,
                               endColumn |-> 31,
                               module |-> "AgentRing" ] ],
                       << 5,
                          [ Nodes |-> (0 :> [Load |-> 2] @@ 1 :> [Load |-> 2]),
                            Agent |->
                                [ Loc |-> 1,
                                  LastLoad |-> 0,
                                  ReadyToMove |-> FALSE,
                                  Task |-> 0 ],
                            CanCreate |-> FALSE ] >> >>,
                    << << 5,
                          [ Nodes |-> (0 :> [Load |-> 2] @@ 1 :> [Load |-> 2]),
                            Agent |->
                                [ Loc |-> 1,
                                  LastLoad |-> 0,
                                  ReadyToMove |-> FALSE,
                                  Task |-> 0 ],
                            CanCreate |-> FALSE ] >>,
                       [ name |-> "LookAndAct",
                         location |->
                             [ beginLine |-> 63,
                               beginColumn |-> 3,
                               endLine |-> 75,
                               endColumn |-> 24,
                               module |-> "AgentRing" ] ],
                       << 6,
                          [ Nodes |-> (0 :> [Load |-> 2] @@ 1 :> [Load |-> 1]),
                            Agent |->
                                [ Loc |-> 1,
                                  LastLoad |-> 1,
                                  ReadyToMove |-> TRUE,
                                  Task |-> 1 ],
                            CanCreate |-> FALSE ] >> >> },
              state |->
                  { << 1,
                       [ Nodes |-> (0 :> [Load |-> 0] @@ 1 :> [Load |-> 0]),
                         Agent |->
                             [Loc |-> 0, LastLoad |-> 0, ReadyToMove |-> TRUE, Task |-> 0],
                         CanCreate |-> TRUE ] >>,
                    << 2,
                       [ Nodes |-> (0 :> [Load |-> 2] @@ 1 :> [Load |-> 0]),
                         Agent |->
                             [Loc |-> 0, LastLoad |-> 0, ReadyToMove |-> TRUE, Task |-> 0],
                         CanCreate |-> TRUE ] >>,
                    << 3,
                       [ Nodes |-> (0 :> [Load |-> 2] @@ 1 :> [Load |-> 0]),
                         Agent |->
                             [Loc |-> 1, LastLoad |-> 0, ReadyToMove |-> FALSE, Task |-> 0],
                         CanCreate |-> TRUE ] >>,
                    << 4,
                       [ Nodes |-> (0 :> [Load |-> 2] @@ 1 :> [Load |-> 2]),
                         Agent |->
                             [Loc |-> 1, LastLoad |-> 0, ReadyToMove |-> FALSE, Task |-> 0],
                         CanCreate |-> TRUE ] >>,
                    << 5,
                       [ Nodes |-> (0 :> [Load |-> 2] @@ 1 :> [Load |-> 2]),
                         Agent |->
                             [Loc |-> 1, LastLoad |-> 0, ReadyToMove |-> FALSE, Task |-> 0],
                         CanCreate |-> FALSE ] >>,
                    << 6,
                       [ Nodes |-> (0 :> [Load |-> 2] @@ 1 :> [Load |-> 1]),
                         Agent |->
                             [Loc |-> 1, LastLoad |-> 1, ReadyToMove |-> TRUE, Task |-> 1],
                         CanCreate |-> FALSE ] >> } ]

and to reformat counter-examples as, e.g., json:

EXTENDS TLCExt, Json

...

PostCondition == JsonSerialize("OneBitMutextNoSymmetry.json", ToTrace(CounterExample))
[
{"x":{"A":false,"B":false},"pc":{"A":"ncs","B":"ncs"},"unchecked":{"A":[],"B":[]},"other":{"A":"A","B":"A"}},
{"x":{"A":false,"B":false},"pc":{"A":"e1","B":"ncs"},"unchecked":{"A":[],"B":[]},"other":{"A":"A","B":"A"}},
{"x":{"A":false,"B":false},"pc":{"A":"e1","B":"e1"},"unchecked":{"A":[],"B":[]},"other":{"A":"A","B":"A"}},
{"x":{"A":false,"B":true},"pc":{"A":"e1","B":"e2"},"unchecked":{"A":[],"B":["A"]},"other":{"A":"A","B":"A"}},
{"x":{"A":false,"B":true},"pc":{"A":"e1","B":"e3"},"unchecked":{"A":[],"B":[]},"other":{"A":"A","B":"A"}},
{"x":{"A":false,"B":true},"pc":{"A":"e1","B":"e2"},"unchecked":{"A":[],"B":[]},"other":{"A":"A","B":"A"}},
{"x":{"A":false,"B":true},"pc":{"A":"e1","B":"cs"},"unchecked":{"A":[],"B":[]},"other":{"A":"A","B":"A"}},
...
]

What's missing is to add ITFTrace to TLC and to add the command-line flag. Any takers?

lemmy commented 2 years ago

This is done:

markus@avocado [BlockingQueue]
-> % tlc -note BlockingQueueTrace.tla -dumpTrace json BQ-$(date +%s).json
TLC2 Version 2.18 of Day Month 20?? (rev: 386eaa1)
Running breadth-first search Model-Checking with fp 52 and seed -2854446944189387096 with 1 worker on 10 cores with 7282MB heap and 64MB offheap memory [pid: 32500] (Mac OS X 12.2.1 aarch64, Homebrew 17.0.2 x86_64, MSBDiskFPSet, DiskStateQueue).
Parsing file /Users/markus/src/TLA/_specs/models/tutorials/BlockingQueue/BlockingQueueTrace.tla
[...]
Semantic processing of module BlockingQueueTrace
Starting... (2022-02-16 17:37:49)
Implied-temporal checking--satisfiability problem has 1 branches.
Computing initial states...
Finished computing initial states: 1 distinct state generated at 2022-02-16 17:37:50.
Error: Action property line 74, col 17 to line 74, col 29 of module BlockingQueue is violated.
Error: The behavior up to this point is:
State 1: <Initial predicate>
/\ buffer = <<>>
/\ i = 1
/\ waitSet = {}

[...]

State 16: <put line 156, col 8 to line 166, col 73 of module BlockingQueueTrace>
/\ buffer = <<"p1", "p1", "p2">>
/\ i = 16
/\ waitSet = {}

"CounterExample written: BQ-1645061869.json"
2529 states generated, 1169 distinct states found, 63 states left on queue.
The depth of the complete state graph search is 16.
Finished in 00s at (2022-02-16 17:37:50)

markus@avocado [BlockingQueue]
-> % cat BQ-1645061869.json
[
{"buffer":[],"i":1,"waitSet":[]},
{"buffer":[],"i":2,"waitSet":["c1"]},
{"buffer":["p1"],"i":3,"waitSet":[]},
{"buffer":["p1","p1"],"i":4,"waitSet":[]},
{"buffer":["p1"],"i":5,"waitSet":[]},
{"buffer":["p1","p1"],"i":6,"waitSet":[]},
{"buffer":["p1"],"i":7,"waitSet":[]},
{"buffer":["p1","p1"],"i":8,"waitSet":[]},
{"buffer":["p1","p1","p1"],"i":9,"waitSet":[]},
{"buffer":["p1","p1","p1"],"i":10,"waitSet":["p1"]},
{"buffer":["p1","p1"],"i":11,"waitSet":[]},
{"buffer":["p1","p1","p1"],"i":12,"waitSet":[]},
{"buffer":["p1","p1","p1"],"i":13,"waitSet":["p1"]},
{"buffer":["p1","p1","p1"],"i":14,"waitSet":["p1","p2"]},
{"buffer":["p1","p1"],"i":15,"waitSet":["p1"]},
{"buffer":["p1","p1","p2"],"i":16,"waitSet":[]}
]

@konnov @danwt Adding support for ITFTrace is a matter of writing a TLA+ definition that converts the CounterExample into your format and serializing your format with JsonSerialize. The code-level changes will be a one-liner in (TLC.java](https://github.com/tlaplus/tlaplus/blob/386eaa19f4d610781a79c95730638ba193cec614/tlatools/org.lamport.tlatools/src/tlc2/TLC.java#L589-L595).

danwt commented 2 years ago

@lemmy thanks this is really cool. I noticed some things (TLC2 Version 2.18 of Day Month 20?? (rev: 386eaa1))

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: Index 5 out of bounds for length 5
    at tlc2.TLC.handleParameters(TLC.java:593)
    at tlc2.TLC.main(TLC.java:308)
lemmy commented 2 years ago

@danwt Thanks, the documentation has been updated, and the ArrayIndexOutOfBoundsException is now correctly handled.

I considered -continue but left its implementation for later. What's your use case (safety vs. liveness, online vs. offline, ...)?