Same inputs yield different results when the tool is run twice

[bunnie]

I'm running Vivado 2022.1 and Python 3.9, and invoking the tool with this command:

python3.9 find_fault_bits.py betrusted_soc.bit betrusted_soc_route.dcp -r -d Running off of commit 415fab146e3f50931b5c3ec6e38277ad1fb76908

If I run this command twice in a row, I get different outputs for e.g. betrusted_soc_sample_bits.json.

One run, for example, yields this:

[
    [
        [
            "00020aa0",
            "027",
            "31"
        ]
    ],
    [
        [
            "00020691",
            "051",
            "22"
        ]
    ],
    [
        [
            "00020694",
            "051",
            "22"
        ]
    ],
    [
        [
            "00020694",
            "052",
            "00"
        ]
    ],
    [
        [
            "0000002a",
            "000",
            "00"
        ]
    ],
    [
        [
            "00400120",
            "002",
            "15"
        ],
        [
            "00400215",
            "004",
            "07"
        ]
    ]
]

and the next will give this

[
    [
        [
            "00020ca2",
            "086",
            "31"
        ]
    ],
    [
        [
            "00020e15",
            "080",
            "09"
        ]
    ],
    [
        [
            "00000904",
            "093",
            "07"
        ]
    ],
    [
        [
            "0000090e",
            "094",
            "11"
        ]
    ],
    [
        [
            "0000002a",
            "000",
            "00"
        ]
    ],
    [
        [
            "00401320",
            "004",
            "15"
        ],
        [
            "00001515",
            "083",
            "07"
        ]
    ]
]

Here's what the diff of the two is:

4,5c4,5
<             "00020aa0",
<             "027",
---
>             "00020ca2",
>             "086",
11,13c11,13
<             "00020691",
<             "051",
<             "22"
---
>             "00020e15",
>             "080",
>             "09"
18,20c18,20
<             "00020694",
<             "051",
<             "22"
---
>             "00000904",
>             "093",
>             "07"
25,27c25,27
<             "00020694",
<             "052",
<             "00"
---
>             "0000090e",
>             "094",
>             "11"
39,40c39,40
<             "00400120",
<             "002",
---
>             "00401320",
>             "004",
44,45c44,45
<             "00400215",
<             "004",
---
>             "00001515",
>             "083",

Is this the expected outcome? My assumption is the tool would list all the problem bits and it would be deterministic, but perhaps I am not understanding the nature of the tool correctly.

To reproduce, you can copy these files (temporarily staged in a weird directory, will be removed eventually):

https://ci.betrusted.io/trng/betrusted_soc_route.dcp https://ci.betrusted.io/trng/betrusted_soc.bit

[bunnie]

Saw response in other thread. Understood now on what is supposed to happen, this is expected behavior.