Closed bunnie closed 2 years ago
I fixed that oversight, thanks for letting us know!
Still trying to get the tool to run -- how long does it usually take to generate output on a medium-sized design (XC7S50 at 70% utilization)? I've let the script run for 24 hrs now, probably I've configured something wrong I'm guessing and this is not the expected runtime...
You are correct, that is not the expected runtime. I am aware of some performance issues that might occur on smaller designs (funnily enough) when running find_fault_bits.py.. I am working on fixing that right now, but again that only really applied to a tiny design with a single utilized CLB tile. I will let you know when I push those fixes up as it does rework some of the functions a bit which may solve your issue.
It will also include a debug flag which will print some timing information whenever important functions finish, so we could potentially narrow down where the issue is occurring if it is not fixed in my next commit.
In the meantime, are there any optional flags that you set on the command line when running find_fault_bits.py? Have you sourced a version of Vivado?
EDIT: I forgot to mention that typical runtime on my machine has been 90-120 seconds, so if it has taken 24 hours something has definitely gone wrong.
@bunnie - it might be helpful to share your design files with them.
Is there a verbose mode that gives feedback that BFAT is doing something?
It could be that there are too many bits to evaluate. BFAT is designed to evaluate a handful of bits and we haven't looked at large bit lists yet.
We do know that the Vivado interface is very slow and we are working on adding a RapidWright interface to speed things up.
@wirthlin - There is somewhat of a verbose mode for the main BFAT tool (outputs timing details after major functions return), and I have made one for find_fault_bits.py
as well but it has not been made public yet.
Though, I doubt that the issue is too many bits, as they are running find_fault_bits.py which only generates a list of 7 fault bits. The issue is likely within find_fault_bits.py
itself.
@bunnie - Commit 415fab1 should improve the performance of the script. I can't say for sure whether it will fix your problem without running your design through it myself, though. I also added a flag -d
which will print timing details after major function returns in the tool. So, if you still run into issues and do not want to send your design files, I would appreciate it if you could send the console log so I can determine which function you are getting stuck in.
Having the latest vivado helped a lot. I was hoping it could run under 2020.1 but it wasn't, so I bit the bullet and did a 2022.1 install. The tool is running now and generating output, takes about a minute to run.
I'm using Python 3.8...I noticed your docs say to use Python 3.9. Usually I'm able to get away with a slightly older Python tho... but, could this explain why every time I run the tool, I get a different answer that seems to consist of exactly one line of JSON?
For example, on one run I get this:
[
[
[
"00000a1a",
"096",
"31"
]
],
[
[
"00001487",
"065",
"06"
]
],
[
[
"0000148e",
"065",
"07"
]
],
[
[
"0000148a",
"065",
"07"
]
],
[
[
"0000002a",
"000",
"00"
]
],
[
[
"00020220",
"097",
"15"
],
[
"00001595",
"093",
"07"
]
]
]
and then the next run -- on the exact same bitstream and dcp file -- I get this:
[
[
[
"00000da2",
"068",
"31"
]
],
[
[
"00400415",
"070",
"09"
]
],
[
[
"00020105",
"030",
"04"
]
],
[
[
"0002010e",
"030",
"27"
]
],
[
[
"0000002a",
"000",
"00"
]
],
[
[
"0040039a",
"000",
"15"
],
[
"00020215",
"095",
"07"
]
]
]
I'd like to avoid upgrading Python on this machine just because it's going to affect every other thing I do with it, if I can. Vivado was just more about freeing up the disk space; setting the version is easy enough with a local script. Could also be maybe there is a way to specify a second Python version for a specific environment and I just don't know that trick, but mostly I just run with what Ubuntu packages and ships in 20.04 LTE.
I can also upload the bitstream and DCP if it helps.
Ah. I have just discovered you can install Python 3.9 on Ubuntu 20.04 and it doesn't overwrite the system default, and you can just run it as python3.9
.
So -- now I'm running Vivado 2022.1, Python 3.9, on a design compiled using Vivado 2022.1, and finding that I still have that same behavior -- I'm reported exactly one failure in certain categories but the exact failure changes from run to run, even though the inputs are the same.
I'll open a new issue for this.
That is actually the intended output of the script. It is supposed to generate an arbitrary list of failt bits which create some of the most common types of faults you will see when running BFAT, such as a LUT initialization bit flip or an open within a net. It will not generate the same output every time.
Steps to reproduce:
b
in itExample:
Design name is 'betrusted_soc'. Header looks like this:
This line:
https://github.com/byuccl/bfat/blob/42c4a788721a6729a65585d0cc3ae371e69ed580/bitread.py#L98
Finds the letter
b
(ord value 98) in the design name and decides that is the beginning of the part number record, but it is actually the design name.Workaround:
Will simply lop off the name of the design and allow the script to run on the resulting
_trunc.bit
file.A more permanent solution might be to parse the bitstream to look for a more robust sentinel. I'm not so familiar with the .bit format to recommend what that would be, but maybe searching for the trailing and leading '00', so a sequence of [0x00, 0x62, 0x00], would be robust since the file name is terminated with a
;
character and not a null. See also http://www.pldtool.com/pdf/fmt_xilinxbit.pdf. This sequence would work for any part number length that is shorter than 255 characters (a longer length would put a 0x01 after the 0x62), but I don't know of any Xilinx part numbers that are that long.