search

chipsalliance / synlig

SystemVerilog support for Yosys
Apache License 2.0
167 stars 21 forks source link

sv-tests that work for yosys but not for yosys with plugin #228

Open tgorochowik opened 2 years ago

tgorochowik commented 2 years ago

Here is a script that can be used in the future to find new tests:

#!/usr/bin/env python

import csv
from collections import defaultdict

data = defaultdict(set)

with open('report.csv') as csvfile:
    report = csv.reader(csvfile, delimiter=',')
    for row in report:
        if row[3] == "True":
            data[row[1]].add(row[0])

print('\n'.join(list(data['Yosys'].difference(data['VanillaYosysUhdmPlugin']))))

The report.csv file is generated by sv-tests. The file from the last build can be found here: https://github.com/chipsalliance/sv-tests-results/blob/gh-pages/report.csv

msierszulski commented 2 years ago

Updated script:

#!/usr/bin/env python3

import csv
from collections import defaultdict

data = defaultdict(set)

with open('report.csv') as csvfile:
    report = csv.reader(csvfile, delimiter=',')
    for row in report:
        if row[3] == "True":
            data[row[1]].add(row[0])

l = data['Yosys'].difference(data['UhdmYosys'])
l2 = []

with open('report.csv') as csvfile:
    report = csv.reader(csvfile, delimiter=',')
    for row in report:
        for r in l:
            if row[1] == 'UhdmYosys':
                if row[0] == r:
                    l2.append(str(row[0] + "," + row[3] + "," + row[4]))

print('Test passed but should failed:')
for r in l2:
    if r.endswith("False,0"):
        print(r.split(',')[0])

print('\nTest failed but should passed:')
for r in l2:
    if not r.endswith("False,0"):
        print(r.split(',')[0])

And updated list :

Test passed but should failed:

Test failed but should passed: