fvutils / pyvsc

Python packages providing a library for Verification Stimulus and Coverage
https://fvutils.github.io/pyvsc
Apache License 2.0
113 stars 26 forks source link

Skewed random distribution for lists of random size #121

Closed msmftc closed 2 years ago

msmftc commented 2 years ago

Random lists of type randsz_list_t get a new random size during randomize() calls, with required constraints on the size property determining the range of possible sizes. In tests with small size ranges (max size of 31 or less), the random size is strongly biased to the largest possible size when it ought to be evenly distributed across the range. For instance, when a list of unsigned integers is constrained to have a size in the range [1..10], a size of 10 is selected in about 70% of all randomize() calls. This bias towards the largest possible size exists for all size ranges less than or equal to [1..31], such as [1..5], [1..10], [1..16], and [1..20].

If the size range is wider, such as [1..100], then I observe a strong bias toward a subset of large sizes. For instance, when the size range is [1..100] then sizes in the subrange [67..100] are more than 10x likely to be selected than in the subrange [1..66].

The code below demonstrates the problem for a randsz_list_t with its size constrained to [1..10].

import vsc

@vsc.randobj
class RandConstrData:
    def __init__(self):
        self.a = vsc.randsz_list_t(vsc.uint8_t())

    @vsc.constraint
    def list_size_cnstr(self):
        self.a.size in vsc.rangelist((1,10))

    def __str__(self):
        outStr = ""
        for i in range(self.a.size):
            outStr += f"a[{i}] = {self.a[i]}" + (", " if i<self.a.size-1 else "")
        return(outStr)

@vsc.covergroup
class Size_cg:
    def __init__(self):
        self.with_sample(item = vsc.uint16_t())

        self.a_size = vsc.coverpoint(self.item, bins = {
            "size": vsc.bin_array([], [1,10])
        })

data = RandConstrData()
data_cg = Size_cg()

for i in range(1000):
    data.randomize()
    data_cg.sample(data.a.size)

    # Uncomment to see raw data.
    # print(f"{i}:  size: {data.a.size},  {data}")

vsc.report_coverage(details=True)

Example output from the code above:

    CVP a_size : 100.000000%
    Bins:
        size[0] : 32
        size[1] : 36
        size[2] : 32
        size[3] : 32
        size[4] : 35
        size[5] : 16
        size[6] : 21
        size[7] : 27
        size[8] : 28
        size[9] : 741
msmftc commented 2 years ago

I re-ran several variations of this test with PyVSC 0.6.9, and the random distribution is no longer skewed. The improvements that were released to solve other issues appear to have solved this issue also.