fvutils / pyvsc

Python packages providing a library for Verification Stimulus and Coverage
https://fvutils.github.io/pyvsc
Apache License 2.0
113 stars 26 forks source link

Skewed distribution for constraints using vsc.rng() #127

Closed msmftc closed 2 years ago

msmftc commented 2 years ago

@mballance,

When I use vsc.rangelist(vsc.rng(min, max)) to describe a range of contiguous integers inside a constraint, the distribution is extremely skewed for small ranges and somewhat skewed for large ranges. In constrast, if I explicitly list the integers (as in vsc.rangelist(5, 6, 7, ... N)) then the distribution is good.

The example code below illustrates the issue. Property A is constrained using vsc.rng(0, 19) while property B is constrained with a list of integers from 0 to 19. Randomization skews A to always be 19, while B is well distributed.

Property C shows that for longer ranges the skew is not as bad, but is still significant. C is constrained using vsc.rng(0, 1999). Randomization skews C to nearly always be in the upper half of the range, [1000..1999].

A partial solution could be for PyVSC to internally expand short vsc.rng ranges into explicit lists of integers.

import vsc

@vsc.randobj
class Selector:
    def __init__(self):
        self.a = vsc.rand_uint64_t()
        self.b = vsc.rand_uint64_t()
        self.c = vsc.rand_uint64_t()

    @vsc.constraint
    def ab_c(self):
        self.a.inside(vsc.rangelist(vsc.rng(0, 19)))
        self.b.inside(vsc.rangelist(0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19))
        self.c.inside(vsc.rangelist(vsc.rng(0,1999)))

selector = Selector()
aStr = "A:"
bStr = "B:"
cStr = "C:"
for i in range(12):
    selector.randomize()
    aStr += "\t" + str(selector.a)
    bStr += "\t" + str(selector.b)
    cStr += "\t" + str(selector.c)
print(aStr)
print(bStr)
print(cStr)
A:      19      19      19      19      19      19      19      19      19      19      19      19
B:      13      16      19      1       7       11      6       2       9       3       6       18
C:      1749    1669    1984    1843    1671    1151    1276    1210    1429    1935    1487    1383
tmeissner commented 2 years ago

I have a similar problem when trying to randomize in small sets, for example:

@vsc.randobj
class constraints(object):
    def __init__(self):
        self.key = vsc.rand_bit_t(8)

    @vsc.constraint
    def c(self):
        self.key in vsc.rangelist(0, 255)
cr = constraints()

for i in range(1000):
    # Get now random stimuli
    cr.randomize()
    assert cr.key == 255

The assertion never hits, I always get 255.

mballance commented 2 years ago

Hi @msmftc and @tmeissner, I've made a couple of improvements make better use of reachable-domain information when applying randomization (available in the 0.6.8 release). Thanks for providing the examples above. I've incorporated the examples you have above as random-distribution tests. I'll close this for now, but feel free to reopen if needed.

msmftc commented 2 years ago

The random distribution is much better now, thank-you.