Fuzzy testing and QuickCheck/SmallCheck

[theinze]

Are there any opportunities for the above-mentioned approaches ?

[joha2]

I don't know how this works. Could you please provide an example or wiki link? :-) I think there are opportunities:

• quickcheck could be used on test which may give zero independently from the input numbers (see test_localcoordinates)
• what's fuzzy testing?

[theinze]

That's a pretty good start in test_localcoordinates.py for quickcheck, I'll look into this ...

[theinze]

see https://github.com/joha2/pyrate/pull/22

@joha2: Generated random data currently quite limited (i.e., floats in [0.,1.]), I would like to feed in arbitrary floats, would that make sense ?

[joha2]

@theinze Yes this would make indeed sense, since the transform and its inverse have to yield the original vector for arbitrary input vectors. Btw: The last test was about testing whether the back transformation of LocalCoordinate itself works. The tilt and decenter parameters are in relation to the parent coordinate system and if you feed in the tilt and decenter with negative values and in reverse order for the child you should end up at the parent coordinates and parent basis system.

[theinze]

I am getting a test fail for:

Falsifying example: test_transform_points_inverse(x=0.0, y=0.0, z=134217728.0, tx=0.0, ty=0.0, tz=0.0, xloc=array([[  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          0.00000000e+00],
       [  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          0.00000000e+00],
       [  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          1.00000000e-08]]))

with

def test_transform_points_inverse(x, y, z, tx, ty, tz, xloc):
    """
    Sequential local/global and back transformation yields original point.
    """
    lc1 = LocalCoordinates(name="1",
                           decx=x, 
                           decy=y,
                           decz=z,
                           tiltx=tx,
                           tilty=ty,
                           tiltz=tz)
    xglob = lc1.returnLocalToGlobalPoints(xloc)
    xloc2 = lc1.returnGlobalToLocalPoints(xglob)
    assert np.allclose(xloc2-xloc, 0)

I guess that is also something expected, due to rounding and error propagation ?

Are there any implicit subrange constraints for coordinates and transformation values ?

[theinze]

Same same for testtest_transform_tensors_invariant_scalarproduct with small epsilon value:

Falsifying example: test_transform_tensors_invariant_scalarproduct(x=0.0, y=0.0, z=0.0, tx=0.0, ty=0.0, tz=0.0, v1loc=array([[  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          0.00000000e+00],
       [  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          0.00000000e+00],
       [  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          1.00000000e-08]]), v2loc=array([[  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          0.00000000e+00],
       [  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          0.00000000e+00],
       [  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
          1.00000000e-08]]), t1loc=array([[[  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00],
        [  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00],
        [  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00]],

       [[  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00],
        [  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00],
        [  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00]],

       [[  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00],
        [  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00],
        [  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           1.00000000e-08]]]), t2loc=array([[[  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00],
        [  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00],
        [  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00]],

       [[  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00],
        [  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00],
        [  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00]],

       [[  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00],
        [  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00],
        [  0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
           0.00000000e+00,   0.00000000e+00,   1.00000000e-08,
           0.00000000e+00]]]))

[joha2]

@theinze as far as I know there are no subrange constraints. In the constructor the basis vectors are constructed by using the Rodriguez matrix for a given axis. But for the transformation you showed here (tilts identical to zero, x,y identical to zero, z large), there should not be any problems. Sounds odd to me. Can you identify the part of the code where the errors are generated?

[theinze]

@joha2 I can look into that ... I just wanted to make sure that it's not something one would expect.

In the first example, e.g., there is a large z-value for the local coordinates and the matrix equals zero except one element having a small value 1.00000000e-08. Should the assertion also hold for an example like that ?

[joha2]

Since there are only multiplications involved I would expect that the conditions hold up to numerical accuracy 1e-16. I see that z ~ 1e8 so maybe there is some error propagation such that 1e-16 1e8 = 1e-8 -> fails. For the function f(x, y) = x y the error propagation is something like df = dx y + x dy, maybe this is the reason for the failure. Maybe we have to modify the tests to put these things into account. Is there a good computer science trick for doing that? :-)

[theinze]

Thanks for your clarification and I'll try to look into this ...