confluentinc / ducktape

System integration and performance tests
10 stars 93 forks source link

Objects not supported in @matrix annotation #384

Open kc596 opened 1 year ago

kc596 commented 1 year ago

@confluentinc/quality-eng is tagged for visibility

Describe the bug @matrix annotation is working only if list of primitives are used and not if list of objects are used.

To Reproduce You can use following example to reproduce :

@dataclass
class Car:
    x: int
    y: int

@dataclass
class Bus:
    x: int
    y: int

# Declare global var of list of objects :
CARS = [Car(1,2)]
BUSES = [Bus(1,2), Bus(2,3)]

# Following annotation works :
@matrix(car_index=[0], bus_index=[0,1])
def test(self, car_index: int, bus_index: int):
    car = CARS[car_index]
    bus = BUSES[bus_index]

# However, this one is not working :
@matrix(car=CARS, bus=BUSES)
def test(self, car: Car, bus: Bus):
    pass

Expected behavior The tests should work even when list of objects are passed into @matrix.

imcdo commented 11 months ago

I Don't agree with this, as arguments should be always json serializable, so they can be passed in via the command line if someone wants to use the exact params.

stan-is-hate commented 11 months ago

+1 to @imcdo , ducktape needs to keep any paramters json-serializable, otherwise it breaks existing functionality.

stan-is-hate commented 11 months ago

If your objects are json-serializable though, I think I've seen them work with parameterized tests