search

litestar-org / polyfactory

Simple and powerful factories for mock data generation
https://polyfactory.litestar.dev/
MIT License
1.06k stars 83 forks source link

Disable randomization option #18

Closed thorin-schiffer closed 2 years ago

thorin-schiffer commented 2 years ago

While randomizing the input is a good idea, big test suits want to remain determined, so the reason of failure can be investigated properly. In multiple places in the codebase random.choice is used, so if a field belongs to a composite type like Union, random results can be expected even if faker seed is set.

What would be very cool, is a global option or similar where randomization can be disabled completely and only determined results are delivered.

Goldziher commented 2 years ago

While randomizing the input is a good idea, big test suits want to remain determined, so the reason of failure can be investigated properly. In multiple places in the codebase random.choice is used, so if a field belongs to a composite type like Union, random results can be expected even if faker seed is set.

What would be very cool, is a global option or similar where randomization can be disabled completely and only determined results are delivered.

Yes, I agree it would be very cool :). I looked into it a while back - but it required quite a bit of work to implement. Basically this requires creating an instance of random.Random and seeding it. Then using this instance instead of the calls to choice and all the other calls to random methods (there are several places of these). The problem is that it creates a stronger coupling inside the code, so it requires some smart design to be in place.

Another option, which can go together with the above, or be an alternative, is automatic "fixture" creation. I have this option in another library of mine in TypeScript- the factory result is written to a file one the test i ran. The next time the test is ran, the file is loaded and compared with the model. If there is no structural difference - i.e. the keys and value types, are the same, the result is returned. If there is a structural difference, the fixture is re-created. This is basically like snapshot testing in JS.

Anyhow, would you like to contribute? I am afraid I have no time to dedicate to major feature in this library at present because I am focusing on starlite.

You are also welcome to join the discord server if you want to chat etc. https://discord.gg/AcctkU5m

thorin-schiffer commented 2 years ago

Looks cool! Atm I'm doing this:

@pytest.fixture
def patch_pydantic_factories(monkeypatch):
    import random
    monkeypatch.setattr(random, "choice", lambda l: l[0])

which is too simple for the wider use, of course. I could dig into it, but don't expect quick result :D

Goldziher commented 2 years ago

Great, take your time! So I will keep this issue open for the time being.

thorin-schiffer commented 2 years ago

Why not use Faker instance that is already has a seeding option? Or it's Random object?

Goldziher commented 2 years ago

Why not use Faker instance that is already has a seeding option? Or it's Random object?

you can pass an instance of faker and seed it if you want - it's already an option exactly for this use case. The problem is that the library also requires data generation outside the scope of faker, and the logic is decoupled in a way that makes it difficult to use a single instance of Random.

Goldziher commented 2 years ago

Merged @eviltnan - thanks a lot. If you would like to help maintain this library, please join discord and lets chat - https://discord.gg/AcctkU5m

thorin-schiffer commented 2 years ago

Will do! I guess this issue can be closed :D Let's talk next year. Happy new year 🎆🎆🎆🎆🎆🎆