search

Betterment / alchemist

A Flutter tool that makes golden testing easy.
MIT License
268 stars 36 forks source link

request: Create MacOS Folder for Intel and Apple Silicon #114

Open jxstxn1 opened 8 months ago

jxstxn1 commented 8 months ago

Is there an existing feature request for this?

Command

Differ between Intel Macs & Apple Silicon Macs

Description

Having two MacOS folders, one for Intel and one for ARM. File Structure could be like this:

.
└── macos/
    ├── intel
    └── arm

or 

.
├── macos-intel
└── macos-arm

Reasoning

If you run Golden Tests with an Intel Mac, it will produce different Golden Images than an Apple Silicon Mac and vice versa. To be able to test on both platforms reliable, it would be great to have two folders. One for each architecture.

Additional context and comments

No response

Bassiuz commented 8 months ago

Very interested in this fix. We are currently bypassing this by forcing all machines to use the X86_64 version of Flutter to generate image and run the tests for comparison; which 'works' but decreases test time a bit.

We could also use this to let ARM machines generate both the intel and arm goldens if wanted.

btrautmann commented 2 months ago

This seems fairly doable. My initial thought is within goldenTest we could detect the architecture and pass that along to the FilePathResolver. I think we'd want to have a caching mechanism so we're not invoking Process.run for every test. I think it would be safe to have this be global because we shouldn't be changing platforms/architectures in the middle of a suite run.

@Bassiuz @jxstxn1 let me know if that sounds fine.

PS I'm curious if you all have examples of some of the differences you've seen between the 2 architectures. Mainly just curious.

jxstxn1 commented 2 months ago

@btrautmann the approach seems fine for me.

We noticed the difference on the Edge of a PieChart from fl_chart. Got a diff with 278px/0,00%-0,01% difference xD