splitwise / super_diff

A more helpful way to view differences between complex data structures in RSpec.
https://splitwise.github.io/super_diff/
MIT License
991 stars 54 forks source link

Replace Zeus with process forking strategy #271

Closed jas14 closed 1 month ago

jas14 commented 1 month ago

Summary

Integration tests without Zeus are very slow because we run each example in a new process, which has to load all dependencies and frameworks – like RSpec, SuperDiff, ActiveSupport, etc – from scratch.

Zeus improved on this, but has a client-server architecture and has been proving somewhat difficult to use. If we instead load the dependencies and then run RSpec in a new forked subprocess, we don't have to reload anything loaded before fork time, and don't have to use Zeus.

Performance

Run on a 10-core Apple M1 Pro with 16 GB RAM.

Bench:

time ./bin/rspec spec/integration/rspec

Before

387.17s user 146.91s system 95% cpu 9:18.77 total

After

88.45s user 21.43s system 97% cpu 1:52.32 total

This represents a ~5x speed-up.

github-actions[bot] commented 1 month ago

:book: A new version of the docsite has been published at: https://splitwise.github.io/super_diff/branches/speed-up-integration-tests/91fe143eff6806c708e5a0eb5931c4b915b87673

github-actions[bot] commented 1 month ago

:book: A new version of the docsite has been published at: https://splitwise.github.io/super_diff/branches/speed-up-integration-tests/58705a1a40e5321fdb8f054a98cdb5b83e20e3e0

github-actions[bot] commented 1 month ago

:book: A new version of the docsite has been published at: https://splitwise.github.io/super_diff/branches/speed-up-integration-tests/ec7a32f9770134896c58117080a6774d39817439

github-actions[bot] commented 1 month ago

:book: A new version of the docsite has been published at: https://splitwise.github.io/super_diff/branches/speed-up-integration-tests/9a15b48e838ecd770d3df92a5c58a5ba4e8ad889

github-actions[bot] commented 1 month ago

:book: A new version of the docsite has been published at: https://splitwise.github.io/super_diff/branches/speed-up-integration-tests/ad73d2824ce3772918c7f61db23dae630e71e554