search

GraphiteEditor / Graphite

2D vector & raster editor that melds traditional layers & tools with a modern node-based, non-destructive, procedural workflow.
https://graphite.rs
Apache License 2.0
7.31k stars 387 forks source link

Better testing infrastructure #360

Open Keavon opened 2 years ago

Keavon commented 2 years ago

We have, so far, written a minimal amount of tests in order to focus on rapid prototyping without major concern for breakage, in the absence of actual users. As the scope of the project grows, the permutations of cases where regressions can crop up is making it harder to detect bugs purely by hand. Users might also care about stability, even though 0.1 is alpha.

We should build testing infrastructure to help create tests. Thanks to the project architecture which is split between the Editor Frontend, Editor Backend, Graphene, and (eventually) Charcoal, with the message dispatch system tying them together, it should be pretty easy to write tests in general by writing many tests using messages. Some infrastructure can include a way of recording messages for easy copy and paste into tests, thereby turning manual tests into automated tests.

This issue can serve as a good place to discuss other forms of testing infrastructure to help make it easy to write and maintain efficacious tests.

bjorn3 commented 2 years ago

https://matklad.github.io/2021/05/31/how-to-test.html is potentially relevant.

KarateCowboy commented 3 months ago

Hello. I am interested in taking a gander at this. I am new to Rust but very experienced with TypeScript and general BDD. Not to toot my own horn, but when it comes to BDD I'm kind of like a 600 lb silverback among tamarins. The article @bjorn3 linked to says that nobody likes to write tests. Well, I am the exception to the rule. I find wrapping tests around existing functionality is also a great, safe way to become very, very familiar with a project's code-base.

My general philosophy is to keep tests in separate files from the code which they are testing. Usually in something like the cargo standard 'tests' directory. I don't get so picky about unit versus integration tests. Most projects have pitiful coverage. IMO, developers have no business quibbling about whether a test should go in the 'integration' folder or the 'unit' folder when their coverage is below 75%. It's like arguing about the spices on your steak when all you have is hamburger patty.

I like runners like mocha for JS/TS and RSpec for Ruby. However, there is nothing much like that in Rust, so I have been familiarizing myself with spectral and the built-in rust test. Oh, and I've also gotten Cucumber for rust working and acquired some familiarity.

I think there is great potential for growth here: I can grow in Rust proficiency and drip my own testing proficiency onto the team. Iron sharpening iron, as a good book once wrote. Thoughts?

KarateCowboy commented 3 months ago

PS: Maybe I could do a little proof of concept as a demo and then flesh it out more from there?

Keavon commented 3 months ago

Hi @KarateCowboy, that's great to hear you're interested in helping with our testing infrastructure! It could definitely use some dedicated attention.

Are you perhaps able to join our Discord server to chat with us about it? It's easier to have a conversation than type long-form comments.

But in general, a system for testing the outputs of all our nodes would be very valuable. There are probably other areas that could be good candidates for setting up testing systems as well that will arise during a conversation in Discord with other members of the Graphite team.

Thanks!