Add more shards and apps

straight-shoota commented 3 years ago

I'd like to add more projects to improve the surface area of tests. We've already discovered two probable regressions in 1.1.0 from the invidious project. So I think that would be a good candidate to add here, for example.

Criteria to include projects here are: Because they are important for the Crystal ecosystem, a large code base (thus using many features) or using specific features of the compiler or standard library. And of course it should be reasonably easy to run test (ideally just checkout and run make test).

Having projects run their own specs on nightly and pre-release builds of Crystal is of course also valuable. I'd encourage every maintainer to do that (it's really easy on GHA with https://github.com/crystal-lang/install-crystal btw.)

Some proposals:

Please add more suggestions.

beta-ziliani commented 3 years ago

Well, we need to be sure that maintainers will want our PRs if we find an issue in their code. I mean, some breakages might be good. I say this because I don't know how maintained are some of the listed projects.

beta-ziliani commented 3 years ago

We can always contact the maintainers in doubt. I'm not sure how active is Lilith, for instance, but I understand that this project might shake the language in a way few other projects might do.

straight-shoota commented 3 years ago

Yeah, I don't see the purpose of test-ecosystem to make sure these specific libraries work well with an upcoming release. For me it's primarily about identifying unintended breaking changes.

beta-ziliani commented 3 years ago

The thing is what we do if we break something for a good reason. We need to be able to tell the developers how to fix their code (and expect the library to progress, otherwise it has to be removed from the list).

beta-ziliani commented 3 years ago

Let remove lilith from the list, it won't be updated and it doesn't compile now.

crystal-lang / test-ecosystem

Add more shards and apps #5