Create a CI or other infrastructure for measuring D's progress and performance

JinShil commented 5 years ago

Description

We need to be able to measure how the compiler and the standard library perform over time. This project would create a CI, or other infrastructure, that would monitor metrics like compile-time, compiler's memory consumption, artifact's runtime performance, artifact's memory consumption, artifact's binary size, etc., so that so we can measure, quantitatively, D's progress over time.

It could even be expanded to include other non-performance metrics such as numbers of contributors, top contributors, lines of code changed per unit time, etc.

What are rough milestones of this project?

The project implementer is encouraged to propose their own ideas and vision for this project.

How does this project help the D community?

This project would help prioritize work and measure, quantitatively, D's progress as a project.

Recommended skills

The project implementer should have some skills benchmarking software
The project implementer should have some skills creating and using charts and other forms of data visualization to convey information at a glance

Point of Contact

@JinShil can be contacted to further elaborate on the project's goals. However, there is significant artistic license granted to those willing to do the work.

References

Mozilla's Are We Sim Yet
D's AutoTester Metrics - These are not performance metrics but they are an example of kinds of visuals that would help D, as a project, measure its progress.

wilzbach commented 5 years ago

@wilzbach would be happy to help out on question regarding this project too.

JinShil commented 5 years ago

A similar project for rust

Geod24 commented 5 years ago

There's https://blog.thecybershadow.net/2015/05/05/is-d-slim-yet/ but I don't know how well it's maintained.

dlang / project-ideas