add requirements management policy

[dhellmann]

This document describes the policies for adding and updating build and runtime dependencies of all InstructLab components.

[dhellmann]

FYI @n1hility @dhellmann @tiran @russellb @mairin

[tiran]

Several months ago, I experiemented with https://pypi.org/project/pip-compile-multi/ to generate a set of constraint files for our entire dependency chain. Yesterday I revived my old PR and tried again. pip-compile-multi breaks because packages like flash-attn have broken packaging metadata. Doug filed a ticket a while ago, but upstream hasn't agreed to accept the fix.

Related InstructLab ticket and PR to constraint versions:

• https://github.com/instructlab/instructlab/issues/1851
• https://github.com/instructlab/instructlab/pull/1863

[tiran]

The topic of version bounds came up in internal channels. The blog post https://iscinumpy.dev/post/bound-version-constraints/ explains very well how to deal with version constraints in Python. It links to several other blog posts about semver and version handling in Python ecosystem, too.

[dhellmann]

Great document, thanks! Do you think it would be beneficial to include a practice of documenting any changes to the dependency requirements? This could include the rationale for setting or changing version ranges, providing context for future maintainers and downstream builds?

That sounds like a good follow-up for someone else to do, yes.

[dhellmann]

EDIT: "squash and merge" is not enabled in this repository so please do it locally and push again, thanks!

Done