Add bootstrapping and tagging facilities

[vkottler]

https://github.com/project-81/firmware/issues/5

We have evolved some system bootstrapping things from shell scripts to Python, now we need to put that Python in ~one place.

Also hit: https://github.com/vkottler/vmklib/issues/62

edit should probably always be a project-specific task, that tags things appropriately

[ashishnagar19]

I think you should make a package which contains all your scripts and then, when you need it you can install it using pip. Also, just make a requirement.txt and add this package in it. So that system just install requirements.

[vkottler]

@ashishnagar19 hello! Thanks for the input. What you're describing is precisely what this package is - it exposes extra/magic make-like behavior by including a bunch of handy targets: https://github.com/vkottler/vmklib/blob/master/vmklib/data/conf.mk (this file includes others)

I wasn't satisfied with that because make is not great at cross-platform tasks (e.g. Windows vs. Unix paths). It also registers and exposes targets that are implemented in Python.

For a typical Python project, my workflow is essentially:

• mk python-lint - runs a number of linters in parallel (pylint, flake8, ruff, isort, black)
• mk python-format - runs isort and black (not at the same time!) and tells them to apply changes
• mk python-test - runs pytest, runs all tests
• mk python-test-<some pattern> runs tests that match some pattern

Some other handy ones like mk python-edit generate tags with ctags and then opens EDITOR + sources the local virtual environment.

You can see some of these targets called out in the CI definition: https://github.com/vkottler/vmklib/blob/master/.github/workflows/python-package.yml

Because a lot of this ends up as boilerplate common per-project, I generate a lot of project files with these templates: https://github.com/vkottler/config/tree/master/python/templates

I also have a functional project template that uses this strategy: https://github.com/vkottler/python-package-template

For this issue that I cut, I'm looking to add even more capability to this. Such as being able to clone git repositories and build projects that loosely follow some kind of ./configure setup.

[vkottler]

This package + datazen are allowing me to manage ~8 public pypi packages, that are all tested on Windows + Mac + Linux, across all relevant versions of Python (3.7-11).

[vkottler]

Task for tagging implemented in https://github.com/vkottler/experimental-lowqa/blob/master/experimental_lowqa/edit.py#L35 (+ in general that repository can host additional workflow tasks).