Tox Updates

[c-ryan-k]

This overhauls our existing tox.ini to support the following local and pipeline test infrastructure. I highly advise giving their docs a quick overview, as it is much more in depth than this preview will cover.

Local Tox

This changes nothing about how you currently run tests - local pytest and virtualenvs are untouched and unrelated. In order to perform any of these new tasks, you must install tox (currently in dev_requirements, so already part of dev setup) and have the azure-cli repo cloned alongside your extension repo.

It supports local test configurations for the following environments (passed to tox with -e "env"):

• Linting
  • In order to run flake8 and pylint locally on your code, simply run tox -e lint
• Various Python and AZ CLI Versions
  • The tox environment string (passed to -e) will be parsed as such: py{thon,3.7,3.8,3.9,3.10}-az{min,cur,dev}-{int,unit}
  • Multiple options can be used inside of each {matrix} section (or just pick one): tox -e "py{3.7,3.10}-az{min,dev}-unit"
  • Current CLI version support is as follows:
  • azmin installs the minimum supported CLI version (currently 2.32.0)
  • azcur installs the latest released CLI version from PyPi
  • azdev installs the CLI from your local (../azure-cli) CLI instance
  • In order to list all environments, you can type tox -av, which will display them all in a list:

If you choose not to select a specific python version (which is also the current default), you can use python instead, to invoke whichever version python invokes in your environment.

Important notes

• If you simply run tox, the current default environments (which can be changed inside tox.ini) are:
  • lint: which runs flake8 and pylint
  • python-azdev-unit: which runs unit test only, on your default python version, against your local (pathed to ../azure-cli) CLI core version.
• The first time you run a new environment in tox, it will perform some setup tasks and dependency installation. It can look for changes in tox.ini and other related settings, but in order to rebuild a local environment you need to run tox with the -r switch.

Tox Pipeline

This also introduces a new tox pipeline (more work to come), which adds switches for each python version and azure CLI version, and distributes those tasks to simultaneous jobs:

This is currently a standalone pipeline, but the goal is to eventually configure this pipeline to utilize the existing built wheel and test SDK from our current pipelines to ensure our tests pass on all possible python and Azure CLI versions with very minimal user configuration.

---

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Thank you for contributing to the IoT extension!

This checklist is used to make sure that common guidelines for a pull request are followed.

General Guidelines

Intent for Production

• [ ] It is expected that pull requests made to default or core branches such as dev or main are of production grade. Corollary to this, any merged contributions to these branches may be deployed in a public release at any given time. By checking this box, you agree and commit to the expected production quality of code.

Basic expectations

• [ ] If introducing new functionality or modified behavior, are they backed by unit and/or integration tests?
• [ ] In the same context as above are command names and their parameter definitions accurate? Do help docs have sufficient content?
• [ ] Have all the relevant unit and integration tests pass? i.e. pytest <project root> -vv. Please provide evidence in the form of a screenshot showing a succesful run of tests locally OR a link to a test pipeline that has been run against the change-set.
• [ ] Have linter checks passed using the .pylintrc and .flake8 rules? Look at the CI scripts for example usage.
• [ ] Have extraneous print or debug statements, commented out code-blocks or code-statements (if any) been removed from the surface area of changes?
• [ ] Have you made an entry in HISTORY.rst which concisely explains your user-facing feature or change?

Azure IoT CLI maintainers reserve the right to enforce any of the outlined expectations.

A PR is considered ready for review when all basic expectations have been met (or do not apply).

[c-ryan-k]

Here's an example of the tox pipeline running

[vilit1]

do we need to update dev_requirements (ex: add tox)

Also would be nice if you can put the pr description into the page we have about testing

[c-ryan-k]

do we need to update dev_requirements (ex: add tox)

Also would be nice if you can put the pr description into the page we have about testing

tox is already in our dev_requirements - but I can absolutely work on a "tox testing guide", with this PR description as a base (and some examples of course)

[digimaun]

I'm thrilled about this contribution

[digimaun]

do we need to update dev_requirements (ex: add tox) Also would be nice if you can put the pr description into the page we have about testing

tox is already in our dev_requirements - but I can absolutely work on a "tox testing guide", with this PR description as a base (and some examples of course)

Will that tox testing guide be part of this PR or a follow up PR?