Closed gundalow closed 2 years ago
The import
sanity test has been extended to cover plugins. Previously it only covered modules.
There are several scenarios which are likely to cause errors to show up in collections as a result of this change:
ImportError
- Just like modules, plugins must now gracefully handle ImportError
for all imports which are not provided by the Python standard library, collection dependencies, Ansible's dependencies, or Ansible itself. For example, a collection plugin that imports kerberos
must do so by wrapping it in a try/except ImportError
handler, just as would be done if it were a module.(See https://docs.ansible.com/ansible/devel/dev_guide/testing/sanity/import.html for more details and examples of how to handle it correctly.)
In Bullhorn 20
pylint
Sanity Test UpdateThe pylint
sanity test in ansible-core's devel branch no longer ignores used-before-assignment
errors (https://github.com/ansible/ansible/pull/73639). These errors are usually not false positives, so if your collection's sanity tests start failing because of this, better look carefully and not just ignore these errors.
In Bullhorn 20
ipaddress
in ansible.netcommon 2.0.0 has been removedIn case your collection uses the vendored ipaddress
module from ansible.netcommon, either directly (import ansible_collections.ansible.netcommon.plugins.module_utils.compat.ipaddress
) or indirectly (import ansible.module_utils.compat.ipaddress
, see ansible-core's module_utils routing): it has been removed from ansible.netcommon 2.0.0 (released 3 days ago) and these imports will no longer work!
a) If you need it for code that only supports Python 3.5 or newer, you can replace it with import ipaddress
(https://docs.python.org/3/library/ipaddress.html);
b) Otherwise, you can (i) rewrite your code to not use it, (ii) vendor it in your collection (example), or (ii) require the user to install it manually for Python 2.6/2.7 (pip install ipaddress).
In Bullhorn 21
devel
: flags module options that might reveal secret dataThere is a new test in validate-modules (ansible/ansible#73508) which reports module options that look like they could contain secret data (by the option name), but do not have no_log
set. Examples are things like password
, key
or token
.
Since the test aims at catching as many real cases as possible, it unfortunately also produces a lot of false positives. DO NOT use ignore-2.11.txt
entries to ignore these! Instead, add no_log=False
to the module option's argument spec entry to flag this specific option as "contains no secret data". Ignoring the error for a file will also disable this check for new module options, and this this a really bad idea!
(See also https://github.com/ansible-collections/overview/issues/45#issuecomment-774472384)
An update to the previous post, since I've now seen that in multiple places: if you have to add no_log=True
somewhere, this is usually a security vulnerability and should be treated as such! That means: it should be properly documented in the changelog (if you're using antsibull-changelog: section security_fixes
), and a new release which fixes them should be released soonish.
If the affected modules have been migrated from other collections, and/or from Ansible 2.9, you should backport these security fixes to the other collections and to Ansible's stable-2.9 branch.
(If you are just marking false-positives with no_log=False
, that's not a vulnerability and doesn't need special attention.)
A new version of the Azure Pipelines test container image is available (1.9.0
). The following collections should update to using the new image. Please check them off once they have been updated to use the new image.
The ansible/ansible repository now has a stable-2.11
branch, and the version of the devel
branch has been bumped to 2.12.0.dev0
. This means that if you have an tests/sanity/ignore-2.11.txt
file in your collection, you need to copy it to tests/sanity/ignore-2.12.txt
. Also if your collection is included in Ansible, please make sure that CI also tests against the new stable-2.11
branch. Ansible 4.0.0 will be based on ansible-core 2.11, and thus your collection must work with stable-2.11
.
(We still recommend to also test against the devel
branch to be informed early about breaking changes. But testing against stable-2.11
is required by the inclusion criteria, now that stable-2.11
exists.)
Goodbye include:
long live import_X/include_X
https://github.com/ansible/ansible/pull/71262
All core lookups were moved to support the plugin config system https://github.com/ansible/ansible/pull/74108 In the future we will deprecate 'custom inline string parsing', for exampe:
lookup('ini', 'key filename=ansible.ini section=whateves')
becomes
lookup('ini', 'key', filename='ansible.ini', section='whateves')
This uses the 'python function argument parsing' and allows for better escaping and less confusion with k=v pairs.
@bcoca can you expand on what you mean by deprecation? Does that apply only to the lookups in core? Or will internal changes somehow force or attempt to force that no lookups can perform "inline string parsing"?
@briantist in the end the plugin can do what it wants. But to be consistent with the documentation/configuration declaration it should not do it's own parsing, so it can deliver predictable results as well as keep common escaping and other input rules.
Also I'm trying to eliminate the miriad of inline string parsers, each with it's own conventions, rules and limitations:
https://github.com/ansible/ansible/blob/stable-2.11/lib/ansible/plugins/lookup/ini.py#L73 https://github.com/ansible/ansible/blob/stable-2.11/lib/ansible/plugins/lookup/first_found.py#L135 https://github.com/ansible/ansible/blob/stable-2.11/lib/ansible/plugins/lookup/csvfile.py#L147
And the 'common' one that most didn't use: https://github.com/ansible/ansible/blob/stable-2.11/lib/ansible/parsing/splitter.py#L49
@briantist in the end the plugin can do what it wants. But to be consistent with the documentation/configuration declaration it should not do it's own parsing, so it can deliver predictable results as well as keep common escaping and other input rules.
Thanks. I ask as someone who migrated a lookup to use config manager and argument parsing, but kept the string parsing for compatibility. Unfortunately I think most users are still using (and prefer) the old style, much to my chagrin. I expect a lot of pushback if I attempt to deprecate it.
But what I like is that this does give me some support for not allowing this in future lookup plugins within the collection. I had been wrestling internally whether or not to support string style for familiarity for end users, even though I'd have preferred not to.
I know deprecation notices in lookups still aren't working (for anyone curious: https://github.com/ansible/ansible/pull/73240 ), but if they were, would this deprecation be detected automatically? It seems like it couldn't be since it's not really done in a common function (as you've shown in your edit).
Edit: I didn't even know there was a common function..
stable-2.11 has been branched
The ansible/ansible repository now has a
stable-2.11
branch, and the version of thedevel
branch has been bumped to2.12.0.dev0
. This means that if you have antests/sanity/ignore-2.11.txt
file in your collection, you need to copy it totests/sanity/ignore-2.12.txt
. Also if your collection is included in Ansible, please make sure that CI also tests against the newstable-2.11
branch. Ansible 4.0.0 will be based on ansible-core 2.11, and thus your collection must work withstable-2.11
.(We still recommend to also test against the
devel
branch to be informed early about breaking changes. But testing againststable-2.11
is required by the inclusion criteria, now thatstable-2.11
exists.)
https://github.com/ansible-collections/dellemc.enterprise_sonic/pull/39
If you maintain a collection that uses ansible-test
to run unit tests, the following changes made to the devel
version of ansible-test
may impact your collection:
modules
must reside under tests/unit/plugins/modules/
to be detected as module unit tests. If they reside in another location, they will be detected as tests for controller-side code, which will prevent them from running under Python 3.7 and earlier.module_utils
must reside under tests/unit/plugins/module_utils/
to be detected as module_utils unit tests. If they reside in another location, they will be detected as tests for controller-side code, which will prevent them from running under Python 3.7 and earlier.modules
or module_utils
tests are no longer able to import from ansible.*
, except for ansible.module_utils.*
.@mattclay
Unit tests detected as
modules
ormodule_utils
tests are no longer able to import fromansible.*
, except foransible.module_utils.*
.
I have a module_utils
unit test that's now failing to import from ansible.plugins import AnsiblePlugin
. It was passing yesterday/earlier today.
Fail: https://github.com/ansible-collections/community.hashi_vault/runs/2385869734?check_suite_focus=true#step:5:461 Line: https://github.com/ansible-collections/community.hashi_vault/blob/main/tests/unit/plugins/module_utils/test_hashi_vault_option_adapter.py#L10
This module_util is intended to be used by both modules and plugins. The tests need to test the plugin functionality to test that (the util itself doesn't import from ansible.*
).
What's the purpose of making this intentionally not work?
@briantist Importing ansible.plugins.*
under Python 3.7 and earlier is no longer supported, since starting with Ansible 2.12 the controller requires Python 3.8 or later. Although doing so does not result in import failures or syntax errors yet, that will be changing very soon as we start taking advantage of newer Python features in the controller code. Since modules and module_utils are expected to work on Python versions older than what the controller supports, this means that unit tests for those cannot depend on code that only works newer Python versions.
If there are module or module_utils unit tests that depend on ansible.plugins.*
or other ansible
code outside of ansible.module_utils
, then there are a few options available:
@mattclay I see, thank you for the explanation.
If I were to
relocate the unit tests so they are executed as controller tests
What would be an appropriate place for that? Is test/unit/plugins/plugin_utils/
recognized as controller-only? That seems like it would be the ideal location.
@briantist There is no standard location for controller-only module/module_utils unit tests. However, if the tests are located any place other than tests/unit/plugins/modules/
or tests/unit/plugins/modules_utils/
they will be considered controller-only. Keep in mind that ansible-test
change detection (--changed
option) will no longer trigger tests for modules or module_utils if they have been placed in a non-standard location.
It looks like community.hashi_vault
supports Python 2.7 and later. Are you planning on continuing to support Python 2.7 and later, while only running unit tests on Python 3.8 and later?
It looks like
community.hashi_vault
supports Python 2.7 and later. Are you planning on continuing to support Python 2.7 and later, while only running unit tests on Python 3.8 and later?
Uncertain. May only support Py 2.7 and later on the Ansible core versions that support it (it would be a little weird to have supported python 2.7+ and suddenly yank it, while still supporting earlier Ansible versions..), so maybe continuing to run the units on earlier py versions, when coupled with earlier core versions... I'll have to see if/how I can do that effectively.
@briantist dropping support for a Python version means you have to release a new major version. As long as you run unit tests also with older Ansible/ansible-base/-core versions (and support these versions with your collection), I wouldn't just drop support because devel
no longer supports them :)
I think we're agreeing @felixfontein ! I don't want to drop support. But I now have to figure out how to split/arrange my unit tests for module utils that are intended to be used both controller and remote side, so that I can run the tests successfully. I'll work on it.. in the meantime my tests are going to be failing.
devel
branchFreeBSD 13.0 is now available for testing in CI (--remote freebsd/13.0) for collections testing with ansible-test from devel
.
freebsd/11.4
will be dropped from ansible-test later in the 2.12 development cycle, so collections testing against devel
are encouraged to target FreeBSD 12 (currently freebsd/12.2
) and 13 (currently freebsd/13.0
) for their tests, dropping 11.
The devel
version of ansible-test
now requires Python 3.8 or later for sanity tests (except compile
and import
).
If you run sanity tests using --docker
(or --docker default
) no changes are necessary. However, if you use the --python
option or your own environment to run using older Python versions, sanity tests will be skipped.
The devel
version of ansible-test
now supports an optional collection-specific configuration file in tests/config.yml
.
Currently there is one configuration option available to declare the Python versions a collection supports for testing.
For example, if a collection requires Python 3.6 or later to run modules, the following configuration could be used:
modules:
python_requires: '>= 3.6'
For collections which consist of modules that depend on ansible-connection
, the following config is recommended:
modules:
python_requires: controller
See the sample configuration file for additional details.
Due to the recent security incident, we now have our own copy of the bash uploader hosted in S3. Update any reference to https://codecov.io/bash
to use the new URL.
Fedora 34 (--docker fedora34
) has been added to ansible-test
on the devel
branch. Update collections to test against Fedora 33 and 34.
Fedora 32 will be going EOL soon, and will be removed from ansible-test
on devel
later in the ansible-core 2.12 development cycle.
Reminder about the ansible release schedule:
Python 3.10 is now supported by the devel
branch version of ansible-test
. The default
container image has been updated to include Python 3.10.0b1.
When testing a collection with ansible-test sanity
or ansible-test units
, be aware of the following:
--python
option, tests will automatically be run using Python 3.10. You may see new test failures under this version of Python that will require resolution.--python
option, be sure to update your test configuration to include Python 3.10.Periodically throughout the Ansible 2.12 development cycle, the default
container will be updated to include newer pre-release versions of Python 3.10, until the final version has been released. The final version of Python 3.10 is currently expected on October 4th -- the same day as Ansible 2.12 RC1.
All, Just a heads up that there will be some maintenance happening on ansible.galaxy.com this Saturday at 0800 UTC lasting up to an hour. We don't anticipate any downtime, though there is always the risk.
If you are interested in the future of Galaxy we have a short survey to help us prioritise the feature work, please let us know your thoughts via short survey (background)
Thanks
That should say Saturday 19th for galaxy.ansible.com maintenance, updated.
Be aware that Ansible Galaxy unexpectedly made a change to require mandatory requires_ansible
in meta/runtime.yml
file for uploading collection[1][2]. And after you add it you'll probably will hit ansible-test
failure in sanity checks if you use it with --venv
option, since it can't check this field because of failing requirement[3].
Worth to mention you must have meta/runtime.yml
now for your collection to be uploaded.
[1] https://github.com/ansible/galaxy-importer/pull/122 [2] https://github.com/ansible/galaxy-importer/pull/124 [3] https://github.com/ansible/ansible/issues/75353
(Added to Bullhorn 31)
The default
container image has been updated to include Python 3.10.0b4
Projects using Azure Pipelines to test against the ansible-core
devel
branch have been checked to verify their test matrix is up-to-date.
A report on each project's status, as well as the required and recommended actions are explained below.
Each project is given a status as follows:
Skipped
- The project was skipped because it does not use platforms that were evaluated. No action is necessary.Current
- The project uses platforms that were evaluated, and all are current. No action is necessary.Update
- The project uses platforms that were evaluated, and one or more changes are indicated.The types of changes are as follows:
Add
- The platform is already tested by the project, but the version is not. Add the version to the matrix.Remove
- The platform and version are deprecated and will be removed from ansible-test
in the future. Remove the version from the matrix.Consider
- The platform is not yet tested by the project. Consider adding the version to the matrix.IMPORTANT: These changes should only be made for the portion of the test matrix tested against the
devel
branch ofansible-core
.
^ Related issue https://github.com/ansible-collections/overview/issues/179
The PR adding RHEL 8.4 to ansible-test has been merged. If you were seeing failures as a result of adding/trying to add RHEL 8.4, they should be resolved now.
Ansible-test's validate-module sanity check in ansible-core's devel branch has a new feature which checks that _info and _facts modules have supports_check_mode=True
(https://github.com/ansible/ansible/pull/75324). (See Ansible's conventions which mandate that _info and _facts modules should do that.)
Please make sure to not use Fix
with this issue following in your PRs. If you (or the person/bot merging) has sufficient rights, this issue will be closed. Thanks :)
ansible.posix:main has been fixed in ansible-collections/ansible.posix#237 like below:
The pylint
version in ansible-test
has been updated to 2.9.3.
As a result, new pylint
errors may appear when testing collections against the devel
version of ansible-test
.
We are happy to announce that the registration (free) for the Ansible Contributor Summit is open.
Which day should you attend?
If you are wondering what it means to "contribute" to Ansible, or if you have been using Ansible for a while and are thinking of ways to get more involved with the project and community, then Day 1 is for you!
Among other topics, there will be:
If you have already been contributing or participating in our previous Contributor Summits, you are still welcome to attend Day 1, however we think that Day 2 will be more interesting for you.
Among other topics, there will be:
In addition, we will be running a Hackathon throughout the week.
More details will be available soon. Please register here so that you will be informed of the updates!
See you at the summit!
The devel
version of ansible-test
(part of the upcoming ansible-core
2.12 release) now supports integration testing with separate hosts and Python versions for the controller and target.
If you maintain a collection which runs tests using Python 3.7 or earlier, please pay special attention to the following, as your tests will be required to run in a "split" configuration involving two separate Python interpreters, possibly on separate hosts.
Collection authors may need to update their tests to work correctly with split testing. Most issues will be due to tests assuming that the controller and target are the same host, such as:
delegate_to: localhost
.delegate_to: localhost
(or other forms of localhost
access) when intending to connect to the target.There are also other things that are likely to cause problems with tests, such as:
pip
CLI instead of the Ansible pip
module. If the pip
CLI must be used, use one of the following:
python -m pip
{{ ansible_playbook_python }} -m pip
{{ ansible_python_interpreter }} -m pip.__main__
{plugin_type}_{plugin_name}
naming convention for integration test targets for controller plugins.cryptography
or pyopenssl
on the target without constraining the versions installed.ansible-core devel (part of the upcoming ansible-core 2.12 release) requires Python3.8+ on the controller now. In addition to a simple version check that provides a friendly-ish error, we've removed a number of places with obvious Py2 compat, and made a few Python3.8+ code changes.
The base
and default
containers for ansible-test
will be updated several times this week ahead of code freeze:
ansible-test
. If your collection has been relying on pre-installed requirements in the default
container, instead of declaring/installing them, then tests may fail as a result of the change.On Monday, after creation of the stable-2.12
branch, support for Python 2.6 will be removed from the ansible devel
branch.
On Thursday, pip constraints not used by ansible-test
will be removed from ansible-test
in the devel
branch.
Collections which depend on these constraints to install their requirements will need to provide them instead of depending on ansible-test
to do so. The constraints which will be removed can be found here: https://github.com/ansible/ansible/pull/75763
The core team made a decision earlier this week to change the point when we create the stable-2.12
branch to happen at Feature Freeze. This change was made to accommodate the core team no longer performing releases (another team at Red Hat now performs all of the core upstream and downstream releases), and due to a bug in GitHub that prevents us from enabling and disabling branch protections.
With the change in schedule, we've also decided for this time only to delay bumping the version in devel to 2.13 at stable branching until Wednesday September 29.
We're planning some revised guidance for devel testing in CI in the near future, and once that has become more well defined we will provide that information.
If you maintain a collection that is tested against the devel
branch with ansible-test sanity
, be sure to make a copy of your tests/sanity/ignore-2.12.txt
file as tests/sanity/ignore-2.13.txt
before September 29th to avoid test failures. Keep the two files in sync until the version in devel
changes, at which point only the 2.13 file should be updated for devel
.
Reminder - Day 1 of the Ansible Contributor Summit and the Hackathon start tomorrow (Tuesday, 28 September 2021)!
For more info and the latest updates, please see the Ansible Contributor Summit 2021.09 HackMD.
Reminder - Day 2 of the Ansible Contributor Summit and the last day of the Hackathon start tomorrow (Friday, 1 October 2021)!
For more info and the latest updates, please see the Ansible Contributor Summit 2021.09 HackMD.
ansible's devel branch's version got bumped to 2.13.0.dev0. So if you didn't copy tests/sanity/ignore-2.12.txt to tests/sanity/ignore-2.13.txt (in case that file had contents), you'll notice soon if you're testing against devel :)
Removal of Python 2.6 support from the devel
branch is expected to be completed today. This was previously announced for Monday, but was delayed until the version bump in devel
occurred.
Collection maintainers may need to make the following changes once the removal is complete:
ignore-2.13.txt
file, you will need to remove them.
NO LONGER IN USE
Please subscribe to ansible-collections/news-for-maintainers
Details of what's changed
In accordance with the Community decision, we have created the news-for-maintainers repository for announcements of changes impacting collection maintainers (see the examples) instead of this issue.
Watch
button in the upper right corner on the repository's home page.Issues
.