Closed gundalow closed 2 years ago
tests/sanity/requirements.txt
from collectionsThe file tests/sanity/requirements.txt
was put into collections by mistake, it should be removed.
See #43 for details
Update CI to run against RHEL 7.8/
See #44 for details
Feedback needed
We have a continuing need for changelogs (or release notes) to cover:
There are various proposals outlined https://github.com/ansible-collections/overview/issues/18 Please comment and vote for preferred solution.
Update CI to run against RHEL 8.2.
See #54 for details.
Use meta/routing.yml for deprecated and removed status.
Do collection owners want a DOCUMENTATION field to cover preview vs stable status for a module??
See https://github.com/ansible-collections/overview/issues/57 for details.
All, we've just merged https://github.com/ansible/ansible/pull/68646 This means that deprecated modules and plugins
_
meta/routing.yml
(see community.general for an example)Update CI to run against Fedora 32.
See #59 for details.
ANSIBLE_METADATA
In Ansible 2.9 and earlier we had the concept of ANSIBLE_METADATA
, to express support level and deprecation. With the move to Collections, and support being defined if a Collection is in Galaxy vs Automation Hub, ANSIBLE_METADATA is no longer needed.See https://github.com/ansible-collections/overview/issues/57 for background
Please bulk remove the ANSIBLE_METADATA
section from all modules in your collection and reference this issue in the PR so we can track what's been done.
[X] community.general https://github.com/ansible-collections/community.general/pull/339
[x] community.network https://github.com/ansible-collections/community.network/pull/35
[x] community.aws https://github.com/ansible-collections/community.aws/pull/70?
[ ] your collection...
The devel
version of ansible-test now validates removed_in_version
and deprecated_aliases
(ansible/ansible#66920). For collections, these are assumed to be collection version numbers, i.e. in semver format. Version numbers not in that format - for example verison numbers taken from ansible/ansible during the migration - are not valid semver versions and thus fail validation.
Please ensure that deprecation version numbers are valid collection version numbers.
Alternatively, if you do not have a deprecation / versioning plan yet (and thus no idea how to rewrite Ansible versions to collection versions), temporarily update tests/sanity/ignore-2.10.txt
as in ansible-collections/community.general#337.
[x] community.general
[x] community.network
[ ] community.aws
[ ] ansible....
[ ] partner1....
We're going to set the date for our next Contributor Summit soon, please let us know what works best for you! https://doodle.com/poll/d8ttii3zpv5a2ic5
The devel
version of ansible-test
has been upgraded to use the latest version of the default test container.
This may result in new test failures.
There are several issues that are likely to occur:
--docker
or --docker default
options for testing your collection, re-run all tests to check for new issues.ansible-test
commands use different files:
tests/unit/requirements.txt
tests/integration/requirements.txt
tests/integration/integration.requirements.txt
tests/integration/windows-integration.requirements.txt
tests/integration/network-integration.requirements.txt
Before migration there were test groups configured for ansible/ansible
on Shippable that used "smoke tests" as an optimization when all tests were triggered. These groups were used for tests such as Windows, AWS, etc.
These smoke tests groups do not make sense for collections, and their presence could result in too few (or possibly no) tests running on those collections when certain changes are made.
If you are the maintainer of a collection which was migrated from ansible/ansible
and are using copies of the original Shippable scripts (tests/utils/shippable/*.sh
) you should remove smoke test configuration from your collection:
1) Remove CI groups in tests/integration/targets/*/aliases
which contain smoketest
.
2) Update scripts in tests/utils/shippable/*.sh
to remove smoketest
logic. Look for the string smoketest
as well as for use of the --changed-all-target
option for ansible-test
.
The next PR day will be Wednesday 17th June 2020 from 1400UTC
See https://github.com/ansible/community/issues/407#issuecomment-640223103 for further info. Hope to see you there!
The default
test container for ansible-test
has been reduced from a compressed size of about 1.6 GB to 700 MB. This was partially accomplished by removing pre-installed Python packages that are not required by ansible-test
.
Some collection unit and integration tests may have been depending on those pre-installed requirements. As a result, some tests may now fail or be skipped.
If you are the maintainer of a collection and are using ansible-test
to run unit or integration tests, make sure to declare Python test requirements in your collections:
tests/integration/requirements.txt
tests/unit/requirements.txt
Separate requirements can be used for each type of integration test if required:
tests/integration/integration.requirements.txt
tests/integration/windows-integration.requirements.txt
tests/integration/network-integration.requirements.txt
When any of these files are present, ansible-test
will use pip
to install them if the --docker
, --venv
or --requirements
options are used.
When today (AFAIK) ansible-base's stable-2.10
branch will be forked from devel
, and devel
's version bumped to 2.11.0.dev0, ansible-test sanity
will look for ignore entries in tests/sanity/ignore-2.11.txt
and no longer in tests/sanity/ignore-2.10.txt
. (The ansible-test version from stable-2.10
will use that file.)
So in case you have entries in tests/sanity/ignore-2.10.txt
and test with devel, sanity tests will start failing until you copy tests/sanity/ignore-2.10.txt
to tests/sanity/ignore-2.11.txt
.
The ansible/ansible existing 2.10 porting guide has entries that really belong with the individual collections. Similarly, changelog fragments are being removed from ansible/ansible if they are not for code within the ansible/ansible repo.
Please move any changelogs or porting guide entries from ansible/ansible to your collection if you haven't already.
I cannot edit the above comment, however please see https://github.com/ansible/ansible/pull/68498 as well for removed changelog fragments (it was the first and largest)
Here's a summary of how deprecation by version or by date works on various levels. The collection name used in ansible-base (i.e. ansible/ansible) is ansible.builtin
.
deprecated
entry in DOCUMENTATION
with:
why
, alternative
(required)removed_in: <version>
or removed_at_date: <date>
removed_from_collection
is automatically inserted if not present)deprecation
entry in meta/runtime.yml
with:
warning_text
(optional)removal_version: <version>
or removal_date: <date>
DOCUMENTATION
(for option itself, for ini
entry, for vars
entry, for env
entry)
why
(required)alternative
(optional)version: <version>
or date: <date>
collection_name
is automatically inserted if not present)removed_in_version="<version>"
or removed_at_date="<date>"
removed_from_collection="<namespace.name>"
(required)name="<alias>"
(required)collection_name="<namespace.name>"
(required)version="<version>"
or date="<date>"
Display.deprecated()
and AnsibleModule.deprecate()
calls
msg
(required)version="<version>"
or date="<date>"
collection_name="<namespace.name>"
(required)Also, stable-2.10 has been branched.
2 things I've come across in the Windows collections that is probably a good idea for others to check
tests/unit/requirements.txt
file generated from migration had a lot of packages that are not needed for every collectiontests/utils/shippable/check_matrix.py
if copied from ansible/ansible
would have the wrong repo to check in the code
community.general
has donethe M()
and :seealso:
sections in each module DOCUMENTATION block need to use FQCN.
Please update M() and :seealso: to include FQCN.
For modules in a collection: M(community.general.apt_rpm)
For modules in `ansible-base: M(ansible.builtin.yum)
As you may have heard, we have decided to make AnsibleFest 2020 a virtual experience. We hope to use this opportunity to collaborate, connect, and chat with more automation fans than ever before. It is exciting to think about how many more people will be able to join in on the automation conversation.
AnsibleFest Virtual Experience will be a free, immersive, multi day event on 13-14 October, 2020, that will deliver empowering content from customer keynotes, breakout sessions, access to Ansible experts, and more.
This is a great opportunity to share your automation story with thousands of open source enthusiasts and technology experts.
On Monday 12th October (to be confirmed) we host another virtual Contributors Summit as part of AnsibleFest, more details on that closer to the time.
We'd love to hear from the community, especially those like yourselves that have started their journey with collections.
Call for proposals is now open!
Submit your automation story here by 15th July 2020.
You can see previous videos and slides here
version_added
in collectionsPlease note that version_added
in a collection always refers to the collection's version numbers (in particluar, they must be of the form x.y.z
)! So please do not use version_added: "2.10"
in collections, but instead put version_added: <version of your collection the feature was added>
in there.
(If you absolutely want to refer to a version of another collection, you can add version_added_collection: namespace.name
next to version_added
. The name of ansible-base is ansible.builtin
.)
version_added
in collections is now better validated (modules only)ansible/ansible#70869 fixed validation of version_added
for modules in collections: now also the top-level version_added
is checked, and the correct error codes are returned when version_added
fails validation for (sub-)options and (sub-)return values.
We've published the tentative release schedule for the ansible-2.10 package: https://groups.google.com/forum/#!topic/ansible-devel/srweNQ92pJw Important dates from that schedule:
Please see the full post to know what's expected by those dates.
In the future this information will live on docs.ansible.com.
The 2020-08-18 date is of particular importance. This means that by that day, you MUST have released 1.0.0 for it to appear in Ansible 2.10. If you only have a 0.x.y release by then, we will not automatically include a newer version since according to semver, anything can happen and no stability guarantees are made. So please make sure that you released 1.0.0 by then.
We've just introduced an Alpine3 distro-test-container for ansible/ansible, and included it our CI matrix. This container may be of use in collections as well. One important note is that we tried to not install many packages that replace functionality in BusyBox, so we get a better test matrix against BusyBox.
Since I've seen that in quite a few collections now: if you use ansible-test sanity --docker -v --color --python 3.6
for running sanity tests (and nothing else), you must remove the --python 3.6
part! The parameter causes that several tests are not run (like compile / import tests for Python versions that are not 3.6). Only use the --python
parameter to ansible-test sanity --docker
if you know exactly what it does, and you are sure that you are calling ansible-test sanity
multiple times to really cover all tests!
Minor change to the release schedule. We're going to leave feature freeze for ansible-2.10.0 development until ansible-2.10.0beta1 is released next week. If you have features you still want to get in to your collection in time for 2.10.0, you still have time to upload them to galaxy.ansible.com. If this causes you issues, please let me know. (abadger1999 on irc.freenode.net, tkuratom redhat.com for email)
Some more changes to the pre-release schedule. These are due to the one week slip of ansible-base-2.10.1: https://groups.google.com/forum/#!topic/ansible-devel/618bSHJ7K64
The new dates would be:
08-Sep-2020: ansible-2.10.0b2
15-Sep-2020: ansible-2.10.0rc1 (moved from 10-Sep)
17-Sep-2020: IRC meeting where any blockers should be proposed and discussed.
22-Sep-2020: If no blockers are found, ansible-2.10.0 final will release as previously scheduled.
Some more information on the IRC meeting to decide on blockers for the Ansible-2.10.0 release:
If you find something major enough to block the Ansible release, please add a comment on the Community Meeting Agenda: https://github.com/ansible/community/issues/539 asking us to consider whether the issue is a blocker. If the issue has a ticket open elsewhere, linking to the ticket will be appreciated.
If at all possible, attend the IRC meeting on Thursday, September 17th, 2020, 18:00 UTC to state your case as to why the issue should block the release. The meeting takes place in the #ansible-community channel on irc.freenode.net. If you are new to irc, you can use their online webchat to access freenode: https://webchat.freenode.net/ You'll need to register to participate in our channels: https://freenode.net/kb/answer/registration I highly recommend registering and testing that you can speak in the #ansible-community channel prior to the meeting.
We'll decide at the meeting whether to slip the Ansible-2.10.0 final release for any of the listed blockers.
Important: If the Meeting Agenda does not contain any comments listing potential blockers when the meeting starts on Thursday, we'll end the meeting early. We will wait for five minutes or so to see if anyone proposes a blocker at the start of the meeting itself but otherwise we'll cut the meeting short. So please, if you have a candidate blocker; get your issue listed on the Community Agenda ASAP.
ansible-galaxy login
for role/collection publishersIf you use the ansible-galaxy
CLI to publish roles or collections to Galaxy, you may care about this. The GitHub API that underlies the ansible-galaxy login
command is being removed in November. Without it, users doing role or collection publish
operations via the Galaxy CLI will need to find their Galaxy token interactively in a browser and pass it to the CLI with --token
(as the ability to generate a token from the CLI will no longer function). We're not sure what people do today or how much this option is used out in the world, so please read and weigh in at https://github.com/ansible/ansible/pull/71628 in the next few days if you have an opinion.
It is now possible to test against macOS 10.15 using anisble-test
>=2.10.0. The existing OS X 10.11 images are still available.
ansible-test integration --remote macos/10.15 [test]
ansible-test shell --remote macos/10.15
The macOS testing is now being done on newer hosts, so test execution for macOS 10.15 and OS X 10.11 should be much faster now.
If your collection's integration tests use a copy of prepare_http_tests, you want to look at https://github.com/ansible/ansible/pull/71841/files for how to adjust it to run with macOS 10.15. That PR might also be useful for other changes needed.
ansible-test sanity
validates more semantic versioning properties for collectionsansible/ansible#71679 added more validation of version_added
and removal versions (for deprecations) in collections so that the versions mentiones there satisfy semantic versioning:
version_added
versions must be minor or major releases);Common problems in the past have been:
Please download the tarball of the latest release of your collection to check whether it includes something that shouldn't be there, if you're not sure.
In yesterday's public Ansible Community Meeting the community signed off on the Ansible-3.0.0 release schedule.
Package ansible-core
(previously known as ansible-base
): The Engine plus the builtin
plugins
Package ansible 2.10
The ~70 collections that hold the modules in ansible-2.9. Depends on ansible-core
The rest of this document is about the ansible
package
ansible
package will move to semver (MAJOR.MINOR.PATCH
)ansible
release will be 3.0.0
, no ansible 2.11
releaseFor full details click here
ansible/ansible#72625 added validation for deprecation and tombstone dates and versions in meta/runtime.yml. This should stop you from accidentally forgetting a deprecated redirect scheduled for removal, or adding an invalid removal version (removals must must happen in major releases according to semver).
ansible-test integration
has been updated to no longer pre-install the virtualenv
package on Python 3.x for FreeBSD or RHEL instances provisioned with the --remote
option.
On Python 3.x, it is recommended that you do not install the virtualenv
package as part of your tests. Instead, when running tests on Python 3.x, use the Python built-in venv
module.
Below are recommendations on how to handle potential failures in tests as a result of this change:
virtualenv
directly in shell scripts, switch to using source virtualenv.sh
or source virtualenv-isolated.sh
instead. The former uses --system-site-packages
while the latter does not. These scripts are provided by ansible-test
. Starting with Ansible 2.11 (devel
branch) they support both venv
and virtualenv
venv
support has been suggested for Ansible 2.9 and 2.10. Until that occurs, installing virtualenv
for Ansible 2.9 and 2.10 will serve as a temporary work-aroundANSIBLE_TEST_PREFER_VENV
environment variable to opt-in to using venv
.virtualenv
parameter for the pip
Ansible module, then be sure to specify the virtualenv_command
parameter as well. See the pip integration test for an example.virtualenv
in Ansible command/shell/raw tasks, consider using the pip
Ansible module instead as explained above. Alternatively, use the command search shown in that example to determine the appropriate command to invoke.NOTE: The upcoming macOS 11 image for ansible-test
will not include virtualenv
. Future updates to the Linux test containers used by ansible-test
for Python 3.x may also exclude virtualenv
.
macOS 11.1 with Python 3.9.1 is now available in CI (--remote macos/11.1
). As mentioned previously, it does not contain virtualenv
.
If you are testing Ansible 2.10 or later, including devel
, please switch to testing with macOS 11.1.
For Ansible 2.9, continue to use osx/10.11
. We do not plan to fix tests for stable-2.9
to pass using the macos/11.1
image.
devel
branchFreeBSD 11.4 and FreeBSD 12.2 are now both available for testing in CI (--remote freebsd/11.4
and --remote freebsd/12.2
respectively), for those using ansible-test
from the devel
branch.
devel
branchUbuntu 20.04 (--docker ubuntu2004
) is now available for use with ansible-test
on the devel
branch.
Note that due to packaging issues with python3-virtualenv
in Ubuntu 20.04, users may find that some tests fail if the tests use virtualenv. In this case, users are referred to this announcement which provides useful information on how to resolve these failures.
rstcheck
sanity test no longer used for collectionsThe rstcheck
sanity test in ansible-test
(devel
branch) is now limited to running against ansible/ansible and is no longer used for collections. See https://github.com/ansible/ansible/pull/73391 for additional details.
Fedora 30 and 31 are unsupported and have reached end-of-life.
If your collection is tested against either of these releases, update your testing to use Fedora 32 and 33 instead.
Support for these older releases will be dropped from the devel version of ansible-test in the future (timeframe TBD).
no_log=True
!I created a WIP PR for ansible-test's validate-modules (based on a branch by @gundalow) which looks for module options that potentially should have had no_log=True
set to prevent secrets from showing up in log files: https://github.com/ansible/ansible/pull/73508 Please run this PR on your collections (install ansible-core from that branch, run ansible-test sanity --docker -v --test validate-modules
) to see if it spots something that should be hidden. If you have suggestions of both how to broaden the option names to search for, and how to prevent more false-positives, please mention them in the PR!
Please note that I know that the PR produces a certain number of false positives. This is annoying, but I think it's more important to catch more information leaks. Also, false positives can be marked with no_log=False
to indicate "I know the name looks like it is a secret, but it is definitely not". This shuts off the warning for a specific option without needing an ignore.txt entry (which would disable the warning for all options in the module).
If you're wondering whether something like that is needed for plugins: not really, since plugins have no mechanism to mark options as secret - mostly since there's no automatic code which could accidental log them. The plugin itself is responsible for making sure they are not logged (or leaked in another way). AnsibleModule
usually logs the module call with all parameters; for example, on one of my servers, grepping for ansible
in the syslog quickly found:
Feb 05 16:02:56 hostname.com.example.com ansible-ansible.legacy.stat[9280]: Invoked with path=/path/to/file follow=False get_checksum=True checksum_algorithm=sha1 get_md5=False get_mime=True get_attributes=True
If a module option is marked with no_log=True
, it will not be dumped here. If you have an option secret_api_key
which is not marked as no_log=True
, the secret API key provided by the user ends up in the log (if the user doesn't mark the whole task with no_log: true
). So it's really important that modules use no_log=True
for everything that shouldn't be logged.
Hey, the azure pipelines container has been updated to include an additional dependency and will speed up runs. This change is purely for performance.
Please change the entry
resources:
containers:
- container: default
image: quay.io/ansible/azure-pipelines-test-container:1.7.1
to
resources:
containers:
- container: default
image: quay.io/ansible/azure-pipelines-test-container:1.8.0
in your .azure-pipelines/azure-pipelines.yml
.
See the report below for updates that may be required for collections tested on Azure Pipelines.
The entries marked Remove
must be completed by March 1st to avoid test failures.
Projects using Azure Pipelines to test against the ansible-core
devel
branch have been checked to verify their test matrix is up-to-date.
A report on each project's status, as well as the required and recommended actions are explained below.
Each project is given a status as follows:
Skipped
- The project was skipped because it does not use platforms that were evaluated. No action is necessary.Current
- The project uses platforms that were evaluated, and all are current. No action is necessary.Update
- The project uses platforms that were evaluated, and one or more changes are indicated.The types of changes are as follows:
Add
- The platform is already tested by the project, but the version is not. Add the version to the matrix.Remove
- The platform and version are deprecated and will be removed from ansible-test
on March 1st. Remove the version from the matrix.Consider
- The platform is not yet tested by the project. Consider adding the version to the matrix.IMPORTANT: These changes should only be made for the portion of the test matrix tested against the
devel
branch ofansible-core
.
ansible-test
with --docker
If your collection is tested using https://dev.azure.com/ansible/ see the previous announcement instead.
If your collection is tested with ansible-test
using the --docker
option, continue reading.
IMPORTANT: These changes only affect tests against the Ansible devel
and upcoming stable-2.11
branches.
On March 1st, the following containers will no longer be supported by the --docker
option:
If your collection is tested against any of these containers, please update your tests to use the following instead:
These changes must be completed by March 1st for any tests run against the Ansible devel
branch to avoid test failures. Changes should not be made for tests run against existing Ansible stable-X
branches.
pylint
Sanity TestThe pylint
sanity test for ansible-test
has been updated to pylint
2.6.0 in the devel
branch. This is the latest version of pylint
and it brings with it several changes:
As a result of this update you may begin seeing new sanity test failures. These issues should be resolved, or as a temporary measure, ignores should be added.
If you encounter any of the following issues, please open an issue in the https://github.com/ansible/ansible/ repo so they can be addressed:
NO LONGER IN USE
Please subscribe to ansible-collections/news-for-maintainers
Details of what's changed
In accordance with the Community decision, we have created the news-for-maintainers repository for announcements of changes impacting collection maintainers (see the examples) instead of this issue.
Watch
button in the upper right corner on the repository's home page.Issues
.