TRANSFORM_INVALID_GROUP_CHARS doesn't document valid group patterns

xarses commented 5 years ago

SUMMARY

With the addition of the TRANSFORM_INVALID_GROUP_CHARS. Other than reading the source, it was not clear which characters must be avoided going forward, only that the warning (with -vvvv) points out which characters you currently use that are invalid.

Please clarify that you are pushing the names to be valid python vars. This is missing from the documentation for the cfg option, warning and online documentation

(https://github.com/ansible/ansible/commit/d241794daa6d413e6447890e2a4f11e0d818cf0e#diff-b77962b6b54a830ec373de0602918318R122)

There appears to be no mention of this on https://docs.ansible.com/ansible/latest/porting_guides/porting_guide_2.8.html either.

ISSUE TYPE

Documentation Report

COMPONENT NAME

group

ANSIBLE VERSION

ansible 2.8.0
  config file = /home/awoodward/ansible-skynet/ansible.cfg
  configured module search path = [u'/home/awoodward/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.5 (default, Apr  9 2019, 14:30:50) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]

CONFIGURATION

n/a

OS / ENVIRONMENT

n/a

ADDITIONAL INFORMATION

n/a

bcoca commented 5 years ago

To clarify some misconceptions, part of them were to due to my mistakes and making initial messages unclear, the latest versions have fixes to some of the issues people keep bringing up here, other fixes are still incomming:

Just to say this once, clearly , YOU WILL ALWAYS BE ABLE TO USE DASHES IN GROUP NAMES also dots and other characters that are now considered 'invalid', just not by default. This 'default' is what is being deprecated, the default will be 'safe' in 2.11, but you will always have an option to 'opt in' to the old behaviour.

And to explain how and why we got here:

First, group names were ALWAYS sanitized, they just had different inconsistent rules, depending on the inventory type you were using, scripts were all over the place, YAML and INI formats did each their own thing. The major change was 'centralizing and normalizing sanitation', this was decided back in 2.4 but was not totally implemented until 2.8. The intent was to provide a norm or baseline all could use safely in Ansilbe, that said we recognize there are many people using 'unsafe' or 'invalid' characters for variables so we made this configurable, not only globally but also by some of the inventory plugins.

The initial implementation had some issues and a lot of discussion (no we don't decide this in hiding, we have public meetings in irc to which you are all welcome, https://github.com/ansible/community/blob/master/meetings/README.md) and a lot of the feedback was incorporated (These are also logged so you can go back to see the discussions and reasoning, but to avoid 'log dumpster diving', I'll explain most of issues below). After going out in 2.8 we got another round of feedback and we have been fixing the some bugs, like always getting a deprecation, not just when using the default and specially with the wording of the documentation and warnings.

'Why Python names?' Mostly because Ansible uses Python and JInja (which also uses Python) and some uses of groups (mostly in our early examples, but also plenty of 3rd party ones) can create errors in playbooks, i.e stuff: '{{ groups.gropup-name-with-dash ... }}' or worse a group.name.with.dots. This creates confusion for many users that want to use the Jinja feature of 'dot notation for variable access' and this is why the default should be safe for all users. Most people on this post might disagree with this, but this is a real issue for a lot of people and should not be a 'trap' waiting for new or old Ansible users. Those that 'opt out' are then responsible for avoiding the breaking usage in other parts of Ansible.
'What if I liked that each inventory had different sanitation?' Well, you can still turn off the 'central' sanitation and enable the one for your specific inventory source, most popular new inventory plugins that substitute the old scripts have had options added to emulate the script behaviour, worst case scenario, you can still use the inventory scripts .
'Why not hostnames/are hostnames next?' Like for groups, sanatization of hostnames has always existed, but has not changed, hostnames have a different requirements, like being DNS resolvable for network connections or a valid path for chroot connections. Also, fortunately, there are little to no examples of using hostnames in dot notation, this has not been a common practice and would become a problem if people suddenly started using them, but unlike group names this is something we have avoided until now. If it becomes a problem going forward ... I don't see a good solution either.

Note that this specific ticket (not good enough description/information) is something I'm addressing already, hope to have fix out soon. As for the rest of the discussion, the devs don't use Github as a forum, some tickets devolve into that, the previous ticket that is closed and also has a long thread was ignored until recently, mostly because devs filter out closed issues and expect discussions in IRC the mailing list or new issues.

I hope this addresses all the major issues, as always we are open for discussion, feel free to drop by ML or IRC, we just avoid using github since it is not a good place for such things.

haeringer commented 5 years ago

Thanks a lot for the clarification.

skyscooby commented 5 years ago

Appreciate you taking the time to explain even though it would have been far simpler to stop supporting dot notation and deprecate that support over a few releases. Fewer people use that vs the amount of people who have invalid chars in their group names. Se la vie

bcoca commented 5 years ago

@skyscooby the problem is that it is not Ansible doing that, it is Jinja.

loop-evgeny commented 5 years ago

Just to say this once, clearly , YOU WILL ALWAYS BE ABLE TO USE DASHES IN GROUP NAMES also dots and other characters that are now considered 'invalid', just not by default.

OK, this is good to know, thanks for the clarification. However, the user experience really needs to be improved. You have the "curse of knowledge". Just try to imagine yourselves in the shoes of a user who sees this:

[DEPRECATION WARNING]: The TRANSFORM_INVALID_GROUP_CHARS settings is set to allow bad characters in group names by default, this will change, but still be user configurable on deprecation. This feature will be removed in version 2.10. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg. [WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details

That is a long, long way from

[DEPRECATION WARNING] Group name 'my-servers' contains '-', which will become invalid by default from Ansible 2.11. Set force_valid_group_names in ansible.cfg or the ANSIBLE_TRANSFORM_INVALID_GROUP_CHARS environment variable to "ignore" to suppress this. See https://docs.ansible.com/something for more information.

... which is what I, as a user, would have really wanted to see. It would have saved me over an hour. Now multiply by that by the number of people who have or will run into this issue.

ramilehti commented 5 years ago

Having dashes and dots be invalid in group-names is not a sensible default. People will always have them in their group names. To require them to set yet another variable in a config file to enable sensible behaviour is IMHO indefensible.

Tronde commented 5 years ago

Thanks @bcoca for your comment above. It's much appreciated.

Although I'm not happy with the decision I understand that a discussion took place and a decision was made. If the decision is still open for debate it should be continued at the mailling list or on irc but may not capture this issue.

For the topic of this issue I would like to find the following information in the official documentation and the porting guides, to be aware of this change.

What are group names that are valid by default?
What to do to keep using group names including dashes, hyphens, dots and colons?

Because we use dashes in all our group and host names and we won't change that. So I have to opt-in and change my ansible.cfg every time I setup a new installation/enviroment. That's unfortunate to me but I have to deal with it somehow. The least I would expect is that this is documented in accordingly.

To continue the discussion whether this change is wise or not, I opened a thread in the Ansible Development group.

Best regards,
Tronde

ssbarnea commented 5 years ago

I want to thanks to all contributors on this issue. Based on what I read here I decided to write a blog post https://docs.sbarnea.com/ansible/naming-hosts-and-groups -- Hopefully it would summarize what users need to do.

jctanner commented 5 years ago

@loop-evgeny I agree that we as the core team do have the "curse of knowledge" and it inhibits us from creating docs and errors that are useful to everyone. We also completely rely on the community to help us shape ansible and keep it simple for as many users as possible, so when folks have recommendations on improving our docs and our error/warning messages, I always encourage them to help us out by sending a pullrequest. The message you point out is held in the following file and we would greatly appreciate it if you could send us a PR with the suggested change ...

https://github.com/ansible/ansible/blob/4ef2545eb5d661566e06629015967c2d1b8924e3/lib/ansible/inventory/group.py#L54-L55

loop-evgeny commented 5 years ago

@jctanner Ordinarily I'd be happy to submit a PR to improve a free and useful program that I use. However, the general attitude of the Ansible developers towards usability, their eagerness to close as "working as intended" issues that I consider to be self-evident bugs (even if design bugs) and the fact that Ansible currently has 2025 (that's two thousand!) open PRs gives me very little confidence that my work will not go to waste. If you really want to "rely on the community", as you say, then a substantial culture change is needed IMHO.

mowgli commented 5 years ago

Hmm.. That chance hit me too.

Unfortunately we use network names as group names and that is not easily changable. I would love to opt out for the dot syntax sugar for group names as that is nothing I ever used (Although I use it with other variables).

It would be preferable to use ansible-playbook whatever.yaml -l some.network.to.use in future. Using any other than the network as group name would reduce the usecase massively.

Tronde commented 5 years ago

Hi,
I'm somewhat confused at the moment. Could somebody tell me what I have to set in ansible.cfg to allow invalid chars in group names in the future, please?

equinox0815 commented 5 years ago

force_valid_group_names = ignore

sunshine69 commented 5 years ago

What is the future version of ansible regrading to this issues? Will at some time ansible will reject all dashes in group name without a work around using force_valid_group_names ? (without hearing feedback from users who has suffered by the change, and who has never suffered the issues by using hyphen in group names)

Sorry Just read the comments from @bcoca and happy to see that I would be able to use hyphen if I want to in the future - that is enough for me.

cesarjorgemartinez commented 5 years ago

Hi, I see the same warning, but, I don't understand what I should change, and if we should change it. Is it something related to python? Howto resolve?

If I ignore with force_valid_group_names = ignore, it would be with that necessary, and when I upgrade to Ansible >= 2.10?

Regards, Cesar Jorge

equinox0815 commented 5 years ago

If i understand this correctly. The only thing that is deprecated is the automatic transformation of group names. This means it should be totally fine to set force_valid_group_names = ignore after 2.10 and beyond.

It should also be totally fine to continue to use dashes and whatever you want in group names. What Ansible won't do in the future is to sanatize them so you can use the dotted notation even for "invalid" group names. For example:

Your inventory contains a group named foo-bar.xyz. Now you want to write a template that creates a list of hosts that belong to that group:

{% for host in groups['foo-bar.xyz'] %}
{{ host }}
{% endfor %}

Mind that the following version of the template would not work:

{% for host in groups.foo-bar.xyz %}
{{ host }}
{% endfor %}

This is because the - and the . have special meaning in this case. The dotted notation however would be totally fine if your group would have the name foo_bar_xyz because the template then becomes:

{% for host in groups.foo_bar_xyz %}
{{ host }}
{% endfor %}

Which is of course totally fine.

In an attempt to make things easier for users, Ansible apparently always did some sanitation for group names. This means it was (and until 2.10 still is) possible to use foo_bar_xyz in the above examples even though the group actually is called foo-bar.xyz. Personally i don't think this makes things easier at all and it seems that the core team now also agrees with that. So their next attempt to tackle the issue is to make it impossible to have "invalid" group names in the first place. However as far as i understand it will always be possible to opt-out of this limitation by setting force_valid_group_names = ignore.

Long story short it is actually two different changes that have been intertwined[1] with each other. Whence the confusing name and wording of the warning.

Again this is only how i understand the issue. Please correct me if i'm wrong!

[1] for further details see RFC1925 Paragraph 2, Point (5)

geerlingguy commented 5 years ago

I just wanted to leave my 2¢ since I'm firmly on the side of dashes > underscores. Just doing a quick Google search on the matter, almost universally underscores are labeled as inferior to dashes for any kind of pathing and labeling purposes. They are also more difficult to work with in many text editors and filesystems.

Even if that were not the case, the fact that I see dashes used for things like server labels and groups in the wild a lot more often than underscores means this will be yet another thing I'll have to make sure to add to all my and my clients' ansible.cfg files (I tend to have one per playbook).

I have no problem with Ansible trying to force a stricter default where it improves the experience, but first you came for the dashes in my role names (and sometimes allowed singular exceptions for older roles that were grandfathered in), then you came for dashes in my collections (they are not allowed in any way, shape or form), and now you've come for dashes in my inventory!

It's a war against dashes out there... and I want to draw a line somewhere—in this case, it's the one place where it's actually impossible for me to prevent people from using dashes, as many of the dynamic inventory providers create groups based on server names and labels, and many (if not most) organizations seem to label things using dashes (e.g. us-east-1a, not us_east_1a).

It's not fun having a default that almost always must be overridden to make software work, but it sounds like as of Ansible 2.11, that's going to be the case.

If it's all because some users unfamiliar with Jinja2 and Python don't realize something.with-some-dashes is invalid, I would argue it's better to teach them "if there are dashes, you should use the bracket notation for dict access, e.g. something['with-some-dashes']. You can even intermingle the two if needed. It's not super pure and holistic, but we're not all Rust developers here...

markgoddard commented 5 years ago

Very well put, Jeff. I strongly agree with you here - this change will be so disruptive, and rather than just requiring a one-off migration, will change the workflow of a huge number of users. Ansible will no longer work out of the box.

Hostnames cannot include an underscore, so in a sane world, inventory_hostname would not be forced to. This means that our inventories will now look quite inconsistent, with hostnames that cannot contain underscores, and groups that cannot contain hyphens.

Please, just don't switch the default.

https://en.m.wikipedia.org/wiki/Hostname

Tronde commented 5 years ago

Hi,
I totally agree with Jeff here.

But as @bcoca stated above most of the developers don't look at these discussions here on a regular basis and this issue may not be the right place to discuss the change because it is about the correct documentation.

For discussion please join the thread Is changing the default setting of TRANSFORM_INVALID_GROUP_CHARS a good idea? in Google Groups.

apple4ever commented 5 years ago

Great points Jeff.

It's not fun having a default that almost always must be overridden to make software work, but it sounds like as of Ansible 2.11, that's going to be the case.

This is the big takeaway for me from all these discussions. I understand the problem that needs to be solved, but the solution seems to be the opposite of what is needed. It makes things easier for support but hard for the users - that is a backwards solution.

If it's all because some users unfamiliar with Jinja2 and Python don't realize something.with-some-dashes is invalid, I would argue it's better to teach them "if there are dashes, you should use the bracket notation for dict access, e.g. something['with-some-dashes']. You can even intermingle the two if needed.

This here is the best solution, not breaking things that have been used for years.

nwinkler commented 5 years ago

Great comment from @geerlingguy - spot on!

I'd like to add that as a user of Ansible, why should I need to know what valid Python syntax is? Having used Ansible for a long time now, I understand that Ansible (and its modules) is written in Python, but why should I need to care about that? Exposing that fact to the end user is just bad design.

Similar to only allowing valid JavaScript/Ruby/.NET/whatever notation for things like usernames in a web application. Why would the end user care which language the application is written in?

In addition to that, introducing breaking changes is a difficult topic, I try to avoid that if possible. If I have to make a change, I typically leave in the old, existing behavior as the default, and let people opt-in for the new behavior. Why wasn't this done here? Why do I have to either change my configuration, or worse my whole inventory? Why not the other way around?

ghost commented 5 years ago

If a system requires strictly compliant tokens internally, then the system should internally generate the tokens and create a lookup table that associates the internal tokens with the user data. This way Ansible can change its token rules as needed and limit the impact on the users. Users ought to be able to name their inventory, roles, etc as they or their customers see fit.

tobiasmcnulty commented 5 years ago

It seems to me like this change may have the opposite of its intended effect (to decrease support queries):

Now there is no delimiter supported (by default) both by hostnames (which should be DNS resolvable, i.e., not contain underscores) and group names (which should not contain dashes).

cesarjorgemartinez commented 5 years ago

Definitely should be free to name any hosts

El mié., 14 ago. 2019 16:16, Christian Pointner notifications@github.com escribió:

If i understand this https://github.com/ansible/ansible-documentation/issues/89 correctly. The only thing that is deprecated is the automatic transformation of group names. This means it should be totally fine to set force_valid_group_names = ignore after 2.10 and beyond.

It should also be totally fine to continue to use dashes and whatever you wan't in group names. What Ansible won't do in the future is to sanatize them so you can use the dotted notation even for "invalid" group names. For example:

Your inventory contains a group named foo-bar.xyz. Now you want to write a template that creates a list of hosts that belong to that group:

{% for host in groups['foo-bar.xyz'] %} {{ host }} {% endfor %}

Mind that this version of the template would not work:

{% for host in groups.foo-bar.xyz %} {{ host }} {% endfor %}

This is because the - and the . have special meaning in this case. The dotted notation however would be totally fine if your group would have the name foo_bar_xyz because the template then becomes:

{% for host in groups.foo_bar_xyz %} {{ host }} {% endfor %}

Which is of course totally fine.

In an attempt to make things easier for users, Ansible apparently always did some sanitation for group names. This means it was (and until 2.10 still is) possible to use foo_bar_xyz in the above examples even though the group actually is called foo-bar.xyz. Personally i don't think this makes things easier at all and it seems that the core team now also agrees with that. So their next attempt to tackle the issue is to make it impossible to have "invalid" group names in the first place. However as far as i understand it will always be possible to opt-out of this limitation by setting force_valid_group_names = ignore.

Long story short it is actually two different changes that have been intertwined[1] with each other. Whence the confusing name and wording of the warning.

Again this is only how i understand the issue. Please correct me if i'm wrong!

[1] for further details see RFC1925 http://www.faqs.org/rfcs/rfc1925.html Paragraph 2, Point (5)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ansible/ansible-documentation/issues/89?email_source=notifications&email_token=AA5N2CJCIELW7JWHC6OJ35DQEQHSPA5CNFSM4HPRGLKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4I5U3Y#issuecomment-521263727, or mute the thread https://github.com/notifications/unsubscribe-auth/AA5N2CIVT5PLD2QCAGGBK6LQEQHSPANCNFSM4HPRGLKA .

schrodervictor commented 4 years ago

This is truly a wrong decision in my honest opinion. And for the wrong reason. Reducing the number of support requests, really?

Ansible, as a tool, should not impose language specific details to the end user. I'm already so upset with Terraform imposing me all that Golang down the throat, now the same is happening here with Ansible and the "pythonic" style. Don't get me wrong, I work quite well with both Go and Python, but when it comes to infrastructure as code, why should I care? And what happened with the promise of having YAML dictating the shape of the code base to manage? "Infrastructure as data, that you can read and run", I heard that so many times... As far I know, YAML doesn't care at all with both dashes and underscores.

BTW, there are quite a lot of things out there not supporting underscores. Hostnames, AWS regions and IDs for literally everything, just to mention some really important ones. Good luck with maintaining all the exceptions where the transformation is not supposed to happen...

stefanhorning commented 4 years ago

To people coming here just looking for a quick solution on how to make this go away, just add the line force_valid_group_names = ignore to your ansible.cfg and be happy.

radioactive73 commented 4 years ago

My understanding is you can't use the dot notation for variables with spaces anyways, and while I never create variables with spaces there are unfortunately lots of vendors that return dictionary keys with spaces through json API responses. The sensible option to me seems to switch to the square bracket notation. Hopefully setting force_valid_group_names to ignore doesn't cause ill effects further down the road, who knows what else is planned in the future with this change.

dvntstph commented 4 years ago

This is a pretty dreadful decision particularly when it comes to working with dynamic inventories as with Openstack (and AWS). Instance names and metadata keys containing "forbidden characters" are often returned as inventory items and/or group variables from the underlying cloud. This is going to make life hell for a lot of Openstack (and AWS) admins trying to manage their fleets using meta tags and or instance ID's like: instance-8ca09c33-f255-440f-9544-b0ab318c79d9 meta-os_ubuntu

eightseventhreethree commented 4 years ago

Ansible devs should be taking @geerlingguy opinion seriously. He is one of the biggest contributors to Ansible Galaxy and his Roles are consumed by tons of people. I think this change is really bad for people who have thousands of hosts named like: $env-$role-[0..99]. Are we supposed to go rename everything to appease our Ansible overlords?

nathonfowlie commented 4 years ago

@ssbarnea For one thing, we are making a push to only allow variable names, and other similar keys, that are valid python identifiers. To explain a little further about group names, it causes a problem for users trying to use "dot syntax" such as groups.foo-group, which doesn't do what the user expects. The number of issues and support requests caused by small problems like these has caused us to go down a path to safe guard names to ensure that problems like this do not occur.

For those wanting to keep what we consider invalid characters, can opt out of this functionality.

How long will users be allowed to opt out of this behaviour? Will there be a permanent configuration option to disable this behaviour in all ansible versions moving forward, or will it only be supported until 2.11? I'd be happy for the option to be turned on by default, as long as I always have the option to turn it off.

If this becomes a hard restriction in 2.11+, then you're likely going to find yourself losing customers that are bound by the constraints of their organisation (not all ansible users have the power to dictate the naming conventions used by their company). It seems this change also would introduce a significant challenge for those using ansible to manage cloud infrastructure, where dashes tend to be heavily used.

Tronde commented 4 years ago

Just to remind those who haven't read the whole thread here. There is also a thread on the devel mailling list: https://groups.google.com/forum/#!topic/ansible-devel/bjAcM9ferIw

IMHO this change was a really poor choice. Codebreaking syntax changes in minor release versions holds us back from extending the use of Ansible in our envirionment. Because I won't be able to update Ansible when it breaks the playbooks of my users.

andyfeller commented 4 years ago

But as @bcoca stated above most of the developers don't look at these discussions here on a regular basis and this issue may not be the right place to discuss the change because it is about the correct documentation.

@Tronde : one would argue that contributors AND customers are consulted before stories are written to understand the impact and gather feedback well before someone codes a solution. As several here have mentioned, this is a product mgmt failing we've seen more than once.

Aethylred commented 4 years ago

As an example of the situation @andyfeller describes about this change:

We have a problem with this on our site.

We use Red Hat Identity Manager as an external inventory, we do not control it, it contains many host groups with dashes instead of underscores. This will not be changed (because of all the other things that exist using those names).

So, we need:

To configure Ansible to maintain the current behaviour
Silence the deprecation warning
Do this for command line Ansible and Ansible Tower

geerlingguy commented 4 years ago

FYI PR https://github.com/ansible/ansible/pull/66650 (please no pitchforks there) is slated for 2.10 (as of the current moment), meaning anyone who currently sees this warning would (once they upgrade to 2.10, again assuming that PR is merged) start having playbook failures instead (until they set force_valid_group_names = ignore in an ansible.cfg).

Just posting for visibility. I'm still firmly behind my earlier assertion that this is a user-hostile default, as there are still many dynamic inventory scripts (some part of Ansible itself or now moved into 'official-ish' collections) that generate group names with dashes or other valid DNS characters.

Practically anyone who uses Ansible with AWS is going to have to override the default.

apple4ever commented 4 years ago

@geerlingguy Is that the right PR #? Looks like that points to this issue.

apple4ever commented 4 years ago

FYI this was discussed at a Core Meeting here, starting at 19:06:55

geerlingguy commented 4 years ago

@apple4ever oops, updated the link, it's https://github.com/ansible/ansible/pull/66650

bcoca commented 4 years ago

So i've seen above many comments of things that were already answered/debunked/etc, so just going to link here my previous post .

https://github.com/ansible/ansible-documentation/issues/89

Please don't add new posts that don't add NEW items for discussion as they hide the previous ones that were already answered.

MarkusTeufelberger commented 4 years ago

By the way, where would be a good place to link to in the Python documentation about how valid variable names look like? There's https://docs.python.org/3/reference/lexical_analysis.html#grammar-token-identifier, but that's not really user friendly or readable for people without Computer Science backgrounds.

The reason for asking is that I'm not sure if the actual initial complaint has actually been dealt with. There's just a warning that there's something wrong, but it takes a lot of digging to find out what exactly and how one - if willing or able - could actually choose a valid group name. I would expect at least a clear "The group name foo-bar contains invalid characters (-). Valid group names should be valid Python identifiers (see https://docs.python.org/??? for more information)" message, not just "there are bad characters, check again with -vvvv to actually find out which ones!". Ideally this would also mention that this can be disabled, but could lead to other unexpected problems (like making it really hard for Ansible to distinguish the groups foo-bar, foo.bar and foo_bar).

Currently it's more of an "You did something wrong, fix it" message with no clear way laid out how to proceed which might also have contributed to the strong response here.

Tronde commented 4 years ago

@geerlingguy wrote in comment 56930:

(until they set force_valid_group_names = false in an ansible.cfg)

The documentation does not mention 'false' as a valid value for this key. I've set the value to 'ignore' which should do the trick. But is 'false' an invalid keyword or is it correct and the documentation is not complete here?

Aethylred commented 4 years ago

@bcoca in a previous comment:

Just to say this once, clearly , YOU WILL ALWAYS BE ABLE TO USE DASHES IN GROUP NAMES also dots and other characters that are now considered 'invalid', just not by default. This 'default' is what is being deprecated, the default will be 'safe' in 2.11, but you will always have an option to 'opt in' to the old behaviour.

You've stated repeatedly that the current behaviour can be preserved, but what are the exact ansible.cfg settings required to do this now and squash the deprecation warning.

I've tried as @geerlingguy wrote in comment 56930:

(until they set force_valid_group_names = false in an ansible.cfg)

Which causes my playbooks to fail when it can't find hosts or groups with hyphens (they're coming in from an inventory plugin we wrote BTW, do theses have to do the transform too, or is that done when Ansible ingests the inventory from the plugin?)

beenje commented 4 years ago

I've tried as @geerlingguy wrote in comment 56930:

(until they set force_valid_group_names = false in an ansible.cfg)

Which causes my playbooks to fail when it can't find hosts or groups with hyphens (they're coming in from an inventory plugin we wrote BTW, do theses have to do the transform too, or is that done when Ansible ingests the inventory from the plugin?)

This was mentioned in several comments and is in the documentation. You should use never or ignore.

campee commented 4 years ago

So are we just not supposed to use the EC2 dynamic inventory script anymore since it groups everything by 'us-east-1', 'us-east-2', etc? Or is there a plan to update it? I just went to Ansible's documentation for the EC2 dynamic inventory script and the link to download it on Github doesn't work anymore, so that's interesting.

mkrizek commented 4 years ago

I just went to Ansible's documentation for the EC2 dynamic inventory script and the link to download it on Github doesn't work anymore, so that's interesting.

See https://github.com/ansible/ansible/issues/68419

trombik commented 4 years ago

for those of you who don't bother to read the IRC log, here is the decision, i.e. no decision:

19:15:40 <sivel> I've got to say, that brining this topic up all the time isn't a good use of time
19:15:52 <cyberpear> bcoca nominated it
19:16:07 <felixfontein> I think the aim was to solve this once and for all (like, again :) )
19:16:29 <cyberpear> since bcoca is not here, move on to next topic?
19:16:34 <sivel> honestly, I don't think this is going to be the right forum to make a decision on this
19:16:45 <jillr> +2 moving on
19:16:47 <cyberpear> sivel: what's the correct forum?
19:16:55 <felixfontein> sivel: what is the right forum for making that decision?
19:17:02 <cyberpear> "declaration from Red Hat On High"?
19:17:15 <sivel> I'm going to abstain on that, but this project is not a democracy
19:17:16 <cyberpear> -1 to "declaration from Red Hat On High"
19:17:24 <sivel> too many cooks in the kitchen distract
19:17:45 <sivel> We know the arguments at this point
19:17:59 <sivel> anywho, next topic

yes, someone wrote "feel free to drop by ML or IRC". no, "this project is not a democracy".

sunshine69 commented 4 years ago

yes, someone wrote "feel free to drop by ML or IRC". no, "this project is not a democracy".

To be honest this is wrong with opensource - If it leads to a not popular way - people can fork it, can it be forked?

I can see accepting PR in ansible is terribly slow. Patches looks obviously needed and simple change but it never gets in. Luckily ansible itself is flexible to allow people to use custom plugins however it seems stale will make myself lesser in contributions - or even more hassle to do so.

Feeling a bit sad, really ...

Tronde commented 4 years ago

@sunshine69 I feel your pain. But that is a discussion that should take place on IRC or the Google Group for Ansible Development.

This issue is not the right place for it. Because to few people read here.

andyfeller commented 4 years ago

@sunshine69 I feel your pain. But that is a discussion that should take place on IRC or the Google Group for Ansible Development.

This issue is not the right place for it. Because to few people read here.

While the discussion might be more productive in those other channels, the transparency is appreciated for people following this issue specifically. IRC isn’t everyone’s preference after all.

geerlingguy commented 4 years ago

FYI: Remove deprecation for TRANSFORM_INVALID_GROUP_CHARS was just merged yesterday. There are backport PRs for 2.9 (https://github.com/ansible/ansible/pull/69487) and 2.8 (https://github.com/ansible/ansible/pull/69488) to remove the deprecation warnings there.

ansibot commented 4 years ago

Files identified in the description:

lib/ansible/modules/group.py

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

ansible / ansible-documentation