Several unicode problems all around

[mdellweg]

SUMMARY

The modules do not work well for unicode in various ways. (Not yet finished exploring. Will probably add more.)

• [ ] identify where we have issues with unicode
  • [ ] which Ansible/Python/Apypie/Requests version
  • [ ] which module/type of request
  • [ ] special characters in passwords work in python2, not in python3 (maybe that was apypie 0.1.0)
  • [ ] special characters in name break idempotence of at least architecture and location in python2
• [ ] research how Ansible expects this to be handled (docs are rare)
• [ ] FIX
• [ ] add tests

ISSUE TYPE

• Bug Report

[mdellweg]

The only documentation to this i can find is: https://docs.ansible.com/ansible/latest/dev_guide/developing_python_3.html

[evgeni]

So I wondered how much Ansible and how much everything else is at play here, so I tried with plain Python+Apypie:

python2.7:
  works:
    apypie.Api(uri='https://foreman.example.com', username='üser', password='changeme', api_version=2, verify_ssl=False)
    apypie.Api(uri='https://cforeman.example.com', username='user', password='pässword', api_version=2, verify_ssl=False)
  fails:
    apypie.Api(uri='https://foreman.example.com', username=u'üser', password='changeme', api_version=2, verify_ssl=False) (500)
    apypie.Api(uri='https://foreman.example.com', username='user', password=u'pässword', api_version=2, verify_ssl=False) (401)

python3.7:
  works:
    apypie.Api(uri='https://foreman.example.com', username='üser'.encode('utf-8'), password='changeme', api_version=2, verify_ssl=False)
    apypie.Api(uri='https://foreman.example.com', username='user', password='pässword'.encode('utf-8'), api_version=2, verify_ssl=False)
  fails:
    apypie.Api(uri='https://foreman.example.com', username='üser', password='changeme', api_version=2, verify_ssl=False) (500)
    apypie.Api(uri='https://foreman.example.com', username='user', password='pässword', api_version=2, verify_ssl=False) (401)

the actual testing was done by executing api.resource('hosts').call('index') on the resulting API object.

it seems as long as we pass bytes (that's what Python2's str essentially is), requests does the right thing. Otherwise not.

btw, this is with Apypie 0.2.1 and Requests 2.22.0 on Fedora 30.

[evgeni]

@mdellweg what's the state of this? I think #609 fixes all issues we could think of and also adds tests for the string things. should we consider all TODOs here then done, or do you want to test/research further?

[mdellweg]

I'm fine with merging that PR, closing this one and consider any further Ünicöde bug we find a new one. From reading the docs, this is the best I could come up with. Still i don't like that one is supposed to import anything from _text.