Open sygilber opened 5 years ago
Looked at options - here is one explanation for what's happening: https://stackoverflow.com/questions/3400171/python-utf-8-comparison
To add additional details regarding the set_mapping_rule role idempotence issue, it looks like the "End of Line Sequence" (EOLS) could also be the culprit as I recently encountered the same problem while trying to add new mapping rules.
Also, I was able to fix this issue by converting the "End of Line Sequence" (EOLS) from CRLF (Windows) to LF (Unix).
Furthermore, here's a comparison between a working and non-working file using the "file" program:
file mapping_rule_WORKING.js => mapping_rule.js: ASCII text, with very long lines file mapping_rule_NON_WORKING.js => mapping_rule.js: ASCII text, with very long lines, with CRLF line terminators
Maybe by adding a condition to convert the EOLS format to LF on the input file in the _check method could resolve this idempotency issue?
Just found that if the file encoding of the OAuth mapping rules file is utf-8 (instead of ascii) then the following code comparing both mapping rule content will silently fail and always return False even though both contents are identical (from ibmsecurity.isam.aac.mapping_rules.py):
With some logger.debug() statements added here and there, found that that I was getting UnicodeDecodeError crash when trying to display content of mapping rules file.
At this point, I'm not sure what is the definitive solution if we opt to support utf-8 encoding for input mapping rules files. For now, I am just making sure that the input mapping rules file encoding is ascii, and this resolves idempotency behavior observed when running 'set_mapping_rules' role.