Raspberry4 self-check "Temperature Zone 0" crashes CHECK_MK service on 1.6.0p6/Buster

[msperl]

Hi!

Temperature Zone 0 crashes on a RPI4 on 1.6.0p6 when checking itself. This actually is impacting the CHECK_MK service which effectively crashes when the service for the Temperature Zone is enabled.

Some output is generated in a strange way.

Error is: "'ascii' codec can't decode byte 0xc2 in position 10: ordinal not in range(128)" in

    [
      "/omd/sites/<site>/lib/python/cmk_base/checking.py",
      640,
      "_submit_via_check_result_file",
      "\"\"\" % (host, cmk_base.utils.make_utf8(service), now, now, state, cmk_base.utils.make_utf8(output)))"
    ]

with a payload of:

{'host': u'hostname',
 'now': 1574014972.502506,
 'output': u'OK - 45.8 \xb0C|temp=45.764;70;80;;',
 'service': u'Temperature Zone 0',
 'state': 0}

Monitoring the same node with a check_mk 1.6.0p6 on x86_64 does not show this behavior.

I fear this may be another one of those compiler/python issues...

Martin

[chrisss404]

Hi Martin,

I think this is a Python 2 - 3 issue. I created a small example script using checkmk's make_utf8 function:

#!/usr/bin/env python
# -*- encoding: utf-8; py-indent-offset: 4 -*-

import six

# taken from https://github.com/tribe29/checkmk/blob/339a00bc9c0d94197b81f5b78fc555e200df8686/cmk/utils/misc.py#L36
def make_utf8(x):
    if isinstance(x, six.text_type):
        return x.encode('utf-8')
    return x

output = 'OK - 45.8 \xb0C|temp=45.764;70;80;;'

print(" == checkmk's make_utf8 == ")
print(make_utf8(output))
print('')

print(" == raw output == ")
print(output)
print('')

Running this script with Python 2 and 3 gives the following:

$ python2 example.py 
 == checkmk's make_utf8 == 
OK - 45.8 �C|temp=45.764;70;80;;

 == raw output == 
OK - 45.8 �C|temp=45.764;70;80;;

$ python3 example.py 
 == checkmk's make_utf8 == 
b'OK - 45.8 \xc2\xb0C|temp=45.764;70;80;;'

 == raw output == 
OK - 45.8 °C|temp=45.764;70;80;;

You can see that in Python 3 the string that is returned by make_utf8 is in byte representation which I guess is causing the issue. If you wish to convert bytes to strings you can use b'a string in byte representation'.decode('utf-8').

From my point of view it is better to report this to feedback@check-mk.org, since Python 2 is nearing end of life very soon.

HTH & BR Christian

[msperl]

I have given the feedback, but unfortunately I have not heard back from them...

[msperl]

the only thing I do not understand is why the official docker images on intel (at least on centos8) work fine while the arm packages produced by you do not?

Also when I take the example above and run it inside the check_mk docker container I get:

root@120651ea69cd:/# /opt/omd/versions/1.6.0p6.cre/bin/python /tmp/test.py
 == checkmk's make_utf8 ==
OK - 45.8 �C|temp=45.764;70;80;;

 == raw output ==
OK - 45.8 �C|temp=45.764;70;80;;

root@120651ea69cd:/# /opt/omd/versions/1.6.0p6.cre/bin/python3 /tmp/test.py
bash: /opt/omd/versions/1.6.0p6.cre/bin/python3: No such file or directory
root@120651ea69cd:/# python /tmp/test.py
Traceback (most recent call last):
  File "/tmp/test.py", line 4, in <module>
    import six
ImportError: No module named six
root@120651ea69cd:/# python3 /tmp/test.py
bash: python3: command not found

So we seem to have the issue somewhere else, as the output seems identical.

The difference I see is that the arm packages use python 2.7.16 while the official version uses 2.7.13...

[msperl]

I have now checked also python version differences (raspberry pi stretch 2.7.13 vs buster 2.7.16): and both show the same output, which is identical to the output inside the container on intel.

buster with python 2.7.16:

pi@gateway:~ $ python /tmp/test.py
 == checkmk's make_utf8 ==
OK - 45.8 �C|temp=45.764;70;80;;

 == raw output ==
OK - 45.8 �C|temp=45.764;70;80;;

stretch with python 2.7.13

root@raspcm3:~# python /tmp/test.py
 == checkmk's make_utf8 ==
OK - 45.8 �C|temp=45.764;70;80;;

 == raw output ==
OK - 45.8 �C|temp=45.764;70;80;;

So it probably still is an issue inside python where it reacts differently depending on the SW behave differently.

Maybe a different portion of the code is responsible for the bad output?

[msperl]

So I have now instrumented the method _submit_via_check_result_file like this:

pi@gateway:~ $ diff -u /opt/omd/versions/1.6.0p7.cre/lib/python/cmk_base/checking.py.orig /opt/omd/versions/1.6.0p7.cre/lib/python/cmk_base/checking.py
--- /opt/omd/versions/1.6.0p7.cre/lib/python/cmk_base/checking.py.orig  2020-01-04 10:01:03.803073040 +0000
+++ /opt/omd/versions/1.6.0p7.cre/lib/python/cmk_base/checking.py   2020-01-04 11:08:04.134111764 +0000
@@ -621,9 +621,43 @@

 def _submit_via_check_result_file(host, service, state, output):
-    output = output.replace("\n", "\\n")
+  with open("/tmp/submit_check.log", "a+") as fh:
+    try:
+        fh.write("==========================\nHost: ")
+        fh.write(host)
+        fh.write("\nService: ")
+        fh.write(service)
+        fh.write("\nState: ")
+        fh.write(str(state))
+    except Exception as e:
+        fh.write("Exception " + str(e))
+        fh.write("\n")
+    try:
+        fh.write("\nreplace_newlines: ")
+        output = output.replace("\n", "\\n")
+        fh.write("OK")
+    except Exception as e:
+        fh.write("Exception " + str(e))
+        fh.write("\n")
+        raise e
+    try:
+        fh.write("\nOutput: ")
+        fh.write(output)
+        fh.write("\n")
+    except Exception as e:
+        fh.write("Exception " + str(e))
+        fh.write("\n")
+    try:
+        fh.write("\nOutput-utf8: ")
+        fh.write(cmk_base.utils.make_utf8(output))
+        fh.write("\n")
+    except Exception as e:
+        fh.write("Exception " + str(e))
+        fh.write("\n")
+
     _open_checkresult_file()
     if _checkresult_file_fd:
+      try:
         now = time.time()
         os.write(
             _checkresult_file_fd, """host_name=%s
@@ -639,6 +673,11 @@

 """ % (host, cmk_base.utils.make_utf8(service), now, now, state, cmk_base.utils.make_utf8(output)))

+        fh.write("Successfull")
+      except Exception as e:
+        fh.write("Failed with Exception " + str(e))
+        fh.write("\n")
+        raise(e)

 def _open_checkresult_file():
     global _checkresult_file_fd

on both arm and the intel container:

And I get on arm:

==========================
Host: gateway.elb
Service: Temperature Zone 0
State: 0
replace_newlines: OK
Output: Exception 'ascii' codec can't encode character u'\xb0' in position 10: ordinal not in range(128)

Output-utf8: OK - 54.0 °C|temp=54.043;70;80;;
Failed with Exception 'ascii' codec can't decode byte 0xc2 in position 10: ordinal not in range(128)
==========================

And on the container on intel:

==========================
Host: gateway.elb
Service: Temperature Zone 0
State: 0
replace_newlines: OK
Output: Exception 'ascii' codec can't encode character u'\xb0' in position 10: ordinal not in range(128)

Output-utf8: OK - 52.6 °C|temp=52.582;70;80;;
Successfull
==========================

So both show the same on the "normal" output

But only on string interpolation using % things fail on arm

[msperl]

but strangely in the example code of yours it works perfectly when extended it:

print(" == formated == ")
print("""%s""" % (make_utf8(output)))
print('')

So I start to wonder if there is something else

[msperl]

now I am getting closer: The reason why the example code does not trigger is because the output variable is not utf8 in the example code, but when check_mk runs it is in utf8 format!

Here the extended output (not showing code):

Service: Temperature Zone 0
State: 0
replace_newlines: OK
Output type: <type 'unicode'>

So here now 2 versions of your example code:

pi@gateway:~ $ diff -u /tmp/test.py /tmp/test2.py
--- /tmp/test.py    2020-01-04 12:32:04.842643837 +0000
+++ /tmp/test2.py   2020-01-04 12:32:29.172105231 +0000
@@ -9,7 +9,7 @@
         return x.encode('utf-8')
     return x

-output = 'OK - 45.8 \xb0C|temp=45.764;70;80;;'
+output = u'OK - 45.8 \xb0C|temp=45.764;70;80;;'

 print(" == checkmk's make_utf8 == ")
 print(make_utf8(output))

Running both instances works with the bash locale:

pi@gateway:~ $ /opt/omd/versions/1.6.0p6.cre/bin/python2.7 /tmp/test.py
 == checkmk's make_utf8 ==
OK - 45.8 �C|temp=45.764;70;80;;

 == raw output ==
OK - 45.8 �C|temp=45.764;70;80;;

pi@gateway:~ $ /opt/omd/versions/1.6.0p6.cre/bin/python2.7 /tmp/test2.py
 == checkmk's make_utf8 ==
OK - 45.8 °C|temp=45.764;70;80;;

 == raw output ==
OK - 45.8 °C|temp=45.764;70;80;;

But when changing to LANG="" we get:

pi@gateway:~ $ LANG= /opt/omd/versions/1.6.0p6.cre/bin/python2.7 /tmp/test.py
 == checkmk's make_utf8 ==
OK - 45.8 �C|temp=45.764;70;80;;

 == raw output ==
OK - 45.8 �C|temp=45.764;70;80;;

pi@gateway:~ $ LANG= /opt/omd/versions/1.6.0p6.cre/bin/python2.7 /tmp/test2.py
 == checkmk's make_utf8 ==
OK - 45.8 °C|temp=45.764;70;80;;

 == raw output ==
Traceback (most recent call last):
  File "/tmp/test2.py", line 19, in <module>
    print(output)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xb0' in position 10: ordinal not in range(128)

So we can now replicate the error with LANG="" and the string being utf8!

Note that it fails also for LANG=en_GB but works with LANG=en_GB.utf8

pi@gateway:~ $ LANG=en_GB.utf8 /opt/omd/versions/1.6.0p6.cre/bin/python2.7 /tmp/test2.py
 == checkmk's make_utf8 ==
OK - 45.8 °C|temp=45.764;70;80;;

 == raw output ==
OK - 45.8 °C|temp=45.764;70;80;;

pi@gateway:~ $ LANG=en_GB /opt/omd/versions/1.6.0p6.cre/bin/python2.7 /tmp/test2.py
 == checkmk's make_utf8 ==
OK - 45.8 °C|temp=45.764;70;80;;

 == raw output ==
Traceback (most recent call last):
  File "/tmp/test2.py", line 19, in <module>
    print(output)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xb0' in position 10: ordinal not in range(128)

Same for LANG=POSIXor LANG=C - the exception occurs.

But why does it fail on ARM and not in the container? I have confirmed that in both cases the environment "LANG" is unset.

[chrisss404]

Nice catch Martin.

Strange, it seems that the environment variable is set for me:

root@rpi:~# echo $LANG
en_GB.UTF-8

root@rpi:~# su mysite -
OMD[mysite]:~$ echo $LANG
C.UTF-8

Maybe it is set because I don't have a fresh install of checkmk?

[msperl]

So here now the environments:

arm:

  CONFIG_ADMIN_MAIL =
  CONFIG_APACHE_MODE = own
  CONFIG_APACHE_TCP_ADDR = 127.0.0.1
  CONFIG_APACHE_TCP_PORT = 5000
  CONFIG_AUTOSTART = on
  CONFIG_CORE = nagios
  CONFIG_DOKUWIKI_AUTH = off
  CONFIG_LIVESTATUS_TCP = off
  CONFIG_LIVESTATUS_TCP_ONLY_FROM = 0.0.0.0 ::/0
  CONFIG_LIVESTATUS_TCP_PORT = 6557
  CONFIG_LIVESTATUS_TCP_TLS = on
  CONFIG_MKEVENTD = on
  CONFIG_MKEVENTD_SNMPTRAP = off
  CONFIG_MKEVENTD_SYSLOG = off
  CONFIG_MKEVENTD_SYSLOG_TCP = off
  CONFIG_MULTISITE_AUTHORISATION = on
  CONFIG_MULTISITE_COOKIE_AUTH = on
  CONFIG_NAGIOS_THEME = classicui
  CONFIG_NSCA = off
  CONFIG_NSCA_TCP_PORT = 5667
  CONFIG_PNP4NAGIOS = on
  CONFIG_TMPFS = on
  CORE_NOVERIFY = yes
  CRYPTOGRAPHY_ALLOW_OPENSSL_098 = 1
  HOME = /omd/sites/elbrizo1
  LC_ALL = C
  LD_LIBRARY_PATH = /omd/sites/elbrizo1/local/lib:/omd/sites/elbrizo1/lib
  MAILRC = /omd/sites/elbrizo1/etc/mail.rc
  MANPATH = /omd/sites/elbrizo1/share/man:/omd/sites/elbrizo1/share/man:
  MODULEBUILDRC = /omd/sites/elbrizo1/.modulebuildrc
  MP_STATE_DIRECTORY = /omd/sites/elbrizo1/var/monitoring-plugins
  NAGIOS_PLUGIN_STATE_DIRECTORY = /omd/sites/elbrizo1/var/monitoring-plugins
  OLDPWD = /
  OMD_ROOT = /omd/sites/elbrizo1
  OMD_SITE = elbrizo1
  PATH = /omd/sites/elbrizo1/lib/perl5/bin:/omd/sites/elbrizo1/local/bin:/omd/sites/elbrizo1/bin:/omd/sites/elbrizo1/local/lib/perl5/bin:/omd/sites/elbrizo1/lib/perl5/bin:/omd/sites/elbrizo1/local/bin:/omd/sites/elbrizo1/bin:/omd/sites/elbrizo1/local/lib/perl5/bin:/omd/sites/elbrizo1/local/bin:/omd/sites/elbrizo1/bin:/usr/local/bin:/bin:/usr/bin:/sbin:/usr/sbin
  PERL5LIB = /omd/sites/elbrizo1/local/lib/perl5/lib/perl5:/omd/sites/elbrizo1/lib/perl5/lib/perl5:/omd/sites/elbrizo1/local/lib/perl5/lib/perl5:/omd/sites/elbrizo1/lib/perl5/lib/perl5:
  PERL_MM_OPT = INSTALL_BASE=/omd/sites/elbrizo1/local/lib/perl5/
  PWD = /
  REQUESTS_CA_BUNDLE = /omd/sites/elbrizo1/var/ssl/ca-certificates.crt
  SHLVL = 2
  TERM = xterm
  USER = elbrizo1
  _ = /omd/sites/elbrizo1/bin/nagios

And on intel-container:

  CONFIG_ADMIN_MAIL =
  CONFIG_APACHE_MODE = own
  CONFIG_APACHE_TCP_ADDR = 0.0.0.0
  CONFIG_APACHE_TCP_PORT = 5000
  CONFIG_AUTOSTART = on
  CONFIG_CORE = nagios
  CONFIG_DOKUWIKI_AUTH = off
  CONFIG_LIVESTATUS_TCP = on
  CONFIG_LIVESTATUS_TCP_ONLY_FROM = 0.0.0.0
  CONFIG_LIVESTATUS_TCP_PORT = 6557
  CONFIG_LIVESTATUS_TCP_TLS = on
  CONFIG_MKEVENTD = on
  CONFIG_MKEVENTD_SNMPTRAP = off
  CONFIG_MKEVENTD_SYSLOG = off
  CONFIG_MKEVENTD_SYSLOG_TCP = off
  CONFIG_MULTISITE_AUTHORISATION = on
  CONFIG_MULTISITE_COOKIE_AUTH = on
  CONFIG_NAGIOS_THEME = classicui
  CONFIG_NSCA = off
  CONFIG_NSCA_TCP_PORT = 5667
  CONFIG_PNP4NAGIOS = on
  CONFIG_TMPFS = on
  CORE_NOVERIFY = yes
  CRYPTOGRAPHY_ALLOW_OPENSSL_098 = 1
  HOME = /omd/sites/smk
  LC_ALL = C
  LD_LIBRARY_PATH = /omd/sites/smk/local/lib:/omd/sites/smk/lib
  MAILRC = /omd/sites/smk/etc/mail.rc
  MANPATH = /omd/sites/smk/share/man:/omd/sites/smk/share/man:
  MODULEBUILDRC = /omd/sites/smk/.modulebuildrc
  MP_STATE_DIRECTORY = /omd/sites/smk/var/monitoring-plugins
  NAGIOS_PLUGIN_STATE_DIRECTORY = /omd/sites/smk/var/monitoring-plugins
  OLDPWD = /
  OMD_ROOT = /omd/sites/smk
  OMD_SITE = smk
  PATH = /omd/sites/smk/lib/perl5/bin:/omd/sites/smk/local/bin:/omd/sites/smk/bin:/omd/sites/smk/local/lib/perl5/bin:/omd/sites/smk/lib/perl5/bin:/omd/sites/smk/local/bin:/omd/sites/smk/bin:/omd/sites/smk/local/lib/perl5/bin:/omd/sites/smk/local/bin:/omd/sites/smk/bin:/usr/local/bin:/bin:/usr/bin:/sbin:/usr/sbin
  PERL5LIB = /omd/sites/smk/local/lib/perl5/lib/perl5:/omd/sites/smk/lib/perl5/lib/perl5:/omd/sites/smk/local/lib/perl5/lib/perl5:/omd/sites/smk/lib/perl5/lib/perl5:
  PERL_MM_OPT = INSTALL_BASE=/omd/sites/smk/local/lib/perl5/
  PWD = /
  REQUESTS_CA_BUNDLE = /omd/sites/smk/var/ssl/ca-certificates.crt
  SHLVL = 2
  TERM = xterm
  USER = smk
  _ = /omd/sites/smk/bin/nagios

So It seems as if in the container the LOCALE is set differently.

but logging fh.write("\nLocale: "); fh.write(str(locale.getlocale())) returns Locale: (None, None)

sys.getdefaultencoding()also returns ascii in both cases (arm and intel/x86_64)

So I am not sure where else this may come from...

[msperl]

yes, that may be the case, but note that on the command line I also get:

pi@gateway:~ $ sudo su - elbrizo1
OMD[elbrizo1]:~$ echo $LANG
C.UTF-8

but It (=locale) is set to None,None when running the script /opt/omd/versions/1.6.0p7.cre/lib/python/cmk_base/checking.py

And I still wonder where the difference between ARM and x86_64/intel is when running it as part of the "check-mk machinery"!

[msperl]

Another idea: have others also seen this or is it just me? Maybe all others are not running Buster? Or they are using older/upgraded deploys?

[chrisss404]

Running the following as mysite user:

#!/usr/bin/env python

import locale
import sys

print("locale:")
print(locale.getlocale())

print("")
print("getdefaultlocale:")
print(locale.getdefaultlocale())

print("")
print("encoding:")
print(sys.getdefaultencoding())

gives:

locale:
(None, None)

getdefaultlocale:
('en_US', 'UTF-8')

encoding:
ascii

That means that the default locale is available, but not were you expect it.

I don't have this issue running Buster and an upgraded site from 1.4.0p19.

[chrisss404]

Maybe it is related to the agent.

Which version of the agent are you running? Are you running the agent from your intel/x86_64 installation?

root@rpi:~# check_mk_agent
<<<check_mk>>>
Version: 1.6.0p7
AgentOS: linux
Hostname: rpi

[msperl]

in the end: this is what works:

--- /opt/omd/versions/1.6.0p7.cre/lib/python/cmk_base/checking.py.orig  2020-01-04 10:01:03.803073040 +0000
+++ /opt/omd/versions/1.6.0p7.cre/lib/python/cmk_base/checking.py   2020-01-04 13:47:12.702752502 +0000
@@ -637,7 +637,7 @@
 return_code=%d
 output=%s

-""" % (host, cmk_base.utils.make_utf8(service), now, now, state, cmk_base.utils.make_utf8(output)))
+""" % (host, cmk_base.utils.make_utf8(service), now, now, state, cmk_base.utils.make_utf8(output).decode('utf-8').encode('ascii', 'replace')))

 def _open_checkresult_file():

why it is not needed is unclear...

[msperl]

I am using the deb from the arm install using the output of: find '/omd/sites/{{ check_mk_site_name }}/share/check_mk/agents' -name "*.deb" -type f

[chrisss404]

So it is not related to the agent.

Your fix does not work for me. When I apply your patch I get: OK - 62.3 ?C

[msperl]

it is not working 100%, it just gives me a working check without exceptions - values show and graph.

In the meantime I have verified that p1, p6 and p7 docker images all use python 2.7.16 for check_mk, so we are consistent here...

[msperl]

I have now 2 instances (arm and x86) both running 1.6.0p7 and I have instrumented /opt/omd/versions/1.6.0p7.cre/lib/python/cmk_base/checking.py so that for one specific check I get extra debug output to a file (code - for now - not shared):

Here the output: X86_64 - functional:

===============================
Host:             gateway.elb
Service:          Temperature Zone 0
Orig Output type: <type 'unicode'>
Exception 'ascii' codec can't encode character u'\xb0' in position 10: ordinal not in range(128)
Output type:      <type 'str'>
Output:           OK - 56.0 °C|temp=55.991;70;80;;
Output-str:   OK - 56.0 °C|temp=55.991;70;80;;
final:            host_name=gateway.elb
service_description=Temperature Zone 0
check_type=1
check_options=0
reschedule_check
latency=0.0
start_time=1578188112.9
finish_time=1578188112.9
return_code=0
output=OK - 56.0 °C|temp=55.991;70;80;;

Success

ARM - not functional:

===============================
Host:             gateway.elb
Service:          Temperature Zone 0
Orig Output type: <type 'unicode'>
Exception 'ascii' codec can't encode character u'\xb0' in position 10: ordinal not in range(128)
Output type:      <type 'str'>
Output:           OK - 56.5 °C|temp=56.478;70;80;;
Output-str:       OK - 56.5 °C|temp=56.478;70;80;;
Exception 'ascii' codec can't decode byte 0xc2 in position 10: ordinal not in range(128)

So it is a bit of a mystery why (almost) identical input shows different behaviour.

One of the differences that I find is that "/opt/omd/versions/1.6.0p7.cre/bin/python2.7 -v" is giving slightly different output:

Here the diff:

--- /tmp/pythonv-x86.txt    2020-01-05 01:43:35.074159943 +0000
+++ /tmp/pythonv-arm.txt    2020-01-05 01:43:12.924646939 +0000
@@ -57,10 +57,10 @@
 import _codecs # builtin
 # /opt/omd/versions/1.6.0p7.cre/lib/python2.7/encodings/aliases.pyc matches /opt/omd/versions/1.6.0p7.cre/lib/python2.7/encodings/aliases.py
 import encodings.aliases # precompiled from /opt/omd/versions/1.6.0p7.cre/lib/python2.7/encodings/aliases.pyc
-# /opt/omd/versions/1.6.0p7.cre/lib/python2.7/encodings/ascii.pyc matches /opt/omd/versions/1.6.0p7.cre/lib/python2.7/encodings/ascii.py
-import encodings.ascii # precompiled from /opt/omd/versions/1.6.0p7.cre/lib/python2.7/encodings/ascii.pyc
-Python 2.7.16 (default, Dec 11 2019, 08:07:13)
-[GCC 8.2.0] on linux2
+# /opt/omd/versions/1.6.0p7.cre/lib/python2.7/encodings/utf_8.pyc matches /opt/omd/versions/1.6.0p7.cre/lib/python2.7/encodings/utf_8.py
+import encodings.utf_8 # precompiled from /opt/omd/versions/1.6.0p7.cre/lib/python2.7/encodings/utf_8.pyc
+Python 2.7.16 (default, Dec 11 2019, 16:43:13)
+[GCC 8.3.0] on linux2
 Type "help", "copyright", "credits" or "license" for more information.
 dlopen("/opt/omd/versions/1.6.0p7.cre/lib/python2.7/lib-dynload/readline.so", 2);
 import readline # dynamically loaded from /opt/omd/versions/1.6.0p7.cre/lib/python2.7/lib-dynload/readline.so
@@ -97,7 +97,7 @@
 # cleanup[1] _warnings
 # cleanup[1] zipimport
 # cleanup[1] _sysconfigdata
-# cleanup[1] encodings.ascii
+# cleanup[1] encodings.utf_8
 # cleanup[1] codecs
 # cleanup[1] readline
 # cleanup[1] _locale
@@ -126,5 +126,5 @@
 # cleanup[2] os
 # cleanup sys
 # cleanup __builtin__
-# cleanup ints: 19 unfreed ints
+# cleanup ints: 18 unfreed ints
 # cleanup floats

We see that the compilers are different (gcc 8.2.0 for x86/intel and 8.3.0 for arm)

But more importantly: x86/intel uses import encodings.ascii while arm uses import encodings.utf_8

Where this difference comes from is unclear - it may be related to the compiler, but it may also be different defaults.

How we can control this is also unclear.

for reference here also the diff between the arcitectures for: /opt/omd/versions/1.6.0p7.cre/bin/python2.7 -m sysconfig

--- /tmp/python-x86.txt 2020-01-05 01:24:13.479666729 +0000
+++ /tmp/python-arm.txt 2020-01-05 01:23:30.150620548 +0000
@@ -1,4 +1,4 @@
-Platform: "linux-x86_64"
+Platform: "linux-armv7l"
 Python version: "2.7"
 Current installation scheme: "posix_prefix"

@@ -36,8 +36,8 @@
    CONFINCLUDEDIR = "/include"
    CONFINCLUDEPY = "/include/python2.7"
    COREPYTHONPATH = ":plat-linux2:lib-tk:lib-old"
-   COVERAGE_INFO = "/home/jenkins/workspace/cmk_160/nightly_build_containerized@2/dest/omd/Python-2.7.16/coverage.info"
-   COVERAGE_REPORT = "/home/jenkins/workspace/cmk_160/nightly_build_containerized@2/dest/omd/Python-2.7.16/lcov-report"
+   COVERAGE_INFO = "/root/check-mk-arm/check-mk-raw-1.6.0p7.cre/omd/Python-2.7.16/coverage.info"
+   COVERAGE_REPORT = "/root/check-mk-arm/check-mk-raw-1.6.0p7.cre/omd/Python-2.7.16/lcov-report"
    COVERAGE_REPORT_OPTIONS = "--no-branch-coverage --title "CPython lcov report""
    CPPFLAGS = "-I. -IInclude -I./Include"
    CXX = "g++ -pthread"
@@ -155,11 +155,11 @@
    HAVE_FTRUNCATE = "1"
    HAVE_GAI_STRERROR = "1"
    HAVE_GAMMA = "1"
-   HAVE_GCC_ASM_FOR_X87 = "1"
+   HAVE_GCC_ASM_FOR_X87 = "0"
    HAVE_GETADDRINFO = "1"
    HAVE_GETCWD = "1"
    HAVE_GETC_UNLOCKED = "1"
-   HAVE_GETENTROPY = "0"
+   HAVE_GETENTROPY = "1"
    HAVE_GETGROUPS = "1"
    HAVE_GETHOSTBYNAME = "0"
    HAVE_GETHOSTBYNAME_R = "1"
@@ -199,7 +199,7 @@
    HAVE_KILLPG = "1"
    HAVE_KQUEUE = "0"
    HAVE_LANGINFO_H = "1"
-   HAVE_LARGEFILE_SUPPORT = "0"
+   HAVE_LARGEFILE_SUPPORT = "1"
    HAVE_LCHFLAGS = "0"
    HAVE_LCHOWN = "1"
    HAVE_LGAMMA = "1"
@@ -322,7 +322,7 @@
    HAVE_SYS_NDIR_H = "0"
    HAVE_SYS_PARAM_H = "1"
    HAVE_SYS_POLL_H = "1"
-   HAVE_SYS_RANDOM_H = "0"
+   HAVE_SYS_RANDOM_H = "1"
    HAVE_SYS_RESOURCE_H = "1"
    HAVE_SYS_SELECT_H = "1"
    HAVE_SYS_SOCKET_H = "1"
@@ -358,7 +358,7 @@
    HAVE_UNAME = "1"
    HAVE_UNISTD_H = "1"
    HAVE_UNSETENV = "1"
-   HAVE_USABLE_WCHAR_T = "0"
+   HAVE_USABLE_WCHAR_T = "1"
    HAVE_UTIL_H = "0"
    HAVE_UTIMES = "1"
    HAVE_UTIME_H = "1"
@@ -370,7 +370,7 @@
    HAVE_WORKING_TZSET = "1"
    HAVE_ZLIB_COPY = "1"
    HAVE__GETPTY = "0"
-   HOST_GNU_TYPE = "x86_64-pc-linux-gnu"
+   HOST_GNU_TYPE = "armv6l-unknown-linux-gnueabihf"
    HURD_C_THREADS = "0"
    INCLDIRSTOMAKE = "/include /include /include/python2.7 /include/python2.7"
    INCLUDEDIR = "/include"
@@ -417,7 +417,7 @@
    MACOSX_DEPLOYMENT_TARGET = ""
    MAINCC = "gcc -pthread"
    MAJOR_IN_MKDEV = "0"
-   MAJOR_IN_SYSMACROS = "0"
+   MAJOR_IN_SYSMACROS = "1"
    MAKESETUP = "./Modules/makesetup"
    MANDIR = "/share/man"
    MEMTESTOPTS = "-l -x test_subprocess test_io test_lib2to3 \ -x test_dl test___all__ test_fork1 \"
@@ -425,7 +425,7 @@
    MODLIBS = ""
    MODOBJS = "Modules/threadmodule.o  Modules/signalmodule.o  Modules/posixmodule.o  Modules/errnomodule.o  Modules/pwdmodule.o  Modules/_sre.o  Modules/_codecsmodule.o  Modules/_weakref.o  Modules/zipimport.o  Modules/symtablemodule.o  Modules/xxsubtype.o"
    MODULE_OBJS = "\"
-   MULTIARCH = "x86_64-linux-gnu"
+   MULTIARCH = "arm-linux-gnueabihf"
    MVWDELCH_IS_EXPRESSION = "1"
    OBJECT_OBJS = "\"
    OLDPATH = ":lib-old"
@@ -469,7 +469,7 @@
    PY_CFLAGS = "-fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include -fPIC -DPy_BUILD_CORE"
    PY_FORMAT_LONG_LONG = ""ll""
    PY_FORMAT_SIZE_T = ""z""
-   PY_UNICODE_TYPE = "0"
+   PY_UNICODE_TYPE = "wchar_t"
    Py_DEBUG = "0"
    Py_ENABLE_SHARED = "1"
    Py_UNICODE_SIZE = "4"
@@ -478,7 +478,7 @@
    RANLIB = "ranlib"
    RESSRCDIR = "Mac/Resources/framework"
    RETSIGTYPE = "void"
-   RUNSHARED = "LD_LIBRARY_PATH=/home/jenkins/workspace/cmk_160/nightly_build_containerized@2/dest/omd/Python-2.7.16"
+   RUNSHARED = "LD_LIBRARY_PATH=/root/check-mk-arm/check-mk-raw-1.6.0p7.cre/omd/Python-2.7.16"
    SCRIPTDIR = "/lib"
    SETPGRP_HAVE_ARG = "0"
    SGI_ABI = ""
@@ -492,17 +492,17 @@
    SIZEOF_FLOAT = "4"
    SIZEOF_FPOS_T = "16"
    SIZEOF_INT = "4"
-   SIZEOF_LONG = "8"
-   SIZEOF_LONG_DOUBLE = "16"
+   SIZEOF_LONG = "4"
+   SIZEOF_LONG_DOUBLE = "8"
    SIZEOF_LONG_LONG = "8"
    SIZEOF_OFF_T = "8"
    SIZEOF_PID_T = "4"
-   SIZEOF_PTHREAD_T = "8"
+   SIZEOF_PTHREAD_T = "4"
    SIZEOF_SHORT = "2"
-   SIZEOF_SIZE_T = "8"
-   SIZEOF_TIME_T = "8"
-   SIZEOF_UINTPTR_T = "8"
-   SIZEOF_VOID_P = "8"
+   SIZEOF_SIZE_T = "4"
+   SIZEOF_TIME_T = "4"
+   SIZEOF_UINTPTR_T = "4"
+   SIZEOF_VOID_P = "4"
    SIZEOF_WCHAR_T = "4"
    SIZEOF__BOOL = "1"
    SO = ".so"
@@ -521,7 +521,7 @@
    TESTOPTS = "-l"
    TESTPATH = ""
    TESTPROG = "./Lib/test/regrtest.py"
-   TESTPYTHON = "LD_LIBRARY_PATH=/home/jenkins/workspace/cmk_160/nightly_build_containerized@2/dest/omd/Python-2.7.16 ./python -Wd -3 -E -tt"
+   TESTPYTHON = "LD_LIBRARY_PATH=/root/check-mk-arm/check-mk-raw-1.6.0p7.cre/omd/Python-2.7.16 ./python -Wd -3 -E -tt"
    TESTPYTHONOPTS = ""
    THREADOBJ = "Python/thread.o"
    TIME_WITH_SYS_TIME = "1"
@@ -531,7 +531,7 @@
    UNIVERSALSDK = ""
    USE_COMPUTED_GOTOS = "0"
    USE_TOOLBOX_OBJECT_GLUE = "0"
-   VA_LIST_IS_ARRAY = "1"
+   VA_LIST_IS_ARRAY = "0"
    VERSION = "2.7"
    WANT_SIGFPE_HANDLER = "0"
    WANT_WCTYPE_FUNCTIONS = "0"
@@ -546,13 +546,13 @@
    WITH_VALGRIND = "0"
    X87_DOUBLE_ROUNDING = "0"
    XMLLIBSUBDIRS = "xml xml/dom xml/etree xml/parsers xml/sax"
-   abs_builddir = "/home/jenkins/workspace/cmk_160/nightly_build_containerized@2/dest/omd/Python-2.7.16"
-   abs_srcdir = "/home/jenkins/workspace/cmk_160/nightly_build_containerized@2/dest/omd/Python-2.7.16"
+   abs_builddir = "/root/check-mk-arm/check-mk-raw-1.6.0p7.cre/omd/Python-2.7.16"
+   abs_srcdir = "/root/check-mk-arm/check-mk-raw-1.6.0p7.cre/omd/Python-2.7.16"
    base = "/opt/omd/versions/1.6.0p7.cre"
-   build = "x86_64-pc-linux-gnu"
+   build = "armv6l-unknown-linux-gnueabihf"
    datarootdir = "/share"
    exec_prefix = ""
-   host = "x86_64-pc-linux-gnu"
+   host = "armv6l-unknown-linux-gnueabihf"
    platbase = "/opt/omd/versions/1.6.0p7.cre"
    prefix = ""
    projectbase = "/opt/omd/versions/1.6.0p7.cre/bin"
@@ -560,4 +560,4 @@
    py_version_nodot = "27"
    py_version_short = "2.7"
    srcdir = "."
-   userbase = "/root/.local"
+   userbase = "/home/pi/.local"

So in the end the interesting question may be: how can we control the default encoding used?

Is there a simple setting or do we have to recompile python with different flags?

[msperl]

OK - I can control the default encoding imported with "LANG=..." as well. Similar issue as above (inside the docker container on x86 LANG is unset and hence ascii is used).

Still it does not leave us any wiser as to why arm and x86 are showing different behaviour...

[msperl]

in the end I guess I found a solution:

pi@gateway:~ $ diff -u /opt/omd/versions/1.6.0p7.cre/lib/python/cmk_base/checking.py.orig /opt/omd/versions/1.6.0p7.cre/lib/python/cmk_base/checking.py
--- /opt/omd/versions/1.6.0p7.cre/lib/python/cmk_base/checking.py.orig  2020-01-04 10:01:03.803073040 +0000
+++ /opt/omd/versions/1.6.0p7.cre/lib/python/cmk_base/checking.py   2020-01-05 03:02:35.149749747 +0000
@@ -637,7 +637,7 @@
 return_code=%d
 output=%s

-""" % (host, cmk_base.utils.make_utf8(service), now, now, state, cmk_base.utils.make_utf8(output)))
+""" % (cmk_base.utils.make_utf8(host), cmk_base.utils.make_utf8(service), now, now, state, cmk_base.utils.make_utf8(output)))

 def _open_checkresult_file():

I still do not understand why this works without this on x86/intel while it fails on ARM...

[msperl]

updated the ticket on the check-mk support side with the findings as well - maybe we get that "officially" fixed in 1.6.0p8...

[chrisss404]

Should we keep this open, is it still an issue?

[Kruemmelspalter]

I think it's not an issue anymore because it's about 1.6.0p6 and the latest version is p13

[msperl]

I think ist is solved