Output of subprocesses spoils agentplugin data

[simonmeggle]

While the CMK agent gets fed with the STDOUT of the robotmk plugin, we have to make sure that STDOUT only contains the CMK section header and the XML data of Robot.

The problem here is that we rely on the fact that robot test does not produce any output. For Robot itself this is controlled by the argument "console:none" which mutes Robot's output in general. But there is still no control over the tools which are called by Robot libraries. The chromedriver.exe for example produces this line on STDOUT when starting up chrome:

DevTools listening on ws://127.0.0.1:52843/devtools/browser/db51a7ca-b606-4466-a20f-54b55a7c0738

This is then the first plugin output line (wrong), followed by the section header and XML dat (correct).

It is no solution to tinker with the 3rd party tool options to mute them. This would break with the aim to be fully compatible with every robot test without changing anything in the test code. (Apart from that, we have to consider that there are libs which do not have such an option and will always be noisy).

Furthermore, it does not seem to be so easy to redirect stdout and stderr as this example shows:

robot --console none sum_prices_headless.robot 2>&1 > NUL
DevTools listening on ws://127.0.0.1:52988/devtools/browser/96f16f4a-7e39-4226-898c-157344979ec4

Although both FDs are redirected to NUL, chrome still shows log lines.

We need a reliable solution which guarantees that there is no output at all when calling robot; independently from tools and libraries.

[simonmeggle]

Attempt 1 - unsuccessful:

     from io import StringIO
     # Create the in-memory "file"
     tmp_stdout = StringIO()
     tmp_stderr = StringIO()
     sys.stdout = tmp_stdout
     sys.stderr = tmp_stderr
     rc = run(suite, **suite_options)
     sys.stdout = sys.__stdout__
     sys.stderr = sys.__stderr__

Attempt 2 - start Robot within a separate thread - unsuccessful:

  import threading
  from io import StringIO
  global rc
  rc = None
  rc_avail = threading.Event()
  def robot_run(suite, suite_options):
     # Create the in-memory "file"
     tmp_stdout = StringIO()
     tmp_stderr = StringIO()
     # #Replace default stdout (terminal) with our stream
     sys.stdout = tmp_stdout
     sys.stderr = tmp_stderr

     rc = run(suite, **suite_options)
     rc_avail.set()
     sys.stdout = sys.__stdout__
     sys.stderr = sys.__stderr__
  thread = threading.Thread(target=robot_run, args=(robotdir.joinpath(suite), suite_options))
  thread.start()
  rc_avail.wait()

In both ways, I still get the "DevTools" message from chromedriver:

c:\ProgramData\checkmk\agent\plugins>robotmk.py

DevTools listening on ws://127.0.0.1:53054/devtools/browser/ffc6cf2a-af84-41c3-a381-1b623436a187
<<<robotmk:sep(0)>>>
<?xml version="1.0" encoding="UTF-8"?>
<robot generator="Robot 3.2.1 (Python 3.8.3 on win32)" generated="20200619 14:01:55.564" rpa="false">
<suite id="s1" name="Sum Prices Headless" source="C:\ProgramData\checkmk\agent\robot\sum_prices_headless">
<suite id="s1-s1" name="Sum Prices Headless" source="C:\ProgramData\checkmk\agent\robot\sum_prices_headless\sum_prices_headless.robot">
<test id="s1-s1-t1" name="Gesamtwert Fachliteratur">
<kw name="Open Chrome" type="setup">
...

[simonmeggle]

This seems to be a chrome specific problem (others had this with powershell: https://mlog.club/article/5383512).

Even if suppressing lines from 3rd party tools in general would be much better, this two approaches make it possible to mute chrome completely:

    ${EXCLUDES}    Create list     enable-logging  
    Call Method    ${chrome_options}    add_experimental_option    excludeSwitches  ${EXCLUDES}

or by custom python file to create the webdriver instance (feels better):

import os
from robot.api.deco import keyword
from selenium import webdriver
@keyword('Set Chrome Options')
def create_driver():
    chromeOptions = webdriver.ChromeOptions()
    chromeOptions.add_experimental_option('excludeSwitches', ['enable-logging'])
    ...
    return chromeOptions

[simonmeggle]

https://forum.robotframework.org/t/disable-any-output-of-robot-run/380/2?u=simonm

[mike1098]

Hi Simon, please check my last commit in my repo.

redirect from https://docs.python.org/3/library/contextlib.html should do the trick:

f = StringIO() with redirect_stdout(f), redirect_stderr(f):

Please test and let me know.

[simonmeggle]

Hi Mike, already tried that. I even bypassed the whole robotmk plugin and tried to redirect stdout and stderr when executing robot itself on the commandline. Even then Chrome displays this log lines (and many more):

robot --console none sum_prices_headless.robot 2>&1 > NUL
DevTools listening on ws://127.0.0.1:52988/devtools/browser/96f16f4a-7e39-4226-898c-157344979ec4

For Chrome we have now a way to mute it completely. Let's see if there are other similar cases at all.

I'll soon update the pytest tests for the plugin with more meaningful tests which contain selenium webdriver and headless browsers. This is an important lession for me to write tests which do not only contain foobar and sleep :-)