Add automated reports to COMP

[andersonfrailey]

This PR adds the automated report generation capabilities available in releases 2.3.0 and beyond to COMP. It's still a work in progress because I'm working on making sure the directory the temporary files that are needed actually gets deleted.

[andersonfrailey]

I believe this is ready to go now. All tests pass locally and I've fixed the issue where the directory created to hold the report files wasn't getting deleted. Once it's merged I'll release a new version of Tax-Brain.

Question for @hdoupe, I've set it to read the PDF report into bytes and then return those bytes as part of the downloadable outputs. Is this the correct way to create a downloadable PDF on on C/S? cc @MattHJensen

[hdoupe]

Hey @andersonfrailey , this looks awesome. This looks good in theory, but I've never tested the full system with a PDF. I'm doing some development work this morning, and I'll spin up a TB instance with the automated reports and let you know how things work.

[hdoupe]

The cs-tests pass, but I'm getting this error when I run the model through the webapp. This looks like it might be something on the C/S side with where the gecko driver/path variable is being set. Once I do some more debugging, I'll report back:

Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/site-packages/cs_workers/models/executors/task_wrapper.py", line 69, in async_task_wrapper
    outputs = func(task_id, **(task_kwargs or {}))
  File "/opt/conda/lib/python3.7/site-packages/cs_workers/models/executors/job.py", line 17, in sim_handler
    outputs = functions.run_model(meta_param_dict, adjustment)
  File "/opt/conda/lib/python3.7/site-packages/cs_config/functions.py", line 190, in run_model
    report_outputs = report(tb, clean=True)
  File "/home/Tax-Brain/taxbrain/report.py", line 198, in report
    text_args["distribution_graph"] = export_plot(dist_graph, "dist")
  File "/home/Tax-Brain/taxbrain/report.py", line 59, in export_plot
    export_png(plot, filename=str(full_filename))
  File "/opt/conda/lib/python3.7/site-packages/bokeh/io/export.py", line 97, in export_png
    image = get_screenshot_as_png(obj, width=width, height=height, driver=webdriver, timeout=timeout)
  File "/opt/conda/lib/python3.7/site-packages/bokeh/io/export.py", line 201, in get_screenshot_as_png
    web_driver = driver if driver is not None else webdriver_control.get()
  File "/opt/conda/lib/python3.7/site-packages/bokeh/io/webdriver.py", line 118, in get
    self.current = self.create()
  File "/opt/conda/lib/python3.7/site-packages/bokeh/io/webdriver.py", line 122, in create
    driver = self._create(kind)
  File "/opt/conda/lib/python3.7/site-packages/bokeh/io/webdriver.py", line 140, in _create
    raise RuntimeError("Neither firefox and geckodriver nor a variant of chromium browser and " \
RuntimeError: Neither firefox and geckodriver nor a variant of chromium browser and chromedriver are available on system PATH. You can install the former with 'conda install -c conda-forge firefox geckodriver'.

[hdoupe]

Edit to earlier comment: The tests aren't passing locally. I still need to do some debugging to figure out why.

[andersonfrailey]

Any luck tracking the test failure down, @hdoupe?

[hdoupe]

@andersonfrailey Not yet, but I'll take a look at it this week!

[hdoupe]

@andersonfrailey Sorry for the long delay on this. I fought the good fight against Selenium and company this morning, and I think I'm making progress (although at what cost?!). I wound up having to install google chrome and chromedriver in the docker image as well as a bunch of other packages. However, I think things are going in the right direction:

https://github.com/andersonfrailey/Tax-Brain/compare/addreports...hdoupe:addreports-hdoupe

I'm stuck on this error right now, and will report back once I've had another chance to work on this:

_______________________________________________________________________ TestFunctions1.test_run_model _______________________________________________________________________

self = <test_functions.TestFunctions1 object at 0x7f6c61d76a90>

    def test_run_model(self):
        self.test_all_data_specified()
        inputs = self.get_inputs({})
        check_get_inputs(inputs)

        class MetaParams(Parameters):
            defaults = inputs["meta_parameters"]

        mp_spec = MetaParams().specification(serializable=True)

>       result = self.run_model(mp_spec, self.ok_adjustment)

/opt/conda/lib/python3.7/site-packages/cs_kit/validate.py:221: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/opt/conda/lib/python3.7/site-packages/cs_config/functions.py:190: in run_model
    report_outputs = report(tb, clean=True)
Tax-Brain/taxbrain/report.py:221: in report
    text_args["distribution_graph"] = export_plot(dist_graph, "dist")
Tax-Brain/taxbrain/report.py:79: in export_plot
    with get_webdriver() as webdriver:
/opt/conda/lib/python3.7/contextlib.py:112: in __enter__
    return next(self.gen)
Tax-Brain/taxbrain/report.py:33: in get_webdriver
    web_driver = Chrome(chrome_options=options)
/opt/conda/lib/python3.7/site-packages/selenium/webdriver/chrome/webdriver.py:73: in __init__
    self.service.start()
/opt/conda/lib/python3.7/site-packages/selenium/webdriver/common/service.py:98: in start
    self.assert_process_still_running()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <selenium.webdriver.chrome.service.Service object at 0x7f6c61075a10>

    def assert_process_still_running(self):
        return_code = self.process.poll()
        if return_code is not None:
            raise WebDriverException(
                'Service %s unexpectedly exited. Status code was: %s'
>               % (self.path, return_code)
            )
E           selenium.common.exceptions.WebDriverException: Message: Service chromedriver unexpectedly exited. Status code was: 127

/opt/conda/lib/python3.7/site-packages/selenium/webdriver/common/service.py:111: WebDriverException

[andersonfrailey]

Ah that's a heck of a battle. It seems like all of the plotting packages in Python I've used have a ton of additional dependencies that aren't easy to install or work with if you want to save visualizations automatically. Maybe it's time I finally start to work with matplotlib...

[hdoupe]

@andersonfrailey I've had some success with pyppeteer for taking screenshots of outputs, but that requires installing chromium. I think if you want to take screenshots then you just have to have a browser installed to render the HTML and JS.

Ha, matplotlib would be easier for saving the screenshots, but on the other hand, it's a bummer to have to swap technologies just to make it easier to grab the screenshots.

I can keep tinkering with this. I think the problem is that a couple dependencies are missing. So, I think we're close.

[andersonfrailey]

Lemme know how things go. If need be I'll figure out how to port everything to matplotlib.

[hdoupe]

It looks like there may be an issue with recent versions of the firefox and geckodriver conda packages. I tried installing older versions, but this didn't work either. That being said, I'm going to open an issue/comment in https://github.com/conda-forge/firefox-feedstock/issues/25 once I have a sec to look at this again. It'd be great to not have this blob of apt-get commands sitting in the Tax-Brain repo.

[andersonfrailey]

@hdoupe, I think I'm just going to re-write all of the plotting features using matplotlib. I don't want us to have to jump through hoops installing other packages to not break CS every time there's an update and the PNG files produced by matplotlib look much nicer than those from Bokeh. It'll take a bit of time to make them look as good, but I think it'll be worth it.

[hdoupe]

@andersonfrailey I'm sorry this has been such a hassle to get this on CS. I just opened an issue in the Firefox conda-forge repo to see if they have any advice: https://github.com/conda-forge/firefox-feedstock/issues/40

You're welcome to go ahead with using matplotlib. It'd be nice to get this to work. I'm sure other CS users would appreciate it. On the other hand, it has been quite a pain so far!

[andersonfrailey]

@hdoupe, I think with these latest commits, #134, and #128 we should be pretty much good to go on getting a new Tax-Brain release out and getting it up on CS. All of the taxbrain tests pass locally. I figure it'd be best to rely on #128 for seeing on the CS tests go. Lemme know if there's anything you want me to do/check before merging this.