awslabs / benchmark-ai

Anubis (formerly known as Benchmark AI), measures the goodness of machine learning workloads
Apache License 2.0
16 stars 6 forks source link

Test failures for job-status-trigger and sm-executor modules #1037

Closed tejaschumbalkar closed 3 years ago

tejaschumbalkar commented 3 years ago

Hello,

Since yesterday, I am seeing the CodeBuild failures on PR #1036 and my anubis pipeline.

Locally, checking out the master branch and running the test on job-status-trigger and sm-executor modules, same pytest failures were seen. It seems to be a problem with the master branch.

Also, tried to switch the pipeline source to master branch, the same errors were seen

Below is the steps to reproduce:

  1. sm-executor module
    $ git checkout master
    $ cd sm-executor
    $ make

    This would lead to the following failures

========================================================================= short test summary info ==========================================================================
FAILED tests/sm_executor/test_sm_execution_engine.py::test_run_invalid_descriptor - TypeError: not all arguments converted during string formatting
FAILED tests/sm_executor/test_sm_execution_engine.py::test_run_fail_create_estimator - TypeError: not all arguments converted during string formatting
FAILED tests/sm_executor/test_sm_execution_engine.py::test_cancel_raises_not_found - TypeError: not all arguments converted during string formatting
FAILED tests/sm_executor/test_sm_execution_engine.py::test_cancel_raises - TypeError: not all arguments converted during string formatting
======================================================================= 4 failed, 15 passed in 5.52s =======================================================================
  1. job-status-trigger module
$ git checkout master
$ cd job-status-trigger
$ make

This would lead to the following failures

========================================================================= short test summary info ==========================================================================
FAILED tests/test_main.py::test_main_error - TypeError: not all arguments converted during string formatting
================================================================= 1 failed, 17 passed, 2 warnings in 2.33s =================================================================

Can you please look into the issue as I am blocked for #1036

tejaschumbalkar commented 3 years ago

The root cause of the pytest failures was due an exception thrown by python logger

TypeError: not all arguments converted during string formatting

Fixing the issue by using %-formatting

logger.exception("error %s", err)
tejaschumbalkar commented 3 years ago

Fix incorporated in #1042