search

E3SM-Project / e3sm_diags

E3SM Diagnostics package
https://e3sm-project.github.io/e3sm_diags
BSD 3-Clause "New" or "Revised" License
39 stars 32 forks source link

[Bug]: Latest version of `cdms2=3.1.5` with MPI build of `esmf>=8.4.0`/`esmpy>=8.4.0` breaks VS Code's Testing API with `[WARNING]: yaksa:` #737

Open tomvothecoder opened 11 months ago

tomvothecoder commented 11 months ago

Overview

Since cdms2 was patched with esmpy>=8.4.0 (includes esmf>=8.4.0), I have not been able to use VS Code's Testing API to write tests for e3sm_diags. When VS Code attempts to discover pytest unit tests, the [WARNING]: yaksa: 10 leaked handle pool object is raised immediately which VS Code interprets as an error.

It seems like this warning is related to MPI/MPICH and something in the MPI build of ESMF. I don't see any GitHub issue in the ESMF/esmpy repo about this, but did find these issues from other repos:

This issue is mostly an annoyance because it hinders my ability to write and debug unit tests efficiently.

VS Code Output

2023-10-02 15:22:27.186 [info] Discover tests for workspace name: e3sm_diags - uri: /Users/vo13/Documents/Repositories/E3SM-Project/e3sm_diags
2023-10-02 15:22:27.192 [info] > conda run -n e3sm_diags_dev_658 --no-capture-output python ~/.vscode/extensions/ms-python.python-2023.16.0/pythonFiles/get_output_via_markers.py ~/.vscode/extensions/ms-python.python-2023.16.0/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear --no-cov
2023-10-02 15:22:27.194 [info] cwd: .
2023-10-02 15:22:38.508 [error] Error discovering pytest tests:
 [n [Error]: [WARNING] yaksa: 10 leaked handle pool objects

    at ChildProcess.<anonymous> (/Users/vo13/.vscode/extensions/ms-python.python-2023.16.0/out/client/extension.js:2:241274)
    at Object.onceWrapper (node:events:628:26)
    at ChildProcess.emit (node:events:513:28)
    at maybeClose (node:internal/child_process:1091:16)
    at Socket.<anonymous> (node:internal/child_process:449:11)
    at Socket.emit (node:events:513:28)
    at Pipe.<anonymous> (node:net:322:12)]

Minimum Reproducible Example

I created dummy Python repo with just cdms2, esmpy (MPI build), and pytest in mamba env. I created some dummy Python files and a test file with a single unit test.

I tried to discover tests with VS Code but reproduced the same yaksa error.

2023-10-02 15:25:47.506 [info] Discover tests for workspace name: vscode-esmf - uri: /Users/vo13/Documents/Repositories/E3SM-Project/vscode-esmf
2023-10-02 15:25:47.515 [info] > conda run -n vscode-esmf --no-capture-output python ~/.vscode/extensions/ms-python.python-2023.16.0/pythonFiles/get_output_via_markers.py ~/.vscode/extensions/ms-python.python-2023.16.0/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear --no-cov
2023-10-02 15:25:47.515 [info] cwd: .
2023-10-02 15:25:49.262 [error] Error discovering pytest tests:
 [n [Error]: [WARNING] yaksa: 10 leaked handle pool objects

    at ChildProcess.<anonymous> (/Users/vo13/.vscode/extensions/ms-python.python-2023.16.0/out/client/extension.js:2:241274)
    at Object.onceWrapper (node:events:628:26)
    at ChildProcess.emit (node:events:513:28)
    at maybeClose (node:internal/child_process:1091:16)
    at ChildProcess._handle.onexit (node:internal/child_process:302:5)]

Possible Solutions

  1. Suppress C level warning for yaksa? -- is this possible and why is this error even being raised?
  2. Maybe open up an issue with esmf/esmpy/MPICH repo?

Workarounds

  1. Install the no MPI build of esmf (esmf >=8.4.0 nompi*) in the e3sm_diags dev env -- validated this works by using this yaml file
  2. Run pytest . via CLI while writing tests -- not my preferred method

Additional Context

There is a similar issue with the yaksa warning in xCDAT. However, VS Code's testing API works fine with xCDAT and the latest version of xESMF/esmpy because that warning is raised AFTER regridding tests are executed and not during the test discover phase.