[BUG] Tester doesn't accept legate parameters through command line

[marsaev]

Software versions

Python      :  3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0]
Platform    :  Linux-5.19.0-38-generic-x86_64-with-glibc2.31
Legion      :  legion-control-replication-final-143-g1a5e1d237
Legate      :  24.05.00.dev+31.g824425b6
Cunumeric   :  24.05.00.dev+11.g5552f835.dirty
Numpy       :  1.26.4
Scipy       :  1.12.0
Numba       :  0.59.1
CTK package :  cuda-version-12.4-h3060b56_3 (conda-forge)
GPU driver  :  545.23.08
GPU devices :  
  GPU 0: Quadro GV100
  GPU 1: NVIDIA Graphics Device

Jupyter notebook / Jupyter Lab version

No response

Expected behavior

Tester launches legate with provided command line parameters. Though i'm not exactly sure if that is expected behaviour.

Observed behavior

--capture=sys: error: unrecognized arguments: --logging legate=1

Example code or instructions

/opt/legate/cunumeric# python3 test.py --logging legate=1

This reproduces on integration tests. Examples work just fine though.

Stack traceback or browser console output

(legate) root@marsaev-sc-dt:/opt/legate/cunumeric# python3 test.py --logging legate=1

################################################################################
### 
### Test Suite Configuration (Python)
### 
### * Feature stages       : cpus
### * Test files per stage : 1
### * TestSystem description   : 10 cpus / 2 gpus
### 
################################################################################

################################################################################
### Entering stage: CPU (with 3 workers)
################################################################################

[FAIL] (CPU) 2.82s {01:50:40.44, 01:50:43.26} tests/integration/test_astype.py (exit: 4) 
                                        CPU: Passed 0 of 1 tests (0.0%) in 2.83s

################################################################################
### 
### Exiting stage: CPU
### 
### * Results      : 0 / 1 files passed (0.0%)
### * Elapsed time : 0:00:02.827802
### 
################################################################################

    ----------------------------------------------------------------------------

################################################################################
### FAILURES
################################################################################

[FAIL] (CPU) 2.82s {01:50:40.44, 01:50:43.26} tests/integration/test_astype.py (exit: 4) 
   ERROR: usage: --capture=sys [options] [file_or_dir] [file_or_dir] [...]
   --capture=sys: error: unrecognized arguments: --logging legate=1
     inifile: /opt/legate/cunumeric/pyproject.toml
     rootdir: /opt/legate/cunumeric

   /opt/conda/envs/legate/lib/python3.10/site-packages/_pytest/config/__init__.py:331: PluggyTeardownRaisedWarning: A plugin raised an exception during an old-style hookwrapper teardown.
   Plugin: helpconfig, Hook: pytest_cmdline_parse
   UsageError: usage: --capture=sys [options] [file_or_dir] [file_or_dir] [...]
   --capture=sys: error: unrecognized arguments: --logging legate=1
     inifile: /opt/legate/cunumeric/pyproject.toml
     rootdir: /opt/legate/cunumeric
   For more information see https://pluggy.readthedocs.io/en/stable/api_reference.html#pluggy.PluggyTeardownRaisedWarning
     config = pluginmanager.hook.pytest_cmdline_parse(
   [0 - 7fa671297000]    0.000082 {4}{threads}: reservation ('CPU proc 1d00000000000003') cannot be satisfied

################################################################################
### 
### Overall summary
### 
### * CPU   : 0 / 1 passed in 2.83s
### 
### All tests: Passed 0 of 1 tests (0.0%) in 2.83s
### 
################################################################################

[bryevdv]

Copying response from slack:

That error is coming from pytest which consumes argv I am not sure offhand that command line can be made to work here, unless we want to explicitly and manually duplicate legate options in the tester (which already has ~40 command line options of its own). But LEGATE_CONFIG is always an option and I would say even the preferred option. Even if we duplicated the command options in the tester, it would just use convert them to pass via LEGATE_CONFIG

There's at least five different things potentially competing over argv in a completely uncoordinated manner in this instance (the tester, pytest, the test code itself, legate, and realm) which is why this is such a horrible mess and why using LEGATE_CONFIG is preferred