[BUG]: getDeployDirectory returns

[gerth2]

Problem description

wpilib.getDeployDirectory() is returning tests\deploy for the deploy directory, rather than just deploy. It looks like the underlying wpilib implementation doesn't account for the fact that the working directory for tests is not the same folder as robot.py?

Currently, we're using pathplanner to store .path files in the deploy directory. However, when running the test suite, the code that works on the robot fails due to the extra test in the path.

I think that, at least, during tests, the python implementation might need to be unique to make sure it points back at the correct folder, regardless of where the code is running from.

Operating System

Windows

Installed Python Packages

Package                   Version
------------------------- -----------
2to3                      1.0
anyio                     3.6.2
argon2-cffi               21.3.0
argon2-cffi-bindings      21.2.0
arrow                     1.2.3
astroid                   2.15.2
asttokens                 2.2.1
attrs                     23.1.0
backcall                  0.2.0
bcrypt                    4.0.1
beautifulsoup4            4.12.2
bleach                    6.0.0
cffi                      1.15.1
click                     8.1.3
colorama                  0.4.6
comm                      0.1.3
coverage                  7.2.3
cryptography              40.0.2
debugpy                   1.6.7
decorator                 5.1.1
defusedxml                0.7.1
dill                      0.3.6
executing                 1.2.0
fastjsonschema            2.16.3
fqdn                      1.5.1
idna                      3.4
importlib-metadata        6.8.0
importlib-resources       6.1.0
iniconfig                 2.0.0
ipykernel                 6.22.0
ipython                   8.12.0
ipython-genutils          0.2.0
ipywidgets                8.0.6
isoduration               20.11.0
isort                     5.12.0
jedi                      0.18.2
Jinja2                    3.1.2
jsonpointer               2.3
jsonschema                4.17.3
jupyter                   1.0.0
jupyter_client            8.2.0
jupyter-console           6.6.3
jupyter_core              5.3.0
jupyter-events            0.6.3
jupyter_server            2.5.0
jupyter_server_terminals  0.4.4
jupyterlab-pygments       0.2.2
jupyterlab-widgets        3.0.7
lazy-object-proxy         1.9.0
libusb                    1.0.26b5
libusb-package            1.0.26.2
markdown-it-py            3.0.0
MarkupSafe                2.1.2
matplotlib-inline         0.1.6
mccabe                    0.7.0
mdurl                     0.1.2
mistune                   2.0.5
nbclassic                 0.5.5
nbclient                  0.7.3
nbconvert                 7.3.1
nbformat                  5.8.0
nest-asyncio              1.5.6
notebook                  6.5.4
notebook_shim             0.2.3
packaging                 23.1
pandocfilters             1.5.0
paramiko                  3.1.0
parso                     0.8.3
pickleshare               0.7.5
Pint                      0.20.1
pip                       23.1.2
pkg-about                 1.0.8
platformdirs              3.2.0
pluggy                    1.0.0
prometheus-client         0.16.0
prompt-toolkit            3.0.38
psutil                    5.9.5
pure-eval                 0.2.2
pycparser                 2.21
pyfrc                     2023.0.1
pyfu-usb                  1.0.2
Pygments                  2.15.1
pylint                    2.17.2
PyNaCl                    1.5.0
pynetconsole              2.0.4
pyntcore                  2023.4.3.0
pyrsistent                0.19.3
pytest                    7.3.1
pytest-reraise            2.1.2
python-dateutil           2.8.2
python-json-logger        2.0.7
pyusb                     1.2.1
pywin32                   306
pywinpty                  2.0.10
PyYAML                    6.0
pyzmq                     25.0.2
qtconsole                 5.4.2
QtPy                      2.3.1
rfc3339-validator         0.1.4
rfc3986-validator         0.1.1
rich                      13.6.0
robotpy                   2023.4.3.0
robotpy-apriltag          2023.4.3.0
robotpy-commands-v2       2023.4.3.0
robotpy-cscore            2023.4.3.0
robotpy-ctre              2023.1.0
robotpy-hal               2023.4.3.0
robotpy-halsim-ds-socket  2023.4.3.0
robotpy-halsim-gui        2023.4.3.0
robotpy-halsim-ws         2023.4.3.0
robotpy-installer         2023.0.3
robotpy-navx              2023.0.3
robotpy-pathplannerlib    2023.3.4.1
robotpy-photonvision      2023.4.2
robotpy-playingwithfusion 2023.1.0
robotpy-rev               2023.1.3.2
robotpy-wpilib-utilities  2023.1.0
robotpy-wpimath           2023.4.3.0
robotpy-wpinet            2023.4.3.0
robotpy-wpiutil           2023.4.3.0
Send2Trash                1.8.0
setuptools                65.5.0
six                       1.16.0
sniffio                   1.3.0
soupsieve                 2.4.1
stack-data                0.6.2
terminado                 0.17.1
tinycss2                  1.2.1
tomli                     2.0.1
tomlkit                   0.11.7
tornado                   6.3.1
traitlets                 5.9.0
uri-template              1.2.0
usb                       0.0.83.dev0
wcwidth                   0.2.6
webcolors                 1.13
webencodings              0.5.1
websocket-client          1.5.1
widgetsnbextension        4.0.7
wpilib                    2023.4.3.0
wrapt                     1.15.0
zipp                      3.17.0

Reproducible example code

in robot_init:

wpilib.getDeployDirectory()

[virtuald]

RobotPy uses its own implementation of getDeployDirectory (see https://github.com/robotpy/mostrobotpy/blob/main/subprojects/robotpy-wpilib/wpilib/src/rpy/Filesystem.inc), and the location of the directory is based on where python determines the __main__ module is located, assuming that __main__ has a __file__ attribute.

I tried this on Linux using ./robot.py test and it prints out the expected directory for me. How are you launching your tests?

[gerth2]

I ran more experiments, it looks like it changes based on how I launch.

I created a single testcase and tried running it a few ways, observing the file it put to disk and the contents of that file.

# pylint: disable-all

from wpilib import getDeployDirectory
import logging

def test_deployDirectory():
    with open("deploydirlog.txt", "w") as f:
        deployDir = getDeployDirectory()
        f.write(deployDir)

If I do the same experiment you ran, it is correct.

The results I had were from launching my test suite under vsCode's python debug environment:

        {
            "name": "Test",
            "type": "python",
            "request": "launch",
            "program": "robot.py",
            "console": "integratedTerminal",
            "justMyCode": true,
            "args": [
                "test"
            ],
            "cwd": "${workspaceFolder}"
        }

In this case, the deploydirlog.txt file shows up in the test folder, so something is changing directory there.

I get similarly wonky results when using an extension for running pytest, and clicking "run" next to the single testcase:

c:\Users\chris.vscode\extensions\ms-python.python-2023.20.0\pythonFiles\vscode_pytest\deploy

However, this also does not run the test in the tests folder, rather one directory up, next to robot.py

Looks like vsCode's debug environment or extentions is both messing with where __main__ is at, and possibly working directory?

[gerth2]

maximal reproducible example here from my experiments: https://github.com/RobotCasserole1736/firstRoboPy/tree/chris_test_deployDir

[virtuald]

The only supported way to run pytest tests on RobotPy code is via ./robot.py test, because of the amount of cleanup code necessary -- specifically, see https://github.com/robotpy/pyfrc/blob/b8472df58739031b60c384f3304c6cb56e582f19/pyfrc/test_support/pytest_plugin.py#L114; so for the case where you run pytest directly we can't really fix.

When running python robot.py test via vscode there must be something incorrect about the way that vscode is loading the launched file. According to SO it seems that the main module's file path is made absolute as of Python 3.9, but when running under vscode's launcher this isn't the case.

We could work around this at https://github.com/robotpy/pyfrc/blob/main/pyfrc/mains/cli_test.py#L75 by adding the following:

# In some cases __main__.__file__ is not an absolute path, and some
# internals depend on that being correct. Set it up before we change
# directories
sys.modules["__main__"].__file__ = abspath(sys.modules["__main__"].__file__)

But I'm not particularly happy about that solution. Open to other ideas?

[virtuald]

This should be fixed in the kickoff release due to changes in how the robot code is launched.