Open ikappaki opened 3 weeks ago
This is an interesting one. I haven't really run tests outside of this repository yet, so not surprising I haven't run into this issue.
Aside from the conftest.py
trick, I imagine you could probably just throw an __init__.py
in tests/
and it might work as well? If not, we may need to do the same thing we did in #1036 with the basilisp.test
CLI entrypoint and add an empty entry to the beginning of sys.path
.
I think eventually whatever project tooling I end up building would do something similar to Leiningen or clj and can set the paths for tests and sources correctly.
Aside from the
conftest.py
trick, I imagine you could probably just throw an__init__.py
intests/
and it might work as well? If not, we may need to do the same thing we did in #1036 with thebasilisp.test
CLI entrypoint and add an empty entry to the beginning ofsys.path
.
I think the problem is with the basilisp testrunner. sys.path
already picks up the empty entry, and adding an additional __init__.py
doesn't help:
> basilisp run -c "(println sys/path)"
#py ["" ... "D:\\bas\\issuetests\\src"]
I've created a minimal repo at https://github.com/ikappaki/basilisp-issue-1069 to demonstrate in practice, with the following branches
main
: the minimal reproducible test case. A pytest
fails with the above error.issue/lpy-works-with-empty
: the workaround where you add an empty tests/conftest.py
and it magically works.issue/py-test-works
: replaced src/issuetests/issue_test.lpy
with a tests/issuetests/issue_py_test.py
, and the python test runs fine.checkout issue/still-fails-with-__init__.py
: added the additional tests/issuetests/__init__.py
file (the tests/__init__.py
was alredy present), the lpy test still fails.I think eventually whatever project tooling I end up building would do something similar to Leiningen or clj and can set the paths for tests and sources correctly.
That would be a great addition. However, I believe the issue here is related to the test runner: .py
tests work fine in this setup, but .lpy
tests don’t. There's also this unexplained workaround where you need tests/conftest
for the lpy tests to function properly.
Thanks
I think the problem is with the basilisp testrunner. sys.path already picks up the empty entry, and adding an additional init.py doesn't help:
It's harder to test this, but I didn't add the path fixes to basilisp test
(oops!) so it could still be path related. At least as a short term fix.
I think the problem is with the basilisp testrunner. sys.path already picks up the empty entry, and adding an additional init.py doesn't help:
It's harder to test this, but I didn't add the path fixes to
basilisp test
(oops!) so it could still be path related. At least as a short term fix.
The tests were mainly run using pytest
on the command line, so it’s not specific to basilisp test
. That was just to confirm it fails there too, so we can ignore basilisp test
as the primary source for now I think, unless I'm missing something fundamental (like , pytest
calling into basilisp test
?).
I think we can set this aside for now since the workaround of adding an empty tests/conftest.py
works. I'll try to look into the cause at some point.
thanks
@ikappaki I pulled down the repo to try to understand the issue. I'm noticing that when I do add the ""
empty path to the test
subcommand, that tests will run so long as you retain an empty tests/__init__.py
:
tests
├── __init__.py
└── issuetests
└── issue_test.lpy
=================================================================================================================================== test session starts ====================================================================================================================================
platform darwin -- Python 3.12.1, pytest-8.3.3, pluggy-1.5.0
rootdir: /Users/christopher/Projects/basilisp-issue-1069
configfile: pyproject.toml
plugins: basilisp-0.2.3
collected 1 item
tests/issuetests/issue_test.lpy . [100%]
==================================================================================================================================== 1 passed in 0.01s =====================================================================================================================================
I'm going to push a PR that makes the small change to the tests
subcommand and updates the documentation to include this requirement.
I'd like to fix this correctly in the future, but will need to think a bit more on how to do that.
@ikappaki I pulled down the repo to try to understand the issue. I'm noticing that when I do add the
""
empty path to thetest
subcommand, that tests will run so long as you retain an emptytests/__init__.py
:
Hi @chrisrink10, thanks for looking into this.
It works great now! The key piece I was missing was the #1075 fix that ensures basilisp test
properly passes its arguments to pytest
. Before this fix, I thought basilisp test
was mainly a convenience for basic scenarios, and that we still had to rely on pytest
for comprehensive testing with its wide range of command line options.
Now, I understand that testing should indeed be driven through basilisp test
.
I have two further suggestion, if you don't mind.
pytest
's command-line help from basilisp test -h
? Currently, users would have to switch back to pytest
to find out which arguments are accepted by basilisp test
, which feels a bit unintuitive.(basilisp-pprint-py3.11) PS C:\clj\basilisp-pprint> basilisp test -h
usage: basilisp test [-h] [--generate-auto-inlines [GENERATE_AUTO_INLINES]] [--inline-functions [INLINE_FUNCTIONS]]
[--warn-on-arity-mismatch [WARN_ON_ARITY_MISMATCH]]
[--warn-on-shadowed-name [WARN_ON_SHADOWED_NAME]] [--warn-on-shadowed-var [WARN_ON_SHADOWED_VAR]]
[--warn-on-unused-names [WARN_ON_UNUSED_NAMES]]
[--warn-on-non-dynamic-set [WARN_ON_NON_DYNAMIC_SET]]
[--use-var-indirection [USE_VAR_INDIRECTION]]
[--warn-on-var-indirection [WARN_ON_VAR_INDIRECTION]]
[--include-unsafe-path [INCLUDE_UNSAFE_PATH]] [-p INCLUDE_PATH]
[--data-readers-entry-points [DATA_READERS_ENTRY_POINTS]] [--disable-ns-cache [DISABLE_NS_CACHE]]
[--enable-logger [ENABLE_LOGGER]] [-l LOG_LEVEL]
[--emit-generated-python [EMIT_GENERATED_PYTHON]]
Run tests in a Basilisp project.
positional arguments:
args arguments passed on to Pytest
...
Another related thought: would it be possible to provide both brief and verbose help optoins for the CLI commands? at the moment, the output is cluttered with numerous compiler, runtime and debugging arguments, many of which would not be relevant when you are just looking for testing relating options, for example.
I'm going to push a PR that makes the small change to the
tests
subcommand and updates the documentation to include this requirement.I'd like to fix this correctly in the future, but will need to think a bit more on how to do that.
I'm very satisfied with the current fix, as things are now much clearer! Thanks again.
I have two further suggestion, if you don't mind.
- Could you highlight this more prominently int he documentation, perhaps in the testing section?
Seems reasonable!
- Would it be possible to display pytest's command-line help from basilisp test -h? Currently, users would have to switch back to pytest to find out which arguments are accepted by basilisp test, which feels a bit unintuitive.
I'm not sure if there's an easy way to do this that wouldn't be really ugly, but running basilisp test -- -h
emits the Pytest help text. I think we could definitely make it clearer that anything past --
goes to Pytest.
Another related thought: would it be possible to provide both brief and verbose help optoins for the CLI commands? at the moment, the output is cluttered with numerous compiler, runtime and debugging arguments, many of which would not be relevant when you are just looking for testing relating options, for example.
The main reason I didn't do this was because short arguments become non-mnemonic rather quickly and there are only so many letters available. If you had any suggestion for how to handle that, I'd certainly be willing to consider it.
- Would it be possible to display pytest's command-line help from basilisp test -h? Currently, users would have to switch back to pytest to find out which arguments are accepted by basilisp test, which feels a bit unintuitive.
I'm not sure if there's an easy way to do this that wouldn't be really ugly, but running
basilisp test -- -h
emits the Pytest help text. I think we could definitely make it clearer that anything past--
goes to Pytest.
I was not familiar with the --
convention, and even if I were, I would not have immediately thought of using -- -h
, as simple as it is😅. It would definitely be helpful to mention this in the help output.
Another related thought: would it be possible to provide both brief and verbose help optoins for the CLI commands? at the moment, the output is cluttered with numerous compiler, runtime and debugging arguments, many of which would not be relevant when you are just looking for testing relating options, for example.
The main reason I didn't do this was because short arguments become non-mnemonic rather quickly and there are only so many letters available. If you had any suggestion for how to handle that, I'd certainly be willing to consider it.
My apologies for the confusion, I misused the terminology. What I had in mind was that -h
would only return the specific options for the command. If the user wanted the full list of options, they could it request it separately, with something like -H
or -hh
or --uberhelp
or (even) --help
or something else along these lines. I am not sure how feasible these suggestions are though with the currently utilized command line options library.
Thanks
Hi,
it's not possible to execute
Basilisp
tests in a new project due to the following errorTo reproduce with
poetry
issuetests
project, addbasilisp
andpytest
deps, and install project./tests/issuetests/issue_test.lpy
(deftest xyz [] (is (= 5 5)))
The same error occurs when running
basilisp test
.A workaround I found to bypass this issue is to create an empty
conftest.py
file./tests/conftest.py
(Empty)though this is not mentioned in the official documentation as a requirement https://basilisp.readthedocs.io/en/v0.1.1/gettingstarted.html#project-structure
Running
basilisp test
gives an extra warning, but I this is unrelated to this issueThanks