zaufi
commented
9 months ago As for me, losing the uniqueness of filenames is a big pain in the ass. First of all, cuz an end user has to understand the implementation details of pytest-matcher to understand the rules of shortening those filenames. The very restrictive side effect of this is that pytest-matcher has to pay extra attention to the backward compatibility if we want to change 'em somehow. Secondly, it means pytest-matcher has to have very clean docs about everything %)
So, personally, I'd prefer to offload this pain to the user's ass ;) If he so dares to have such a corner test cases %)
There are few random thughts came to my mind...
Using a hash of a longest part of a filename (callspec-id). I.e., having some option (e.g., smth like --pm-parameterized-test-use-hash) expected_out instance would use one of the hash algorithms to produce <test-function-name>[hash(callspec-id)]. Possibly it can be controlled per test by setting some property on the fixture. E.g., smth like this:
@pytest.mark.parametrize(...)
def test_parametrized(..., capfd, expected_out):
expected_out.use_hash_to_match_callspec = True
...Still possible that the craziest users may have other parts used to make a full path also very-very long (e.g., a class name or a method/function name)...
Give a user total control over path/filename of expectation files. E.g., add an option to the INI file (e.g., pm-expectation-filename-template) that will be processed by Jinja2:
[tool.pytest.ini_options]
pm-expectation-filename-template = "{{base_dir}}/{{class_name|replace('_tester', '')}}/{{function_name|replace('_test', '')}}[{{callspec_id|md5}}]{{system_sfx}}.pattern"And the terminal state of this idea is to give him the ability to specify templates per test case...
xymaxim
As stated in #4, the callspec IDs can be long enough to exceed the
PC_NAME_MAXvalue (and cause an exception).Before creating a pull request, I would like to discuss some details.
Let's say we have a test function with two parametrized arguments and the corresponding node names:
1. Which parts should be truncated?
To have identical filenames with and without using the
--pm-use-system-prefixoption on different platforms, the filename truncation should be limited to a node name. To give an idea, here is an example of shortening the node names to 37 characters:2. Setting the maximum length
While we can set a default value to, say, 100 characters, it would be useful to make this value customizable via the INI file by adding the corresponding field, e.g.
pm-filename-max-node-length.3. Resolving duplicates
After truncation, non-unique filenames may appear. I see two options here.
(a) Use an incremental numbering.
By adding a number just after a callspec ID:
By adding a number suffix to the end of filename:
(b) Use indices of the parametrized arguments:
This will allow identifying used argument values more clearly:
For stacked
parametrizedecorators, the values will not only be mirrored, e.g. '10', '01', ...