la10736 / rstest

Fixture-based test framework for Rust
Apache License 2.0
1.21k stars 43 forks source link

Strip common path prefix #247

Open ivan23kor opened 7 months ago

ivan23kor commented 7 months ago

Strip common path prefixes in test case names, generated with macro #files.

Implementation

After test file paths have been built relative to the base dir (CARGO_MANIFEST_DIR), find the maximum common prefix for all test paths and remove this prefix from the test case names.

Example

rstest/tests/rstest/mod.rs demonstrates the gist of the change:

Common prefix _files is stripped from the paths:

-        .ok("start_with_name::path_1__UP_files_test_sub_folder_from_parent_folder_txt")
-        .ok("start_with_name::path_2_files_element_0_txt")
-        .ok("start_with_name::path_3_files_element_1_txt")
-        .ok("start_with_name::path_4_files_element_2_txt")
-        .ok("start_with_name::path_5_files_element_3_txt")
-        .ok("start_with_name::path_6_files_sub_sub_dir_file_txt")
-        .ok("start_with_name_with_include::path_1_files__ignore_me_txt")
-        .ok("start_with_name_with_include::path_2_files_element_0_txt")
-        .ok("start_with_name_with_include::path_3_files_element_1_txt")
-        .ok("start_with_name_with_include::path_4_files_element_2_txt")
-        .ok("start_with_name_with_include::path_5_files_element_3_txt")
-        .ok("start_with_name_with_include::path_6_files_sub_sub_dir_file_txt")
+        .ok("start_with_name::path_1_files_test_sub_folder_from_parent_folder_txt")
+        .ok("start_with_name::path_2_rstest_files_files_element_0_txt")
+        .ok("start_with_name::path_3_rstest_files_files_element_1_txt")
+        .ok("start_with_name::path_4_rstest_files_files_element_2_txt")
+        .ok("start_with_name::path_5_rstest_files_files_element_3_txt")
+        .ok("start_with_name::path_6_rstest_files_files_sub_sub_dir_file_txt")
+        .ok("start_with_name_with_include::path_1__ignore_me_txt")
+        .ok("start_with_name_with_include::path_2_element_0_txt")
+        .ok("start_with_name_with_include::path_3_element_1_txt")
+        .ok("start_with_name_with_include::path_4_element_2_txt")
+        .ok("start_with_name_with_include::path_5_element_3_txt")
+        .ok("start_with_name_with_include::path_6_sub_sub_dir_file_txt")

Details

  1. Test cases for #files have been updated to expect common prefix stripped.
  2. Minor refactoring: use a simple literal expression for path values instead of conversion expression between String and PathBuf in tests for #files.

Addresses #212.

ivan23kor commented 7 months ago

@la10736 I just noticed through local testing that 7e1f611d9d8f9a53692f07fce34f2936925b7ca6 introduced unintended behaviour - it does not respect exclude glob. 278ccdc55d0332ba8d783b5ecc2e46e621e3873a still works. I am currently debugging.

la10736 commented 7 months ago

THX for this PR!! Really appreciated! Sorry, I need some time for review it... but I'll do it ASAP...

ivan23kor commented 7 months ago

No rush and thank you for getting to reviewing it.

ivan23kor commented 6 months ago

@la10736 could we hop on a call so this would get merged?

la10736 commented 6 months ago

Sorry, I was really busy in the last 2 weeks. Anyway I started to take a look to it and I'm not incline to merge it as it is. The original ticket was about to define how may segments to use. We can expand it to something like auto that fine the minimum path and your implementation go in this way.

I any case I don't want that the test name can leak some information about the file system outside of the project tree: for instance that means you cannot remove the __UP segments because in this case start_with_name::path_2_rstest_files_files_element_0_txt is leaking the folder project name.

I would like to introduce #[segments = (auto|none|number)] syntax where auto is find the root (but cannot go up from the crate root), none use all segments and a number define the maximum length from the end.

ivan23kor commented 6 months ago

I agree that prefix stripping should be optional and not the default behaviour. I will implement the #[segments = (auto|number)] that you suggested, I think it covers all the needed cases. @stonecharioteer, do you see any other variants for #[segments]?

I don't see a point in #[segments = none], as this is the default behaviour.

ivan23kor commented 6 months ago

About leaking paths: upstream rstest behaviour is already leaking paths, if test files are outside of the project root. Even if they are inside, file names are leaked.

I agree that prefix stripping exposes more information about paths than the current behaviour so I will document on the proposed #[segments = auto] that it can leak more path information, while #[segments = none] and #[segments = number] leak only what's inside the project tree (current upstream rstest behaviour).

ivan23kor commented 6 months ago

@la10736 thanks for getting to the review!

la10736 commented 6 months ago

Why don't maintain the actual path rendering (always start from the Cargo.toml position) and implement the 3 policies over on it:

  1. none leave all segments
  2. auto find and strip the common prefix (the default one)
  3. <number> take just the last number segments

It seams to me the best option. If you would I could help on it and write the parser part: just make the business based on a policy enum where the default is auto and I'll implement the parser stuffs.

la10736 commented 6 months ago

Just another small thing... please, can you fix the failed tests? I guess that you missed something when you rebased your PR.