rebar3 ct 'error_in_suite'

schnef commented 6 years ago

Environment

$ rebar3 report "my failing command"
Task: ct
Entered as:
  ct
-----------------
Operating System: x86_64-unknown-linux-gnu
ERTS: Erlang/OTP 20 [erts-9.1] [source] [64-bit] [smp:2:2] [ds:2:2:10] [async-threads:0] [hipe] [kernel-poll:false]
Root Directory: /home/frans/otp/20.1
Library directory: /home/frans/otp/20.1/lib
-----------------
Loaded Applications:
bbmustache: 1.3.0
certifi: 2.0.0
cf: 0.2.2
common_test: 1.15.2
compiler: 7.1.2
crypto: 4.1
cth_readable: 1.3.0
dialyzer: 3.2.2
edoc: 0.9.1
erlware_commons: 1.0.0
eunit: 2.3.4
eunit_formatters: 0.3.1
getopt: 0.8.2
inets: 6.4.2
kernel: 5.4
providers: 1.6.0
public_key: 1.5
relx: 3.23.1
sasl: 3.1
snmp: 5.2.7
ssl_verify_fun: 1.1.2
stdlib: 3.4.2
syntax_tools: 2.1.3
tools: 2.11

-----------------
Escript path: /home/frans/bin/rebar3
Providers:
  app_discovery as auto clean compile compile cover ct deps dialyzer do edoc escriptize eunit get-deps help install install_deps list lock new path pkgs proper release relup report shell state tar tree unlock update upgrade upgrade upgrade version xref

Current behaviour

Running ct gives the error {error,'.#rtps_SUITE can not be compiled or loaded'}. I guess this is because I have the file rtps_SUITE.erl open in emacs and the directory _build/test/lib/rtps/test/ contains some unexpected files:

$ ls -a _build/test/lib/rtps/test/
.#rtps.spec rtps.spec~ #rtps_SUITE.erl#
rtps.spec# rtps_SUITE.beam rtps_SUITE.erl
rtps.spec .#rtps_SUITE.erl rtps_SUITE.erl~

Files starting with a '.' probably should be ignored? The full error message is:

*** CT Error Notification 2017-10-06 08:39:09.869 ***
Error detected: '.#rtps_SUITE can not be compiled or loaded
*** CT 2017-10-06 08:39:09.890 *** Suite Hook
Call to CTH failed: error:{function_clause,
                           [{lists,reverse,
                             [[error_in_suite|undefined]],
                             [{file,"lists.erl"},{line,147}]},
                            {cth_readable_shell,format_path,2,
                             [{file,
                               "/home/tristan/Devel/rebar3/_build/default/lib/cth_readable/src/cth_readable_shell.erl"},
                              {line,273}]},
                            {cth_readable_shell,on_tc_fail,3,
                             [{file,
                               "/home/tristan/Devel/rebar3/_build/default/lib/cth_readable/src/cth_readable_shell.erl"},
                              {line,111}]},
                            {ct_hooks,catch_apply,3,
                             [{file,"ct_hooks.erl"},{line,424}]},
                            {ct_hooks,call_cleanup,3,
                             [{file,"ct_hooks.erl"},{line,187}]},
                            {ct_hooks,call,4,
                             [{file,"ct_hooks.erl"},{line,248}]},
                            {ct_hooks,call,4,
                             [{file,"ct_hooks.erl"},{line,251}]},
                            {ct_hooks,call,3,
                             [{file,"ct_hooks.erl"},{line,209}]}]}

Expected behaviour

Hidden files, starting with a '.' should be ignored.

Fix

In file rebar_prv_common_test.erl, line 526 has:

 SrcFiles = rebar_utils:find_files(From, ".*\\.[e|h]rl\$", false),

The regexp should maybe be:

"^[^\\.].*\\.[e|h]rl\$"

I am not familiar with git and know how to turn all this in a pull request.

ferd commented 6 years ago

That sounds like a reasonable avenue for the fix. Would you be willing to trying to submit a PR (with which I can help)? If not, I can see to address this issue, but it may be done over a different time line.

schnef commented 6 years ago

The change to the regexp does not fix the issue since filelib:fold_files/5 used by rebar_utils:find_files/3 already skips hidden files. In the situation that a test directory contains the test suites, the whole directory is copied, including hidden files and other junk. I have no idea how to fix the issue.

talentdeficit commented 6 years ago

are those files created by emacs? the correct place to handle this is probably in the logic we use to copy the test dirs, altho i'm not sure we should ignore dotfiles by default

schnef commented 6 years ago

Yes, these are open unsaved emacs files. So, when you have a test suite from the test directory open in emacs, the dot-files are created. Also, there are '#'-files and '~' files, but these wont cause trouble.

talentdeficit commented 6 years ago

okay so i think the appropriate place to handle this is with a filter in the compiler (rebar_prv_compile) that drops files that don't match some regex. we can make it configurable via rebar.config but i'm not sure what a good default value would be. i imagine .#blah.erl is not a valid source file name but there's probably other invalid filenames we should try to filter

schnef commented 6 years ago

To make it consistent with the behavior of filelib:fold/5 used for selecting tests suites in rebar_prv_common_test, I would suggest dropping all files starting with a dot, at least on non-windows file systems and not make it configurable. Making this configurable would introduce more complexity and probably will never be used. Also, it would leave people wondering why it doesn't work for individual selected test suites. Weird that filelib:fold/5 skips dot-files. What more hidden features does it have?

erlang / rebar3