search

antonmi / espec

Elixir Behaviour Driven Development
Other
808 stars 62 forks source link

Fail if spec files cannot be compiled #303

Closed dolfinus closed 4 years ago

dolfinus commented 4 years ago

Currently, mix espec command return zero exit code in case when there is some error during spec files compilation step:

bash-5.0$ mix espec
Compiling 1 file (.ex)
==> arkenston

== Compilation error in file spec/arkenston/mutator/user_mutator_spec.ex ==
** (SyntaxError) spec/arkenston/mutator/user_mutator_spec.ex:67: unexpected token: ). The "do" at line 60 is missing terminator "end"
    (elixir 1.10.3) lib/kernel/parallel_compiler.ex:305: anonymous fn/4 in Kernel.ParallelCompiler.spawn_workers/7

        0 examples, 0 failures

        Finished in 0.37 seconds (0.36s on load, 0.0s on specs)

        Randomized with seed 22698
bash-5.0$ echo $?
0

This is very annoying because CI/CD process does not known that testing step was failed. The only way to detect such a failure is to check coverage percentage which is 0:

        0 examples, 0 failures

        Finished in 0.49 seconds (0.48s on load, 0.01s on specs)

        Randomized with seed 82688

----------------
COV    FILE                                        LINES RELEVANT   MISSED
  0.0% lib/arkenston/context.ex                       35        8        8
  0.0% lib/arkenston/guardian.ex                      47        5        5
  0.0% lib/arkenston/helper/auth_helper.ex            45       12       12
  0.0% lib/arkenston/helper/fields_helper.ex          64       15       15
  0.0% lib/arkenston/helper/query_helper.ex          289       36       36
  0.0% lib/arkenston/helper/struct_helper.ex          20        0        0
  0.0% lib/arkenston/helper/translation_helper.      156       55       55
  0.0% lib/arkenston/helper/uuid.ex                   36        8        8
  0.0% lib/arkenston/middleware/handle_fields.e       39        5        5
  0.0% lib/arkenston/mutator/author_mutator.ex        98       26       26
  0.0% lib/arkenston/mutator/user_mutator.ex         171       47       47
  0.0% lib/arkenston/mutator/user_token_mutator       75       15       15
  0.0% lib/arkenston/permissions.ex                   60       10       10
  0.0% lib/arkenston/permissions/author.ex           122       24       24
  0.0% lib/arkenston/permissions/user.ex             233       51       51
  0.0% lib/arkenston/repo.ex                         146       27       27
  0.0% lib/arkenston/resolver/author_resolver.e       39        4        4
  0.0% lib/arkenston/resolver/user_resolver.ex        37        3        3
  0.0% lib/arkenston/subject.ex                      303       12       12
  0.0% lib/arkenston/subject/author.ex               128       18       18
  0.0% lib/arkenston/subject/user.ex                 125       16       16
  0.0% lib/arkenston_web/error_view.ex                16        2        2
[TOTAL]   0.0%
----------------

This PR adds a check after compilation step, exit code will be 1:

bash-5.0$ mix espec
Compiling 1 file (.ex)
==> arkenston

== Compilation error in file spec/arkenston/mutator/user_mutator_spec.ex ==
** (SyntaxError) spec/arkenston/mutator/user_mutator_spec.ex:67: unexpected token: ). The "do" at line 60 is missing terminator "end"
    (elixir 1.10.3) lib/kernel/parallel_compiler.ex:305: anonymous fn/4 in Kernel.ParallelCompiler.spawn_workers/7
bash-5.0$ echo $?
1
antonmi commented 4 years ago

Great! Thank you, @dolfinus !