parroty / excoveralls

Coverage report tool for Elixir with coveralls.io integration.
MIT License
827 stars 178 forks source link

Support for :rpc.block_call/4 #110

Open Eiji7 opened 7 years ago

Eiji7 commented 7 years ago

Hey, do you plan to support :rpc.block_call/4? I have a code that tests multiple local nodes using GenServer state. Check my cluster code: cluster.ex and test for it in multiple_nodes_test.exs. My coverage report is almost 100% except that one thing, so I would like to finish coverage work.

parroty commented 7 years ago

Thanks for the comment. I haven't been able to understand enough about the relationship between :rpc.block_call/4 and uncovered code. Could you elaborate a little bit?

When I locally tried the testing, I'm facing error on multiple_node_test.exs. The "except that one thing" is around handle_info code? (or not relevant?)


  1) test multiple_nodes (ExApi.MultipleNodesTest)
     test/ex_api/multiple_nodes_test.exs:13
     ** (EXIT from #PID<0.594.0>) :not_alive

...

----------------
COV    FILE                                        LINES RELEVANT   MISSED
 92.5% lib/ex_api.ex                                 408       80        6
100.0% lib/ex_api/api.ex                             179        2        0
100.0% lib/ex_api/application.ex                      14        4        0
100.0% lib/ex_api/error.ex                           217       30        0
  0.0% lib/ex_api/event.ex                            36        0        0
100.0% lib/ex_api/feature.ex                          79       16        0
100.0% lib/ex_api/implementation.ex                  155        2        0
100.0% lib/ex_api/kernel.ex                          100        8        0
100.0% lib/ex_api/spec.ex                            144       35        0
100.0% lib/ex_api/validation_helper.ex               169       47        0
[TOTAL]  97.3%
----------------
Eiji7 commented 7 years ago

@parroty: Sure, ExApi module is using a GenServer. All server casts sends event when everything goes well. Each event is send using Phoenix.PubSub.

In tests I'm creating two nodes: first@127.0.0.1 and second@127.0.0.1. After successful initialize and connect that nodes I'm calling server api (again cast functions), so first node sends event using Phoenix.PubSub and that event is catched in second node. Rest of uncovered code in ex_api.ex handles that event and calls do_cast function (cast without sending event), so coverage for ex_api.ex file should be 100%.

parroty commented 7 years ago

Thanks for the description. It might exist some workaround, but I'm not sure enough yet (need investigation with simpler example). ExCoveralls delegates the coverage tracking to erlang's :cover module and the test target codes need to be tapped in the erlang vm.

I think, multi-node test case is a little tricky for unit testing / coverage reporting. How do you think about adding test case for simply calling the corresponding GenServer endpoints?

Also, suggestion is appreciated too :-).

Eiji7 commented 7 years ago

@parroty: I don't want to test do_cast, because it's already well tested. I want to test sharing server state (by sending and receiving events). I will create repo with simpler example.

Eiji7 commented 7 years ago

@parroty: I created repository: https://gitlab.com/ex-open-source/multiple_nodes

Note: There is a simple workaround. It's possible to change file test/multiple_nodes_test.exs at line 12 to:

Cluster.block_call(List.first(@node_names), MultipleNodes, :example, [data])

This workaround calls function that sends data from first node (not on current node), so all other nodes (including current node) will receive this message, but this still does not solves main problem i.e. test code called on another node.

parroty commented 7 years ago

Thanks for the example. At the moment, I don't have good idea for resolving the issue 🤔. I appreciate if you have any viewpoints to pursue. By the way, I'm facing following error in my local environment. Do you have any insight?


➜  multiple_nodes git:(master) iex -S mix
Erlang/OTP 19 [erts-8.2] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Compiling 3 files (.ex)
Generated multiple_nodes app

=INFO REPORT==== 22-Aug-2017::23:23:02 ===
    application: logger
    exited: stopped
    type: temporary
** (Mix) Could not start application multiple_nodes: MultipleNodes.Application.start(:normal, []) returned an error: an exception was raised:
    ** (ArgumentError) argument error
        (elixir) lib/supervisor/spec.ex:175: anonymous fn/1 in Supervisor.Spec.supervise/2
        (elixir) lib/enum.ex:1255: Enum."-map/2-lists^map/1-0-"/2
        (elixir) lib/enum.ex:1255: Enum."-map/2-lists^map/1-0-"/2
        (elixir) lib/supervisor/spec.ex:175: Supervisor.Spec.supervise/2
        (stdlib) supervisor.erl:294: :supervisor.init/1
        (stdlib) gen_server.erl:328: :gen_server.init_it/6
        (stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3```
Eiji7 commented 7 years ago

@parroty: I don't have that error, but I think it's because you are using old version of Elixir. I'm always up to date and have Erlang 20.0 and Elixir 1.5.1-otp-20. I believe that you have a problem with application in mix.exs - especially :extra_applications. Please, see: https://sergiotapia.me/application-inference-in-elixir-1-4-ae9e43e90301

milkwine commented 6 years ago

Is :cover.start/1, :cover.stop/1 can help? http://erlang.org/doc/man/cover.html

Eiji7 commented 6 years ago

@milkwine: I do not checked it, but documentation looks good. However it would be awesome if there would be a way to call them automatically. Do you have any idea for this? To be honest I do not looked on documentation enough deeply. I just reported that it could be solved (of course if possible). Also I'm not sure how well it plays with this library. I think that @parroty could give here more information.