[f] Explanation system

[catriuspham]

Status

Done

Related issues

https://github.com/EastAgile/robber.py/issues/39

Description

Let's implement an explanation system, where all the failure messages will have the following format, from which the second and the third line can be omitted.

BadExpectation:
A = ...
B = ...
C = ...
Expected A to {action} B. 
{additional information}

[hieueastagile]

@catriuspham The diff here is still unclear to me, here is what I got when I tried to compare two unequal dictionary.

Traceback (most recent call last):
  File "test-robber.py", line 8, in <module>
    'a': 1
  File "/home/hieu/.pyenv/versions/2.7.12/envs/robber.py/src/robber/robber/expect.py", line 44, in method
    return klass(self.obj, other, negative_fact, *args, **kwargs).fail_with(self.message).match()
  File "/home/hieu/.pyenv/versions/2.7.12/envs/robber.py/src/robber/robber/matchers/base.py", line 30, in match
    raise BadExpectation(message)
robber.bad_expectation.BadExpectation: 
A = {'a': 1, 'b': 2}
B = {'a': 1}
Expected A to equal B
Diffs:
- {'a': 1, 'b': 2}
+ {'a': 1}

I think it should be more clear, something like:

A = {'a': 1, 'b': 2}
B = {'a': 1}
Expected A to equal B
Diffs:
- {'a': 1, 'b': 2}
+ {'a': 1}
A has 'b' key but B don't

if they had same key, it should be:

A['a'] is 1 but B['b'] is 2

We need to that then the developer who use the library can know where to start fixing their test. Wdyt?

[hieueastagile]

Btw, can we somehow add the color in and remove the traceback?

[catriuspham]

@hieueastagile Great ideas, let's see what I can do

[catriuspham]

@hieueastagile @seri-ea @nhanbt

If you don't know what I'm talking about, please take a look at 3rd comment upward.

About the dict difference message. I'm thinking of 2 outputs. One is that you only notice the user about the very first difference this is what Sure is doing, for example:

d1 = {'a': 1, 'b': 2, 'c': 3, 'd': 4}
d2 = {'a': 1, 'd': 2, 'e': 3, 'f': 4}
expect(d1).to.eq(d2)

Output: A has the key 'b' while B does not

Another case is that you list out all the differences, for the above example, the output would be:

Output: 
A has the key(s) 'b', 'c' while B does not
A['d'] = 4 while B['d']=2

Both are fine for me, what's your preference?

[hieueastagile]

@catriuspham Both are fine. The idea that we want to give the user a clear enough message, then they can start to fix their test, so I think both, let's stick with the simplest option and we can extend later if needed.

[seri-ea]

Agree, both are fine, so whichever gives you an easy time to implement 😄

[coveralls]

Coverage remained the same at 100.0% when pulling f6738d78ec2d3e366e13bbc20df481eda53b4682 on feature/explaination-system into b439251848fd3b5085cdac45130311bfe0facc13 on master.

[coveralls]

Coverage decreased (-0.2%) to 99.798% when pulling eac2c4c21203dff85489d8dacca8789838d0e844 on feature/explaination-system into b439251848fd3b5085cdac45130311bfe0facc13 on master.

[coveralls]

Coverage remained the same at 100.0% when pulling 773714e7a7f1f09d7c390ea976dc7d495cb5291c on feature/explaination-system into b439251848fd3b5085cdac45130311bfe0facc13 on master.

[catriuspham]

@hieueastagile I cannot find a way to properly remove the traceback. With some lines of code, I can make it disappear on the default python shell, but ipython is another story as it has its own way to discover traceback, the same lines of code make ipython yell out some error messages, and it's really annoying. For the color, I can manually colorize each part of the message, however, we need further discussion on what color to use, which part to be colorized... Wdyt?

[hieueastagile]

@catriuspham : How about catching the exception and just print the message? I guess there's no traceback if we caught the exception? About color part, please take a look on sure and rspec of Rails. You can search for Youtube video to pick the color. In general, I think the rule is simple, green for whatever which passed and red for failing tests? Do we need any other colors?

[catriuspham]

"catching the exception and just print the message" means that we will have to get rid of BadExpectation, as there are a lot of tests involved, this is not a small task though. Moreover, our current logic checks if a chain is "matched" and raise exceptions directly, so there's no way to properly catch the exception here without refactoring the base class, which is another big chore. About the color, correct me if I'm wrong, but we currently print message "only" if some tests fail, which means every message will be in red. Also, I don't think sure colorize its messages.

[hieueastagile]

IIRC, sure had that. It could be your configuration for not seeing it, anyway, see this https://www.youtube.com/watch?v=gsgG-JvXsJ0 (8:14) to see how RSpec did that.

Ok, so let's ignore the traceback, just create a story and we can consider it later. But I wonder if sure print the traceback?

[catriuspham]

Sure does keeps the traceback.

[coveralls]

Coverage remained the same at 100.0% when pulling dbf9d9517612390eb1eacdb82f868be81e207732 on feature/explaination-system into b439251848fd3b5085cdac45130311bfe0facc13 on master.

[coveralls]

Coverage remained the same at 100.0% when pulling f8844f498844d521b6badc11ac45559fb5314cda on feature/explaination-system into b439251848fd3b5085cdac45130311bfe0facc13 on master.

[catriuspham]

@hieueastagile I added color to the message, please check if it's what you meant.

[catriuspham]

@hieueastagile About the color, I think making the message red is enough for now. Learning basics of Ruby and rspec needs time. I will raise an issue with more research and suggestions later. Also, this PR is quite big and lives for a long time (more than a month to be precise), keeping it around will decrease the work efficiency. Therefore, in my opinion, we'd better merge it and continue on other features.

[hieueastagile]

@catriuspham Ok, I think it's time to merge this. You can ask Huy to help on rspec if need to. @oyster Can you help us merge this branch?