ovh / venom

🐍 Manage and run your integration tests with efficiency - Venom run executors (script, HTTP Request, web, imap, etc... ) and assertions
Apache License 2.0
1.04k stars 142 forks source link

Assertion ShouldBeLessThan fails #267

Closed LuisHudson25 closed 3 years ago

LuisHudson25 commented 4 years ago

Hello,

It seems that ShouldBeLessThan assertion is not working.

I'm using venom v0.27.0 and the following example: `version: "2"

name: Test Assertions

testcases:

I get the error: "Expected '3' to be less than '10'". I'm not very good in maths, but I think there is a problem here :-)

I tried to use the function go assertions.ShouldBeLessThan and it's working as expected so I think it comes from the venom integration.

Thanks for this wonderful tool. Luis

kevinramage commented 4 years ago

Hello,

I reproduce the issue and the issue come from smartystreets/assertions module. When you compare 2 strings with assertions module, comparison is an alphabetical comparison and in alphabetical order 3 is before 1 (less_than.go line 114) So the assertion "result.systemout ShouldBeLessThan 10" failed and the assertion "result.systemout ShouldBeLessThan 9" passed. To resolve the issue, you must parse the string before send it to assertions module. To do it:

I'm not Venom product owner but I think this solution is acceptable ? @yesnault

Regards

yesnault commented 4 years ago

@kevinramage yes, it's the solution

kevinramage commented 4 years ago

I create PR-278 to fix the issue

yesnault commented 3 years ago

Fixed on master with https://github.com/ovh/venom/commit/47441c3f614fbe9b1aaa121aae611d3bf1b4c16e

Capture d’écran 2020-11-17 à 13 59 05

LuisHudson25 commented 3 years ago

Great, thank you both of you !