Float comparisons failing on unit tests with sebastian/comparator >= 4.0.8

ghost commented 1 year ago

Preconditions and environment

2.4.3-p2
PHPUnit environment with sebastian/comparator >= 4.0.8

Steps to reproduce

Run the MFTF unit tests over Sales, SalesRule and Tax modules.

Expected result

Tests pass.

Actual result

Tests fail on float comparisons

There were 14 failures:
1) Magento\SalesRule\Test\Unit\Model\Rule\Action\Discount\ToPercentTest::testCalculate with data set #0 (3, array(30, 5), array(10, 50, 55), array(50, 45, 60, 55), 100, array(98, 59.5, 119, 80.5))
Expectation failed for method name is "setBaseAmount" when invoked 1 time(s)
Parameter 0 for invocation Magento\SalesRule\Model\Rule\Action\Discount\Data::setBaseAmount(59.49999999999999) does not match expected value.
Failed asserting that 59.49999999999999 matches expected 59.5.
vendor/magento/module-sales-rule/Model/Rule/Action/Discount/ByPercent.php:58
vendor/magento/module-sales-rule/Model/Rule/Action/Discount/ToPercent.php:19
vendor/magento/module-sales-rule/Test/Unit/Model/Rule/Action/Discount/ToPercentTest.php:204
2) Magento\Sales\Test\Unit\Model\Order\Creditmemo\Total\TaxTest::testCollect with data set "tax_amount_not_over_invoiced_tax_amount" (array(array(1.24, 1.24, 0, 0, 16.09, 16.09, 11.14, 11.14, 0, 0, 30)), array(array(array(array(2, 8.26, 0, 8.26, 0, 0, 0, 0), false, 2)), false, array(136.65, 136.65, 30, 1.65, 1.65, Mock_Invoice_f7f01ac5 Object (...))), array(array(array(8.26, 8.26)), array(146.14, 146.14, 11.14, 11.14, 1.24, 1.24)))
Failed asserting that 146.14000000000001 matches expected 146.14.
vendor/magento/module-sales/Test/Unit/Model/Order/Creditmemo/Total/TaxTest.php:121
3) Magento\Sales\Test\Unit\Model\Order\Creditmemo\Total\TaxTest::testCollect with data set "last_partial_creditmemo" (array(array(1.24, 1.24, 1.24, 1.24, 0, 0, 16.09, 16.09, 16.09, 16.09, 11.14, 11.14, 15, 15, 15, 15)), array(array(array(array(3, 12.38, 8.26, 12.38, 8.26, 0, 0, 2), true, 1)), true, array(60.82, 60.82, 0, 0.82, 0.82, Mock_Invoice_f7f01ac5 Object (...))), array(array(array(4.12, 4.12)), array(64.94, 64.94, 4.94, 4.94)))
Failed asserting that 4.940000000000001 matches expected 4.94.
vendor/magento/module-sales/Test/Unit/Model/Order/Creditmemo/Total/TaxTest.php:121
4) Magento\Sales\Test\Unit\Model\Order\Creditmemo\Total\TaxTest::testCollect with data set "partial_invoice_partial_creditmemo_price_incl_tax" (array(array(0, 0, 0, 0, 0, 0, 13.72, 13.72, 9.15, 9.15, 0, 0, 11.43, 11.43, 7.62, 7.62, 0, 0, 0, 0, 0, 0)), array(array(array(array(2, 7.62, 0, 7.62, 0, 11.43, 11.43, 7.62, 7.62, 0), false, 1)), false, array(52.38, 52.38, 0, 0.76, 0.76)), array(array(array(3.81, 3.81)), array(60, 60, 4.57, 4.57)))
Failed asserting that 60.00000000000001 matches expected 60.
vendor/magento/module-sales/Test/Unit/Model/Order/Creditmemo/Total/TaxTest.php:121
5) Magento\Sales\Test\Unit\Model\Order\Creditmemo\Total\TaxTest::testCollect with data set "last_partial_creditmemo_with_partial_shipping_refund" (array(array(1.24, 1.24, 0, 0, 0, 0, 16.09, 16.09, 16.09, 16.09, 9.9, 9.9, 15, 0, 15, 0)), array(array(array(array(3, 12.38, 8.26, 12.38, 8.26, 0, 0, 2), true, 1)), true, array(7.5, 7.5, 60.82, 60.82, 0.82, 0.82, Mock_Invoice_f7f01ac5 Object (...))), array(array(array(4.12, 4.12)), array(65.56, 65.56, 5.56, 5.56)))
Failed asserting that 5.560000000000001 matches expected 5.56.
vendor/magento/module-sales/Test/Unit/Model/Order/Creditmemo/Total/TaxTest.php:121
6) Magento\Sales\Test\Unit\Model\Order\Invoice\Total\TaxTest::testCollect with data set "partial_invoice" (array(array(), array(2.45, 2.45, 0, 0, 53.56, 53.56)), array(array(array(array(3, 24.32, 0, 24.32, 0, 0, 0, 0), false, 1), array(array(3, 24.33, 0, 24.33, 0, 0, 0, 0), false, 2)), false, array(327, 327)), array(array(array(8.11, 8.11), array(16.22, 16.22)), array(353.78, 353.78, 26.78, 26.78)))
Failed asserting that 26.779999999999998 matches expected 26.78.
vendor/magento/module-sales/Test/Unit/Model/Order/Invoice/Total/TaxTest.php:118
7) Magento\Sales\Test\Unit\Model\Order\Invoice\Total\TaxTest::testCollect with data set "partial_invoice_last_invoice" (array(array(array(30, false)), array(2.45, 2.45, 0, 0, 53.56, 53.56)), array(array(array(array(3, 24.32, 16.22, 24.32, 16.22, 0, 0, 2), true, 1), array(array(3, 24.33, 16.22, 24.33, 16.22, 0, 0, 2), true, 1)), false, array(198, 198)), array(array(array(8.1, 8.1), array(8.11, 8.11)), array(214.21, 214.21, 16.21, 16.21)))
Failed asserting that 8.100000000000001 matches expected 8.1.
vendor/magento/module-sales/Test/Unit/Model/Order/Invoice/Total/TaxTest.php:124
8) Magento\Tax\Test\Unit\Model\Calculation\RowBaseCalculatorTest::testCalculateWithTaxInPrice
Failed asserting that 495.49549549549545 matches expected 495.49549549545.
vendor/magento/module-tax/Test/Unit/Model/Calculation/RowBaseCalculatorTest.php:36
9) Magento\Tax\Test\Unit\Model\Calculation\TotalBaseCalculatorTest::testCalculateWithTaxInPriceNoRounding
Failed asserting that 495.49549549549545 matches expected 495.49549549545.
vendor/magento/module-tax/Test/Unit/Model/Calculation/TotalBaseCalculatorTest.php:44
10) Magento\Tax\Test\Unit\Model\Calculation\UnitBaseCalculatorTest::testCalculateWithTaxInPrice
Failed asserting that 495.49549549549545 matches expected 495.4954954955.
vendor/magento/module-tax/Test/Unit/Model/Calculation/UnitBaseCalculatorTest.php:170
11) Magento\Tax\Test\Unit\Model\Plugin\OrderSaveTest::testAfterSave with data set "item_with_shipping_three_tax" (array(array(0.66, 0.66, 11, 'ILUS', array(array(array(6, 'IL', 'IL'), array(5, 'US', 'US')))), array(0.2, 0.2, 3.33, 'CityTax', array(array(array(3, 'CityTax', 'CityTax'))))), array(array(151, 'product', null, array(array(0.11, 0.11, 11, 'ILUS', array(array(array(6, 'IL', 'IL'), array(5, 'US', 'US')))), array(0.03, 0.03, 3.33, 'CityTax', array(array(array(3, 'CityTax', 'CityTax')))))), array(null, 'shipping', null, array(array(0.55, 0.55, 11, 'ILUS', array(array(array(6, 'IL', 'IL'), array(5, 'US', 'US')))), array(0.17, 0.17, 3.33, 'CityTax', array(array(array(3, 'CityTax', 'CityTax'))))))), array(array(123, 'IL', 'IL', 0, 6, 0, 0, 0.66, 0.66, 0, 0.36), array(123, 'US', 'US', 0, 5, 0, 0, 0.66, 0.66, 0, 0.3), array(123, 'CityTax', 'CityTax', 0, 3, 0, 0, 0.2, 0.2, 0, 0.18018018018018)), array(array(116, '35', 6, null, 0.11, 0.11, 0.06, 0.06, 'product'), array(null, '35', 6, null, 0.55, 0.55, 0.3, 0.3, 'shipping'), array(116, '36', 5, null, 0.11, 0.11, 0.05, 0.05, 'product'), array(null, '36', 5, null, 0.55, 0.55, 0.25, 0.25, 'shipping'), array(116, '37', 3.33, null, 0.03, 0.03, 0.03, 0.03, 'product'), array(null, '37', 3.33, null, 0.17, 0.17, 0.17, 0.17, 'shipping')), 116)
Expectation failed for method name is "setData" when invoked 1 time(s)
Parameter 0 for invocation Magento\Framework\Model\AbstractExtensibleModel::setData(Array (...), null) does not match expected value.
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
     'amount' => 0.66
     'base_amount' => 0.66
     'process' => 0
-    'base_real_amount' => 0.36
+    'base_real_amount' => 0.36000000000000004
 )
vendor/magento/module-tax/Model/Plugin/OrderSave.php:166
vendor/magento/module-tax/Model/Plugin/OrderSave.php:48
vendor/magento/module-tax/Test/Unit/Model/Plugin/OrderSaveTest.php:228
12) Magento\Tax\Test\Unit\Model\Plugin\OrderSaveTest::testAfterSave with data set "associated_item_with_empty_order_quote_item" (array(array(0.66, 0.66, 11, 'ILUS', array(array(array(6, 'IL', 'IL'), array(5, 'US', 'US')))), array(0.2, 0.2, 3.33, 'CityTax', array(array(array(3, 'CityTax', 'CityTax'))))), array(array(null, 'product', 151, array(array(0.11, 0.11, 11, 'ILUS', array(array(array(6, 'IL', 'IL'), array(5, 'US', 'US')))), array(0.03, 0.03, 3.33, 'CityTax', array(array(array(3, 'CityTax', 'CityTax')))))), array(null, 'shipping', null, array(array(0.55, 0.55, 11, 'ILUS', array(array(array(6, 'IL', 'IL'), array(5, 'US', 'US')))), array(0.17, 0.17, 3.33, 'CityTax', array(array(array(3, 'CityTax', 'CityTax'))))))), array(array(123, 'IL', 'IL', 0, 6, 0, 0, 0.66, 0.66, 0, 0.36), array(123, 'US', 'US', 0, 5, 0, 0, 0.66, 0.66, 0, 0.3), array(123, 'CityTax', 'CityTax', 0, 3, 0, 0, 0.2, 0.2, 0, 0.18018018018018)), array(array(null, '35', 6, null, 0.11, 0.11, 0.06, 0.06, 'product'), array(null, '35', 6, null, 0.55, 0.55, 0.3, 0.3, 'shipping'), array(null, '36', 5, null, 0.11, 0.11, 0.05, 0.05, 'product'), array(null, '36', 5, null, 0.55, 0.55, 0.25, 0.25, 'shipping'), array(null, '37', 3.33, null, 0.03, 0.03, 0.03, 0.03, 'product'), array(null, '37', 3.33, null, 0.17, 0.17, 0.17, 0.17, 'shipping')), null)
Expectation failed for method name is "setData" when invoked 1 time(s)
Parameter 0 for invocation Magento\Framework\Model\AbstractExtensibleModel::setData(Array (...), null) does not match expected value.
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
     'amount' => 0.66
     'base_amount' => 0.66
     'process' => 0
-    'base_real_amount' => 0.36
+    'base_real_amount' => 0.36000000000000004
 )
vendor/magento/module-tax/Model/Plugin/OrderSave.php:166
vendor/magento/module-tax/Model/Plugin/OrderSave.php:48
vendor/magento/module-tax/Test/Unit/Model/Plugin/OrderSaveTest.php:228
13) Magento\Tax\Test\Unit\Model\Sales\Order\TaxManagementTest::testGetOrderTaxDetails with data set "canadian_weee" (array(array(null, 'weee', 69, 'GST', 'GST', '5', '2.10', '4.10'), array(null, 'weee', 69, 'GST', 'GST', '5', '1.15', '3.10')), array('GST', 'GST', '5', '3.25', '7.20'))
Failed asserting that 7.199999999999999 matches expected '7.20'.
vendor/magento/module-tax/Test/Unit/Model/Sales/Order/TaxManagementTest.php:138
14) Magento\Tax\Test\Unit\Pricing\AdjustmentTest::testExtractAdjustment with data set #2 (true, 10.1, 20.3, -10.2)
Failed asserting that -10.200000000000001 matches expected -10.2.
vendor/magento/module-tax/Test/Unit/Pricing/AdjustmentTest.php:120
FAILURES!

Additional information

See:

I'm guessing what should be done here is using assertEqualsWithDelta() instead.

Release note

No response

Triage and priority

[ ] Severity: S0 - Affects critical data or functionality and leaves users without workaround.
[ ] Severity: S1 - Affects critical data or functionality and forces users to employ a workaround.
[X] Severity: S2 - Affects non-critical data or functionality and forces users to employ a workaround.
[ ] Severity: S3 - Affects non-critical data or functionality and does not force users to employ a workaround.
[ ] Severity: S4 - Affects aesthetics, professional look and feel, “quality” or “usability”.

m2-assistant[bot] commented 1 year ago

Hi @francisco-serfe. Thank you for your report. To speed up processing of this issue, make sure that you provided the following information:

Summary of the issue
Information on your environment
Steps to reproduce
Expected and actual results

Make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, Add a comment to the issue:

@magento give me 2.4-develop instance - upcoming 2.4.x release

For more details, review the Magento Contributor Assistant documentation.

Add a comment to assign the issue: @magento I am working on this

To learn more about issue processing workflow, refer to the Code Contributions.

Join Magento Community Engineering Slack and ask your questions in #github channel.

:warning: According to the Magento Contribution requirements, all issues must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

:clock10: You can find the schedule on the Magento Community Calendar page.

:telephone_receiver: The triage of issues happens in the queue order. If you want to speed up the delivery of your contribution, join the Community Contributions Triage session to discuss the appropriate ticket.

:pencil2: Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel

m2-assistant[bot] commented 1 year ago

Hi @engcom-November. Thank you for working on this issue. In order to make sure that issue has enough information and ready for development, please read and check the following instruction: :point_down:

[ ] 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).
Details
If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.
[ ] 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.
[ ] 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.
[ ] 4. Verify that the issue is reproducible on 2.4-develop branch
Details
- Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
- If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
- If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!
[ ] 5. Add label Issue: Confirmed once verification is complete.
[ ] 6. Make sure that automatic system confirms that report has been added to the backlog.

engcom-November commented 1 year ago

Hi @francisco-serfe , Thank you for reporting and collaboration. "sebastian/comparator" is a 3rd party package / library that doesn't come along with latest / default Magento installation. Code of this package is not part of https://github.com/magento/magento2 git repository. We are closing this issue as community is not able to provide fix for it in this repository. All questions, issue reports and fix for them should be addressed to the corresponding extension owners(support) or the Magento Market place page Thank you.

hostep commented 1 year ago

@engcom-November, what do you mean it doesn't come with a Magento installation? Yes it does, see here...

@francisco-serfe: does it still trigger even if you update phpunit to the latest compatible version?

ghost commented 1 year ago

I didn't pursue the issue since they'd eventually run into it anyways when they updated PHPUnit --with-dependencies.

This repo is locked at 9.5.20 currently, with a ~9.5.20 constraint.
On phpunit 9.5.24, "sebastian/comparator": "^4.0.5",
On phpunit 9.5.25, "sebastian/comparator": "^4.0.8",

I just ended up fixing our phpunit version to 9.5.24 & sebastian/comparator to 4.0.7 and don't plan on updating it until these tests are updated.

hostep commented 1 year ago

Well, reason I'm asking is because I think this was fixed in phpunit 9.5.25, when you can confirm, we don't need to reopen this issue 🙂

ghost commented 1 year ago

Hmm, nope, no dice.

phpunit/phpunit: 9.5.25
sebastian/comparator: 4.0.8

...

Magento\SalesRule\Test\Unit\Model\Rule\Action\Discount\ToPercentTest::testCalculate with data set #0 (3, array(30, 5), array(10, 50, 55), array(50, 45, 60, 55), 100, array(98, 59.5, 119, 80.5))
Expectation failed for method name is "setBaseAmount" when invoked 1 time(s)
Parameter 0 for invocation Magento\SalesRule\Model\Rule\Action\Discount\Data::setBaseAmount(59.49999999999999) does not match expected value.
Failed asserting that 59.49999999999999 matches expected 59.5.

vendor/magento/module-sales-rule/Model/Rule/Action/Discount/ByPercent.php:58
vendor/magento/module-sales-rule/Model/Rule/Action/Discount/ToPercent.php:19
vendor/magento/module-sales-rule/Test/Unit/Model/Rule/Action/Discount/ToPercentTest.php:204

...

hostep commented 1 year ago

Okay, thanks for confirming!

@engcom-November, @sdzhepa: I believe this issue needs to be reopened...

ghost commented 1 year ago

Also should mention I've experienced this on CE 2.4.4-p1 as well.

engcom-Hotel commented 1 year ago

Hello,

We are reopening this issue to process it further.

Thanks

engcom-November commented 1 year ago

Verified the behavior on Magento 2.4-develop local instance and unit tests are getting failed with below steps performed: On running unit tests of mentioned modules from CLI, below errors are observed. Hence confirming the issue. ./vendor/bin/phpunit -c dev/tests/unit/phpunit.xml.dist app/code/Magento/SalesRule/Test/Unit ./vendor/bin/phpunit -c dev/tests/unit/phpunit.xml.dist app/code/Magento/Tax/Test/Unit

Note: There is no issue with MFTF test cases. Mentioned groups (sales, salesRule, tax) running fine in local instance. On running the command: vendor/bin/mftf run:group sales salesRule tax -r , MFTF tests are running fine.

github-jira-sync-bot commented 1 year ago

:white_check_mark: Jira issue https://jira.corp.adobe.com/browse/AC-7097 is successfully created for this GitHub issue.

m2-assistant[bot] commented 1 year ago

:white_check_mark: Confirmed by @engcom-November. Thank you for verifying the issue.
Issue Available: @engcom-November, You will be automatically unassigned. Contributors/Maintainers can claim this issue to continue. To reclaim and continue work, reassign the ticket to yourself.

rannmann commented 1 year ago

I don't know Magento or what your test are actually doing, but I ran into this same issue after upgrading the library in my own Laravel repo, and then found this thread as the top search result. Here's what I found, in case it helps anyone else.

Old tests were passing but weren't actually correct before 4.0.8. You can see the behavior by writing this test:

    public function testFloat()
    {
        // Passes with 4.0.6, fails with 4.0.8.
        $this->assertEquals(0.21399575471878052, 0.21399575471878002);
    }

Previously it was dropping off the last 3 values of precision, ending at 878. Where the bug in our code seems to be is after we save this value to the database. If we set a model's property to the first value, it appears fine, but if we refresh the model from MySQL, that property ends up as the second value.

Our database column is defined as double(21, 20), and even so, it only seems to be accurate to 14 decimals.

Really, none of us should be comparing floats directly anyway (that's why bcmath and assertEqualsWithDelta() exist), but we could get away with it due to a bug in versions before 4.0.8.

melnickforever commented 1 year ago

Do we have any workable for now?

hostep commented 1 year ago

It looks like some commits got added to 2.4-develop branch yesterday that will probably fix this, see: https://github.com/search?q=repo%3Amagento%2Fmagento2+float+comparison+precision&type=commits Unfortunately this wasn't done in a single merge commit, those got mixed up with a bunch of other things in https://github.com/magento/magento2/commit/ab0c0b54309d61d5bc207c73efad214e35516189

So I think this ticket can be closed? Maybe somebody first wants to confirm?

magento / magento2