search

CycloneDX / cyclonedx-python-lib

Python implementation of OWASP CycloneDX
https://cyclonedx.org/
Apache License 2.0
69 stars 40 forks source link

Hash mismatch in TestModelComponentEvidence.test_same_2 #202

Closed bollwyvl closed 2 years ago

bollwyvl commented 2 years ago

Thanks for cyclonedx!

Downstream on conda-forge i'm seeing the following when running the test suite against the as-installed 2.1.1: is there cause for concern? I'll likely just skip it and move on, but hash tricks seem like rather fundamental thing that other features might be relying on for correctness...

=================================== FAILURES ===================================
____________________ TestModelComponentEvidence.test_same_2 ____________________

self = 

    def test_same_2(self) -> None:
        ce_1 = ComponentEvidence(copyright_=[Copyright(text='Commercial'), Copyright(text='Commercial 2')])
        ce_2 = ComponentEvidence(copyright_=[Copyright(text='Commercial 2'), Copyright(text='Commercial')])
>       self.assertEqual(hash(ce_1), hash(ce_2))
E       AssertionError: 6591512646010453410 != -5735625735841950921

test_model_component.py:268: AssertionError
=============================== warnings summary ===============================
test_bom.py: 1 warning
test_model_component.py: 22 warnings
test_output_json.py: 32 warnings
test_output_xml.py: 52 warnings
  $PREFIX/lib/python3.10/site-packages/cyclonedx/model/component.py:739: DeprecationWarning: `license_str` is deprecated and has been replaced with `licenses` to align with the CycloneDX standard
    warnings.warn(

test_bom.py::TestBom::test_metadata_component
  $SRC_DIR/src/tests/test_bom.py:49: DeprecationWarning: Please use assertEqual instead.
    self.assertEquals(metadata.component, hextech)

test_component.py::TestComponent::test_from_file_with_path_for_bom
  $PREFIX/lib/python3.10/site-packages/cyclonedx/model/component.py:731: DeprecationWarning: `namespace` is deprecated and has been replaced with `group` to align with the CycloneDX standard
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED test_model_component.py::TestModelComponentEvidence::test_same_2 - Ass...
============ 1 failed, 197 passed, 3 skipped, 109 warnings in 4.93s ============
versions 
    _libgcc_mutex:        0.1-conda_forge           conda-forge
    _openmp_mutex:        4.5-1_gnu                 conda-forge
    attrs:                21.4.0-pyhd8ed1ab_0       conda-forge
    cyclonedx-python-lib: 2.1.1-pyhd8ed1ab_0        local      
    distlib:              0.3.4-pyhd8ed1ab_0        conda-forge
    filelock:             3.6.0-pyhd8ed1ab_0        conda-forge
    icu:                  70.1-h27087fc_0           conda-forge
    importlib-metadata:   4.11.3-py310hff52083_1    conda-forge
    importlib_resources:  5.6.0-pyhd8ed1ab_0        conda-forge
    iniconfig:            1.1.1-pyh9f0ad1d_0        conda-forge
    jsonschema:           4.4.0-pyhd8ed1ab_0        conda-forge
    ld_impl_linux-64:     2.36.1-hea4e1c9_2         conda-forge
    libffi:               3.4.2-h7f98852_5          conda-forge
    libgcc-ng:            11.2.0-h1d223b6_14        conda-forge
    libgomp:              11.2.0-h1d223b6_14        conda-forge
    libiconv:             1.16-h516909a_0           conda-forge
    libnsl:               2.0.0-h7f98852_0          conda-forge
    libstdcxx-ng:         11.2.0-he4da1e4_14        conda-forge
    libuuid:              2.32.1-h7f98852_1000      conda-forge
    libxml2:              2.9.12-h22db469_2         conda-forge
    libxslt:              1.1.33-h8affb1d_4         conda-forge
    libzlib:              1.2.11-h166bdaf_1014      conda-forge
    lxml:                 4.8.0-py310h5764c6d_1     conda-forge
    ncurses:              6.3-h9c3ff4c_0            conda-forge
    openssl:              3.0.2-h166bdaf_1          conda-forge
    packageurl-python:    0.9.9-pyhd8ed1ab_0        conda-forge
    packaging:            21.3-pyhd8ed1ab_0         conda-forge
    pip:                  22.0.4-pyhd8ed1ab_0       conda-forge
    platformdirs:         2.5.1-pyhd8ed1ab_0        conda-forge
    pluggy:               1.0.0-py310hff52083_3     conda-forge
    py:                   1.11.0-pyh6c4a22f_0       conda-forge
    pyparsing:            3.0.7-pyhd8ed1ab_0        conda-forge
    pyrsistent:           0.18.1-py310h5764c6d_1    conda-forge
    pytest:               7.1.1-py310hff52083_1     conda-forge
    pytest-cov:           3.0.0-pyhd8ed1ab_0        conda-forge
    python:               3.10.4-h2660328_0_cpython conda-forge
    python_abi:           3.10-2_cp310              conda-forge
    readline:             8.1-h46c0cb4_0            conda-forge
    setuptools:           62.0.0-py310hff52083_0    conda-forge
    six:                  1.16.0-pyh6c4a22f_0       conda-forge
    sqlite:               3.37.1-h4ff8645_0         conda-forge
    tk:                   8.6.12-h27826a3_0         conda-forge
    toml:                 0.10.2-pyhd8ed1ab_0       conda-forge
    tomli:                2.0.1-pyhd8ed1ab_0        conda-forge
    tox:                  3.24.5-pyhd8ed1ab_1       conda-forge
    types-setuptools:     57.4.12-pyhd8ed1ab_0      conda-forge
    types-toml:           0.10.4-pyhd8ed1ab_0       conda-forge
    typing-extensions:    3.10.0.2-hd8ed1ab_0       conda-forge
    typing_extensions:    3.10.0.2-pyha770c72_0     conda-forge
    tzdata:               2022a-h191b570_0          conda-forge
    virtualenv:           20.14.0-py310hff52083_1   conda-forge
    wheel:                0.37.1-pyhd8ed1ab_0       conda-forge
    xmldiff:              2.4-pyhd8ed1ab_0          conda-forge
    xz:                   5.2.5-h516909a_1          conda-forge
    zipp:                 3.8.0-pyhd8ed1ab_0        conda-forge
    zlib:                 1.2.11-h166bdaf_1014      conda-forge
jkowalleck commented 2 years ago

that is strange. i would expect these issues in older python implementations, there hashes of tuples and sets were inconsistent.

this test ran on platform linux -- Python 3.10.4, pytest-7.1.1, pluggy-1.0.0 -- $PREFIX/bin/python see https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=486922&view=logs&j=656edd35-690f-5c53-9ba3-09c10d0bea97&t=e5c8ab1d-8ff9-5cae-b332-e15ae582ed2d&l=290

jkowalleck commented 2 years ago

took the hint, and triggered a test run in our own env, on latest master branch - https://github.com/CycloneDX/cyclonedx-python-lib/actions/runs/2104143406 on tag v2.1.1 - https://github.com/CycloneDX/cyclonedx-python-lib/actions/runs/2104149187

jkowalleck commented 2 years ago

Hello @bollwyvl ,

thanks for the report & maintaining it for conda-forge.. Our own tests on v2.1.1 (running in venv via tox) on ubuntu with Python 3.10.4 are passing - see https://github.com/CycloneDX/cyclonedx-python-lib/runs/5855876653?check_suite_focus=true#step:10:16

thing that might affect the tests: we are setting env var "PYTHONHASHSEED=0" in our test setup. see https://github.com/CycloneDX/cyclonedx-python-lib/blob/main/tox.ini#L31 could you do the same and see if the tests pass?

see also: https://docs.python.org/3/using/cmdline.html#envvar-PYTHONHASHSEED

bollwyvl commented 2 years ago

Well: setting the PYTHONHASHSEED to 0 does work, but if it relies on that for regular runtime behavior, that deems dubious. Closing either way, thanks for looking into it.

jkowalleck commented 2 years ago

@madpah FYI. need to check which model actually depends on item order for hashing. afaik none should do so. what am i missing?