search

ing-bank / popmon

Monitor the stability of a Pandas or Spark dataframe ⚙︎
https://popmon.readthedocs.io/
MIT License
497 stars 34 forks source link

Fix warnings in CI build pipeline #280

Open sbrugman opened 1 year ago

sbrugman commented 1 year ago

Fix warnings in CI build pipeline:

We can configure pytest to fail on warnings in pyproject.toml:

filterwarnings = ["error"]

This continues on https://github.com/ing-bank/popmon/pull/262

TanyaKansal commented 2 months ago

@sbrugman can i work on this?

sbrugman commented 2 months ago

Yes, more than welcome!

TanyaKansal commented 2 months ago

can you please assign to me? Also, can we mark it for hacktoberfest 2024? @sbrugman

TanyaKansal commented 2 months ago

can you please add me as assignee @sbrugman

TanyaKansal commented 2 months ago

thanks @sbrugman

TanyaKansal commented 2 months ago

hi @sbrugman could you plz let em know how I can replicate the above warnings on my local system. Also I tried running pytest command and got the following failures: ==================================================================== short test summary info ===================================================================== FAILED tests/popmon/analysis/profiling/test_apply_func.py::test_pull - AssertionError: FAILED tests/popmon/analysis/profiling/test_apply_func.py::test_apply_func_module - KeyError: 'a_mean' FAILED tests/popmon/analysis/profiling/test_apply_func.py::test_variance_comparer - KeyError: 'mae_pull' FAILED tests/popmon/analysis/profiling/test_apply_func.py::test_reference_pull_comparer - KeyError: 'mae_ref_pull' FAILED tests/popmon/analysis/profiling/test_apply_func.py::test_apply_func - AssertionError: FAILED tests/popmon/analysis/profiling/test_apply_func.py::test_apply_func_array - AssertionError: FAILED tests/popmon/analysis/profiling/test_profiles.py::test_fraction_oftrue - AttributeError: `np.stringwas removed in the NumPy 2.0 release. Usenp.bytes_instead. FAILED tests/popmon/analysis/test_hist_numpy.py::test_histogram - AttributeError: module 'pandas._testing' has no attribute 'makeMixedDataFrame' FAILED tests/popmon/analysis/test_hist_numpy.py::test_get_contentType - AttributeError: module 'pandas._testing' has no attribute 'makeMixedDataFrame' FAILED tests/popmon/analysis/test_hist_numpy.py::test_prepare_2dgrid - AttributeError: module 'pandas._testing' has no attribute 'makeMixedDataFrame' FAILED tests/popmon/analysis/test_hist_numpy.py::test_set_2dgrid - AttributeError: module 'pandas._testing' has no attribute 'makeMixedDataFrame' FAILED tests/popmon/analysis/test_hist_numpy.py::test_get_2dgrid - AttributeError: module 'pandas._testing' has no attribute 'makeMixedDataFrame' FAILED tests/popmon/analysis/test_hist_numpy.py::test_check_similar_hists - AttributeError: module 'pandas._testing' has no attribute 'makeMixedDataFrame' FAILED tests/popmon/analysis/test_hist_numpy.py::test_assert_similar_hists - AttributeError: module 'pandas._testing' has no attribute 'makeMixedDataFrame' FAILED tests/popmon/analysis/test_hist_numpy.py::test_datatype - AttributeError: module 'pandas._testing' has no attribute 'makeMixedDataFrame' FAILED tests/popmon/hist/test_histogram.py::test_histogrammar - AttributeError: module 'pandas._testing' has no attribute 'makeMixedDataFrame' FAILED tests/popmon/hist/test_histogram.py::test_histogram_attributes - AttributeError: module 'pandas._testing' has no attribute 'makeMixedDataFrame' FAILED tests/popmon/hist/test_histogram.py::test_sparse_bin_centers_x - AttributeError: module 'pandas._testing' has no attribute 'makeMixedDataFrame' FAILED tests/popmon/hist/test_histogram.py::test_split_hist_along_first_dimension - AttributeError: module 'pandas._testing' has no attribute 'makeMixedDataFrame' FAILED tests/popmon/hist/test_histogram.py::test_sum_entries - AttributeError: module 'pandas._testing' has no attribute 'makeMixedDataFrame' FAILED tests/popmon/hist/test_histogram.py::test_sum_over_x - AttributeError: module 'pandas._testing' has no attribute 'makeMixedDataFrame' FAILED tests/popmon/hist/test_histogram.py::test_project_on_x - AttributeError: module 'pandas._testing' has no attribute 'makeMixedDataFrame' FAILED tests/popmon/hist/test_histogram.py::test_project_split2dhist_on_axis - AttributeError: module 'pandas._testing' has no attribute 'makeMixedDataFrame' FAILED tests/popmon/hist/test_histogram.py::test_datatype - AttributeError: module 'pandas._testing' has no attribute 'makeMixedDataFrame' FAILED tests/popmon/notebooks/test_notebooks.py::test_notebook_basic - jupyter_client.kernelspec.NoSuchKernel: No such kernel named python3 FAILED tests/popmon/notebooks/test_notebooks.py::test_notebook_advanced - jupyter_client.kernelspec.NoSuchKernel: No such kernel named python3 FAILED tests/popmon/notebooks/test_notebooks.py::test_notebook_incremental_data - jupyter_client.kernelspec.NoSuchKernel: No such kernel named python3 FAILED tests/popmon/notebooks/test_notebooks.py::test_notebook_reports - jupyter_client.kernelspec.NoSuchKernel: No such kernel named python3 FAILED tests/popmon/pipeline/test_metrics.py::test_hists_stability_metrics - AttributeError:np.stringwas removed in the NumPy 2.0 release. Usenp.bytesinstead. FAILED tests/popmon/pipeline/test_metrics.py::test_df_stability_metrics - RuntimeError: unknown assigned datatype "<class 'numpy.datetime64'>" FAILED tests/popmon/pipeline/test_report.py::test_hists_stability_report - AttributeError:np.stringwas removed in the NumPy 2.0 release. Usenp.bytes` instead. FAILED tests/popmon/pipeline/test_report.py::test_df_stability_report - RuntimeError: unknown assigned datatype "<class 'numpy.datetime64'>" FAILED tests/popmon/pipeline/test_report.py::test_df_stability_report_self - RuntimeError: unknown assigned datatype "<class 'numpy.datetime64'>" FAILED tests/popmon/pipeline/test_report.py::test_df_stability_report_external - RuntimeError: unknown assigned datatype "<class 'numpy.datetime64'>" FAILED tests/popmon/pipeline/test_report.py::test_df_stability_report_rolling - RuntimeError: unknown assigned datatype "<class 'numpy.datetime64'>" FAILED tests/popmon/pipeline/test_report.py::test_df_stability_report_expanding - RuntimeError: unknown assigned datatype "<class 'numpy.datetime64'>" FAILED tests/popmon/stitching/test_histogram_stitching.py::test_histogram_stitching - RuntimeError: unknown assigned datatype "<class 'numpy.datetime64'>"

can you please suggest the steps to setup the repo on my mac os. is there any documentation available?

sbrugman commented 2 months ago

Could you retry with "numpy < 2"? See https://github.com/ing-bank/popmon/blob/master/pyproject.toml#L23

TanyaKansal commented 1 month ago

hi @sbrugman, i still see 4 of my test cases failing on my local. can you help. Below is the test summary. ==================================================================== short test summary info ===================================================================== FAILED popmon/notebooks/test_notebooks.py::test_notebook_basic - nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell: FAILED popmon/notebooks/test_notebooks.py::test_notebook_advanced - nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell: FAILED popmon/notebooks/test_notebooks.py::test_notebook_incremental_data - nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell: FAILED popmon/notebooks/test_notebooks.py::test_notebook_reports - nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:

following are the dependencies:

Package Version

annotated-types 0.7.0 anyio 4.6.0 appnope 0.1.4 argon2-cffi 23.1.0 argon2-cffi-bindings 21.2.0 arrow 1.3.0 asttokens 2.4.1 attrs 22.2.0 beautifulsoup4 4.12.3 bleach 6.1.0 build 1.2.2 CacheControl 0.14.0 certifi 2024.8.30 cffi 1.17.1 charset-normalizer 3.3.2 cleo 2.1.0 colorama 0.4.6 comm 0.2.2 contourpy 1.3.0 crashtest 0.4.1 cycler 0.12.1 debugpy 1.8.6 decorator 5.1.1 defusedxml 0.7.1 distlib 0.3.8 dulwich 0.21.7 executing 2.1.0 fastjsonschema 2.20.0 filelock 3.16.1 fonttools 4.54.1 fqdn 1.5.1 gitdb 4.0.11 GitPython 3.1.43 histogrammar 1.0.33 htmlmin 0.1.12 idna 3.10 iniconfig 2.0.0 installer 0.7.0 ipykernel 6.29.5 ipython 8.27.0 ipywidgets 8.1.5 isoduration 20.11.0 jaraco.classes 3.4.0 jedi 0.19.1 Jinja2 3.1.4 joblib 1.4.2 jsonpointer 3.0.0 jsonschema 4.23.0 jsonschema-specifications 2023.12.1 jupyter_client 8.6.3 jupyter_core 5.7.2 jupyter-events 0.10.0 jupyter_server 2.14.2 jupyter-server-mathjax 0.2.6 jupyter_server_terminals 0.5.3 jupyterlab_pygments 0.3.0 jupyterlab_widgets 3.0.13 keyring 24.3.1 kiwisolver 1.4.7 MarkupSafe 2.1.5 matplotlib 3.9.2 matplotlib-inline 0.1.7 mistune 3.0.2 more-itertools 10.5.0 msgpack 1.1.0 nbclient 0.5.13 nbconvert 7.16.4 nbdime 4.0.2 nbformat 5.10.4 nest-asyncio 1.6.0 numpy 1.26.4 overrides 7.7.0 packaging 24.1 pandas 1.5.3 pandocfilters 1.5.1 parso 0.8.4 pexpect 4.9.0 phik 0.12.4 pillow 10.4.0 pip 24.2 pkginfo 1.11.1 platformdirs 4.3.6 plotly 5.24.1 pluggy 1.5.0 poetry-core 1.9.0 poetry-plugin-export 1.8.0 popmon 1.4.6 prometheus_client 0.21.0 prompt_toolkit 3.0.48 psutil 6.0.0 ptyprocess 0.7.0 pure_eval 0.2.3 pycparser 2.22 pydantic 2.9.2 pydantic_core 2.23.4 pydantic-settings 2.5.2 Pygments 2.18.0 pyparsing 3.1.4 pyproject_hooks 1.2.0 pytest 8.3.3 pytest_notebook 0.10.0 python-dateutil 2.9.0.post0 python-dotenv 1.0.1 python-json-logger 2.0.7 pytz 2024.2 PyYAML 6.0.2 pyzmq 26.2.0 RapidFuzz 3.10.0 referencing 0.35.1 requests 2.32.3 requests-toolbelt 1.0.0 rfc3339-validator 0.1.4 rfc3986-validator 0.1.1 rpds-py 0.20.0 scipy 1.14.1 Send2Trash 1.8.3 setuptools 75.1.0 shellingham 1.5.4 six 1.16.0 smmap 5.0.1 sniffio 1.3.1 soupsieve 2.6 stack-data 0.6.3 tenacity 9.0.0 terminado 0.18.1 tinycss2 1.3.0 tomlkit 0.13.2 tornado 6.4.1 tqdm 4.66.5 traitlets 5.14.3 trove-classifiers 2024.9.12 types-python-dateutil 2.9.0.20240906 typing_extensions 4.12.2 tzdata 2024.2 uri-template 1.3.0 urllib3 2.2.3 virtualenv 20.26.6 wcwidth 0.2.13 webcolors 24.8.0 webencodings 0.5.1 websocket-client 1.8.0 widgetsnbextension 4.0.13 xattr 1.1.0