Fix numpy 2.0 e2e test fail with lower requirements

ravi-kumar-pilla commented 2 weeks ago

Description

Resolves https://github.com/kedro-org/kedro-viz/actions/runs/9548598519/job/26316143362?pr=1935

 Full exception: `np.float_` was removed in the NumPy 2.0 release. Use `np.float64` instead.

Development notes

Pin numpy in package/features/steps/lower_requirements.txt at 1.26.4
Pin numpy in package/features/steps/cli_steps.py at 1.26.4
Doc link fix at docs/source/publish_and_share_kedro_viz_on_azure.md

QA notes

All tests should pass

Checklist

[x] Read the contributing guidelines
[ ] Opened this PR as a 'Draft Pull Request' if it is work-in-progress
[ ] Updated the documentation to reflect the code changes
[ ] Added new entries to the RELEASE.md file
[ ] Added tests to cover my changes

ravi-kumar-pilla commented 2 weeks ago

Hi Team,

I am trying to fix e2e test failures on kedro-viz. For both the cases below, pinning numpy < 2 resolved the issue. But I am not certain to go ahead with this fix as the actual fix might be in the starters. Could someone please confirm if the starters are working fine with latest release of numpy.

https://github.com/kedro-org/kedro-viz/actions/runs/9548599222/job/26316144191?pr=1935 https://github.com/kedro-org/kedro-viz/actions/runs/9571094566/job/26387337026

NOTE: Kedro-Viz itself does not use numpy but seems like pandas uses numpy. Compatibility matrix mentioned here might help.

cc: @ankatiyar @SajidAlamQB

Thank you

ankatiyar commented 1 week ago

It makes sense to cap numpy in lower_requirements.txt for this failure - https://github.com/kedro-org/kedro-viz/actions/runs/9548599222/job/26316144191?pr=1935 Since pandas==1.15 does not have an upper bound on numpy

But why does it need to be pinned in package/requirements.txt? What commit is this failure related to - https://github.com/kedro-org/kedro-viz/actions/runs/9571094566/job/26387337026? It seems like a missing dependency from kedro-datasets 👀 @ravi-kumar-pilla

ravi-kumar-pilla commented 1 week ago

But why does it need to be pinned in package/requirements.txt? What commit is this failure related to - https://github.com/kedro-org/kedro-viz/actions/runs/9571094566/job/26387337026? It seems like a missing dependency from kedro-datasets 👀 @ravi-kumar-pilla

Hi @ankatiyar , Thank you for looking over the PR. This is not related to any new commit. Our builds were failing all of a sudden without any code change (even the old builds if re-run are failing). The e2e test that fails now is for the below scenario -

Scenario: Execute viz with the earliest Kedro version that it supports 
    Given I have installed kedro version "0.18.3"
    And I have run a non-interactive kedro new with pandas-iris starter
    And I have installed the project's requirements
    When I execute the kedro viz run command
    Then kedro-viz should start successfully

It might be some issue with the pandas-iris starter requirements as Kedro-Viz works fine with the demo-project we have. I will change the starter for the test and try again. But I wanted to bring this to your attention if we need to push changes to the starters due to the numpy release.

Thank you

ankatiyar commented 1 week ago

Okay I created a project with 0.18.3 and tried it and I see what is happening - For this particular test, when a Kedro project's requirements are installed, it does pandas==1.15.x which installs numpy==2.0.0 because it's not capped. It's the same problem but I don't think we should pin the numpy version in package/requirements.txt for all of kedro-viz but only for this specific e2e test.

I don't think anything of Kedro side is breaking since we run our tests only with the latest versions of pandas which works fine with the latest numpy.

kedro-org / kedro-viz