isaac-sim / IsaacLab

Unified framework for robot learning built on NVIDIA Isaac Sim
https://isaac-sim.github.io/IsaacLab
Other
1.86k stars 710 forks source link

[Bug Report] omni.isaac.lab/test/deps/test_scipy.py test fails #733

Open BurakDmb opened 1 month ago

BurakDmb commented 1 month ago

Describe the bug

Hi, I am trying to run defined unit tests, and I think there is a problem with the deprecated interp2d method in Scipy (which was removed in 1.14.0) that is failing the test_scipy.py. Can you provide which version of Scipy is safe to use? Thanks.

Steps to reproduce

I am using a conda environment. You can use this command to reproduce the test failure. python isaaclab/source/extensions/omni.isaac.lab/test/deps/test_scipy.py

Additional context

python isaaclab/source/extensions/omni.isaac.lab/test/deps/test_scipy.py command output when scipy is 1.14.0.

$ python isaaclab/source/extensions/omni.isaac.lab/test/deps/test_scipy.py 

test_interpolation (__main__.TestScipyOperations)
Test scipy interpolation 2D method. ... ERROR

======================================================================
ERROR: test_interpolation (__main__.TestScipyOperations)
Test scipy interpolation 2D method.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/deps/test_scipy.py", line 55, in test_interpolation
    func_interp2d = interpolate.interp2d(y, x, height_field_downsampled, kind="cubic")
  File "/home/burak.demirbilek/miniconda3/envs/rl_framework/lib/python3.10/site-packages/scipy/interpolate/_interpolate.py", line 129, in __init__
    raise NotImplementedError(err_mesg)
NotImplementedError: `interp2d` has been removed in SciPy 1.14.0.

For legacy code, nearly bug-for-bug compatible replacements are
`RectBivariateSpline` on regular grids, and `bisplrep`/`bisplev` for
scattered 2D data.

In new code, for regular grids use `RegularGridInterpolator` instead.
For scattered data, prefer `LinearNDInterpolator` or
`CloughTocher2DInterpolator`.

For more details see
https://scipy.github.io/devdocs/tutorial/interpolate/interp_transition_guide.html

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (errors=1)

After downgrading scipy to 1.13.1, this unit test is still failing.

python isaaclab/source/extensions/omni.isaac.lab/test/deps/test_scipy.py 
test_interpolation (__main__.TestScipyOperations)
Test scipy interpolation 2D method. ... /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/deps/test_scipy.py:55: DeprecationWarning: `interp2d` is deprecated in SciPy 1.10 and will be removed in SciPy 1.14.0.

For legacy code, nearly bug-for-bug compatible replacements are
`RectBivariateSpline` on regular grids, and `bisplrep`/`bisplev` for
scattered 2D data.

In new code, for regular grids use `RegularGridInterpolator` instead.
For scattered data, prefer `LinearNDInterpolator` or
`CloughTocher2DInterpolator`.

For more details see
`https://scipy.github.io/devdocs/notebooks/interp_transition_guide.html`

  func_interp2d = interpolate.interp2d(y, x, height_field_downsampled, kind="cubic")
/home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/deps/test_scipy.py:56: DeprecationWarning: `interp2d` is deprecated in SciPy 1.10 and will be removed in SciPy 1.14.0.

For legacy code, nearly bug-for-bug compatible replacements are
`RectBivariateSpline` on regular grids, and `bisplrep`/`bisplev` for
scattered 2D data.

In new code, for regular grids use `RegularGridInterpolator` instead.
For scattered data, prefer `LinearNDInterpolator` or
`CloughTocher2DInterpolator`.

For more details see
`https://scipy.github.io/devdocs/notebooks/interp_transition_guide.html`

  z_upsampled_interp2d = func_interp2d(y_upsampled, x_upsampled)
FAIL

======================================================================
FAIL: test_interpolation (__main__.TestScipyOperations)
Test scipy interpolation 2D method.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/deps/test_scipy.py", line 70, in test_interpolation
    np.testing.assert_allclose(z_upsampled_RectBivariant, z_upsampled_RegularGridInterpolator, atol=1e-14)
  File "/home/burak.demirbilek/miniconda3/envs/rl_framework/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 1504, in assert_allclose
    assert_array_compare(compare, actual, desired, err_msg=str(err_msg),
  File "/home/burak.demirbilek/miniconda3/envs/rl_framework/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/home/burak.demirbilek/miniconda3/envs/rl_framework/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 797, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Not equal to tolerance rtol=1e-07, atol=1e-14

Mismatched elements: 11779 / 12000 (98.2%)
Max absolute difference: 0.0007767
Max relative difference: 0.42222691
 x: array([[ 4.      , 13.863408, 19.930661, ...,  4.03692 ,  7.169014,
        12.      ],
       [-8.654133, 19.978647, 19.365613, ...,  8.519592, 16.029264,...
 y: array([[ 4.000001, 13.863321, 19.930564, ...,  4.036756,  7.168875,
        12.000002],
       [-8.65405 , 19.977871, 19.365   , ...,  8.519612, 16.029187,...

----------------------------------------------------------------------
Ran 1 test in 0.027s

FAILED (failures=1)

Lastly, this is the overall results for all unit tests, only the test_scipy.py test fails. (scipy==1.13.1)

===================
Test Result Summary
===================
Total: 52
Passing: 44
Failing: 1
Skipped: 7
Timing Out: 0
Passing Percentage: 98.08%
Total Time Elapsed: 0.0h8.0m37.04s

=======================
Per Test Result Summary
=======================
+-----------------------------------------------------------------------------------------------------------------------------------------------+---------+----------+
| Test Path                                                                                                                                     |  Result | Time (s) |
+-----------------------------------------------------------------------------------------------------------------------------------------------+---------+----------+
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/assets/test_articulation.py                      |  PASSED |    19.19 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/assets/test_rigid_object.py                      |  PASSED |    21.94 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/deps/test_scipy.py                               |  FAILED |     0.32 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/deps/test_torch.py                               |  PASSED |     1.14 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/envs/test_env_rendering_logic.py                 |  PASSED |    77.01 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/envs/test_manager_based_env.py                   |  PASSED |     5.98 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/envs/test_null_command_term.py                   |  PASSED |     4.02 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/managers/test_observation_manager.py             |  PASSED |     3.91 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/managers/test_reward_manager.py                  |  PASSED |     3.78 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/markers/test_visualization_markers.py            |  PASSED |     8.06 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/performance/test_kit_startup_performance.py      |  PASSED |     3.14 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/performance/test_robot_load_performance.py       |  PASSED |    39.76 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/scene/test_interactive_scene.py                  |  PASSED |     6.02 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/sensors/test_camera.py                           |  PASSED |    27.17 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/sensors/test_contact_sensor.py                   |  PASSED |    13.41 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/sensors/test_frame_transformer.py                |  PASSED |     9.24 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/sensors/test_ray_caster_camera.py                |  PASSED |    20.82 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/sensors/test_tiled_camera.py                     |  PASSED |    16.73 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/sim/test_build_simulation_context_headless.py    |  PASSED |     6.84 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/sim/test_mesh_converter.py                       |  PASSED |     7.58 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/sim/test_schemas.py                              |  PASSED |     6.55 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/sim/test_simulation_context.py                   |  PASSED |     5.39 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/sim/test_spawn_from_files.py                     |  PASSED |     5.80 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/sim/test_spawn_lights.py                         |  PASSED |     4.28 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/sim/test_spawn_materials.py                      |  PASSED |     4.96 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/sim/test_spawn_meshes.py                         |  PASSED |     7.40 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/sim/test_spawn_sensors.py                        |  PASSED |     3.68 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/sim/test_spawn_shapes.py                         |  PASSED |     6.02 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/sim/test_urdf_converter.py                       |  PASSED |     5.70 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/sim/test_utils.py                                |  PASSED |    13.98 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/terrains/test_terrain_importer.py                |  PASSED |    52.21 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/utils/test_assets.py                             |  PASSED |     3.92 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/utils/test_circular_buffer.py                    |  PASSED |     3.34 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/utils/test_configclass.py                        |  PASSED |     3.32 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/utils/test_delay_buffer.py                       |  PASSED |     3.35 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/utils/test_dict.py                               |  PASSED |     3.26 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/utils/test_math.py                               |  PASSED |     3.08 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/utils/test_string.py                             |  PASSED |     3.24 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/utils/test_timer.py                              |  PASSED |     5.12 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab_assets/test/test_valid_configs.py                     |  PASSED |    34.61 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab_tasks/test/test_environments.py                       |  PASSED |     8.39 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab_tasks/test/wrappers/test_rl_games_wrapper.py          |  PASSED |     8.36 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab_tasks/test/wrappers/test_rsl_rl_wrapper.py            |  PASSED |     8.33 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab_tasks/test/wrappers/test_sb3_wrapper.py               |  PASSED |     8.39 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab_tasks/test/wrappers/test_skrl_wrapper.py              |  PASSED |     8.30 |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/app/test_argparser_launch.py                     | SKIPPED |      N/A |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/app/test_env_var_launch.py                       | SKIPPED |      N/A |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/app/test_kwarg_launch.py                         | SKIPPED |      N/A |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/controllers/test_differential_ik.py              | SKIPPED |      N/A |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab/test/sim/test_build_simulation_context_nonheadless.py | SKIPPED |      N/A |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab_tasks/test/test_data_collector.py                     | SKIPPED |      N/A |
| /home/burak.demirbilek/Documents/rl-framework/isaaclab/source/extensions/omni.isaac.lab_tasks/test/test_record_video.py                       | SKIPPED |      N/A |
+-----------------------------------------------------------------------------------------------------------------------------------------------+---------+----------+
Mayankm96 commented 1 month ago

I have scipy 1.10.1 installed.