nv-legate / cupynumeric

An Aspiring Drop-In Replacement for NumPy at Scale
https://docs.nvidia.com/cupynumeric
Apache License 2.0
623 stars 71 forks source link

Numpy RuntimeWarning not raised by cunumeric #1115

Open CharlelieLrt opened 9 months ago

CharlelieLrt commented 9 months ago

Expected behavior

When running a numpy code, I am getting some warnings such as:

RuntimeWarning: overflow encountered in multiply
  rhoE - 0.5 * num.sum(rhou * rhou, axis=0) / rho

or:

RuntimeWarning: invalid value encountered in add
  return 0.5 * (x[(..., *idx_m)] + x[(..., *idx_p)])

Observed behavior

When running the same code with legate, these warnings are not raised (Note that the warnings are expected).

As discussed with @manopapad this is probably due to the backend libraries such as CuBLAS that do not have a "safe mode", or at least are running without it to avoid some significant overhead. @manopapad suggested that to avoid overhead it is probably better for now to implement checks for Nans or Infs at the user level rather than in cunumeric.