PolicyEngine / policyengine-us

The PolicyEngine US Python package contains a rules engine of the US tax-benefit system, and microdata generation for microsimulation analysis.
https://policyengine.org/us
GNU Affero General Public License v3.0
95 stars 167 forks source link

`Microsimulation.calculate` throws `TypeError: ufunc 'isnan' not supported for the input types` in Colab #2802

Open MaxGhenis opened 11 months ago

MaxGhenis commented 11 months ago

I thought that a bool parameter being a list #2801 was the cause, but it didn't fix it (notebook).

This means any automatically generated microsimulation Python code from the app fails in Colab (works locally).

baseline.calculate("taxable_income", 2023)

throws

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
[<ipython-input-7-6d3393d26ada>](https://localhost:8080/#) in <cell line: 1>()
----> 1 baseline.calculate("taxable_income", 2023)

82 frames
[/usr/local/lib/python3.10/dist-packages/policyengine_core/parameters/helpers.py](https://localhost:8080/#) in contains_nan(vector)
     13         return any([contains_nan(vector[name]) for name in vector.dtype.names])
     14     else:
---> 15         return numpy.isnan(vector).any()
     16 
     17 

TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''
ch1nmay7898 commented 10 months ago

I was just able to make it work on notebook.

MaxGhenis commented 10 months ago

Interesting, thanks @ch1nmay7898! Could you check if we could do anything to avoid the error without restarting the runtime?

ch1nmay7898 commented 10 months ago

Found this helpful: (link)

My understanding: Looks like it is necessary to restart the runtime on Colab for it to register the newly installed versions of dependencies. The warning says that argparse was previously imported in the runtime and to register the argparse-1.4.0 version (which gets installed with policyengine_us), runtime needs to be restarted.

There is a way to restart the runtime programmatically from command line, but that doesn't seem like the best way.

MaxGhenis commented 10 months ago

Could we change our code to work with the argparse version colab uses?

feixie commented 9 months ago

Looks like this TypeError is related to the numpy version and not the argparse version. When I do a pip show numpy on a new Colab notebook I see that it comes with version 1.23.5. Looks like policyengine-core specifically only supports numpy<1.22 and >=1.21, so a runtime restart might be necessary to avoid other issues.

Judging by the error, it looks like the contains_nan handling in the policyengine_core helper might be expecting the vector to be in a format it's no longer in with numpy versions > 1.22. If updating the numpy version of policyengine-core is a possibility, then the next step should be to do that and run this logic to see why contains_nan encountering this error.

feixie commented 9 months ago

Looks like this issue has been fixed on the openfisca-core side: https://github.com/openfisca/openfisca-core/commit/a71d2217f6dc19d3786d24490e1324d2feb2c91f