search

jeffgortmaker / pyblp

BLP Demand Estimation with Python
https://pyblp.readthedocs.io
MIT License
240 stars 83 forks source link

PyBLP in R with reticulate - cannot access simulation_results$product_data due to Error in py_ref_to_r(x) #123

Closed ianhsiehbcn closed 1 year ago

ianhsiehbcn commented 1 year ago

Hi Jeff!

We are trying to run a counterfactual simulation using the PyBLP package in R with reticulate. We use the simulation$replace_endogenous() command to compute the new equilibrium prices and shares, but when we try to access the new data, stored in simulation_results$product_data, we are not able to do it because R cannot open the object, which is a numpy.recarray object. The error message we get is the following one:

Error in py_ref_to_r(x) :
  Conversion from numpy array type 20 is not supported

We have tried several solutions suggested in the pyblp documentation, such as using the pyblp$data_to_dict() function, or the simulations results $to_dict() function, but none of them has worked so far. We would appreciate some help on how to access this object from R, and more generally, some guidance on how to convert recarray objects into some other numpy object which could be opened in R.

Thanks for building this amazing package!!

jeffgortmaker commented 1 year ago

I'm curious why using the data_to_dict function doesn't work? In theory it should convert the numpy.recarray into a dict mapping strings to simple numpy.ndarray objects that reticulate should be able to deal with. Mind trying to access 

pyblp$data_to_dict(simulation_results$product_data)

or even something like

pandas$DataFrame(pyblp$data_to_dict(simulation_results$product_data))

after activating pandas with reticulate and letting me know if either of these works?

ianhsiehbcn commented 1 year ago

Unfortunately, both functions led to the error message quoted above.

jeffgortmaker commented 1 year ago

Oh right, it should be the same solution as https://github.com/jeffgortmaker/pyblp/issues/106#issuecomment-1039597092: using convert = FALSE and py_to_r. Let me know if that works.

jeffgortmaker commented 1 year ago

I think the above will probably fix your issue, so I'm closing this issue for now, but please do keep commenting/re-open the issue if you're still having problems!