Closed nreigl closed 3 years ago
tlamadon
commented
3 years ago Thanks for posting this, we are going to take a look. This is a bit surprising since the circle-ci tests are passing. We have not run the test with python 3.9 though. Did you install with pip?
nreigl
commented
3 years ago Yes, I installed with pip.
adamoppenheimer
commented
3 years ago Hi Nicolas, thank you for pointing out these issues!
To give a short summary of what went wrong, I made mistakes in the documentation and forgot to include the data cleaning step, and the clustering step for the CRE estimator. I also forgot to include code to generate the 'm' column indicating whether the observation belongs to a mover or a stayer (this only applies in the case where you are estimating on long-form non-collapsed data).
I fixed these issues, uploaded a new version to Pip, and updated the documentation. I included the updated code you should run below. Please let us know if this works!
Also note that this code automatically collapses the data (which the previous code didn't). There is really no need to be working with the uncollapsed data as the results should be (almost) numerically identical with lots of computational benefits. The documentation does explain how to run the estimators on non-collapsed data, and feel free to ask if you want more details on running code for that. The notebook here goes into much more detail on all the options you can include (like collapsing the data, etc.).
Below is a more detailed explanation:
The first issue is because the example code is running the estimators on non-collapsed long-format data (I also forgot to include the data cleaning step, although that wouldn't have caused any problems with this specific example since the simulated data is already clean).
The estimator requires the 'm' column to mark observations as belonging to movers or stayers. In previous versions of the package, the FE estimator used to require event study format data but I recently changed the code so it uses long-format data.
BipartitePandas automatically creates the 'm' column when (a) data is collapsed or (b) data is converted to event study, so in previous versions the 'm' column was automatically created no matter how you ran the code.
Now, since the data used in the example code is just the long form, non-collapsed data, there is no 'm' column. There is a function, .gen_m(), that will generate the 'm' column. I updated the code to automatically include this.
The second issue was an issue with the documentation, thank you for pointing that out. I forgot to include 2 steps: first, the data cleaning step, and second the clustering step.
I just uploaded a new version to Pip, version 1.10. Make sure to update it, and try running the following code:
import pytwoway as tw
from bipartitepandas import SimBipartite
# Create SimTwoWay object
sbp_net = SimBipartite()
# Generate data
sim_data = sbp_net.sim_network()
# Below is identical to first example, except we are now using the simulated data
# Create TwoWay object
tw_net = tw.TwoWay(sim_data)
# Clean data
tw_net.prep_data()
# Fit the FE estimators:
tw_net.fit_fe()Then for CRE, add the following:
# Cluster to prepare for CRE
tw_net.cluster()
# Fit the CRE estimator
tw_net.fit_cre()Again, thank you for pointing out these issues! Please let us know if it works now.
adamoppenheimer
commented
3 years ago I'll also leave this example code for running the heteroskedastic correction:
import pytwoway as tw
from bipartitepandas import SimBipartite
# Create SimTwoWay object
sbp_net = SimBipartite()
# Generate data
sim_data = sbp_net.sim_network()
# Below is identical to first example, except we are now using the simulated data
# Create TwoWay object
tw_net = tw.TwoWay(sim_data)
# Clean data
tw_net.prep_data(he=True)
# Fit the FE estimators:
tw_net.fit_fe({'he': True})I just uploaded some more small changes, the most recent version should now be version 1.11.
nreigl
commented
3 years ago I have updated the package via pip and tried to run the provided example code. Everything seems to work now.
I might have run into another problem for the Python code example but I need to test some things first and would open another issue separately.
Thank you for the fast response and fix.
Hi,
in a fresh virtualenv with Python 3.9.6 installed I try to run the code for the simulated example provided in the online documentation.
Expected behaviour
On simulated data, I can fit the FE estimator without exceptions being raised.
Observed behaviour
The function
tw_net.fit_fe()fails with the following errorTraceback (most recent call last):
A similar error is raised when trying to fit the CRE estimator with
tw_net.fit_cre().Traceback (most recent call last):
Requirements