Matching micro moments and convergence Issue

[giheungkim]

Hi Jeff,

I was wondering what the best practice would be if I encounter the following situation:

Suppose I have two parameters I'd like to estimate - 1) mean price coefficient and 2) price x income heterogeneous coefficient.

I have one price IV and one micro moment from the data, E[ income_i x price_j].

In theory, shouldn't this be VERY easy for the optimizer to fit? I have two moments and two parameters.

But I am actually having an opposite problem. Even in this simple model, the micro moment is not getting matched and the optimizer rarely converge.

What would be a starting point to debug/understand what's going on?

Any help would be great!

[jeffgortmaker]

Given a well-specified model without numerical issues, yes, this should be fairly easy for an optimizer to handle.

Of course there are plenty of things that can go wrong!

If you haven't already, I recommend looking through my recent mixtape exercise about adding a single micro moment to a somewhat more complex model. Make sure you're following the relevant best practices outlined there and in the first exercise.

Since you're presumably allowing PyBLP to concentrate out the mean price coefficient, your optimizer should really just be optimizing over a single parameter, the interaction on price x income. One thing you can and should check is to just create a plot of the objective value (using optimization=pyblp.Optimization('return') and method='1s' at different starting values for pi) to see why your optimizer is having trouble minimizing this objective. Could be that the objective is fairly choppy, or something else -- multiple local minima is unlikely given the simplicity of your problem.

[giheungkim]

Hi Jeff,

Thank you so much for the suggestion. I've plotted the criterion with 'return' and with different values, and indeed was able to spot a problem. Didn't entirely solve the issue, but it's the problem of the data rather than the package.

Going forward, is there an option to save criterion value and the parameters at each iteration during the optimization process?

[jeffgortmaker]

Great!

PyBLP overrides SciPy's callback argument, so your best bet is to define a custom Optimization function that wraps the SciPy method you're using with some code that saves the objective and parameters.

[giheungkim]

Thank you Jeff!

I plotted the objective value as well as the difference between model predicted micro moment vs. data moment. I am working with E[ income_i x price_j] to fix idea.

The issue was that the model was not able to match E[ income_i x pricej] from the data no matter how high of \pi{income x price} parameter it tried.

Because I am drawing the agents from the same "survey" I construct the moments with, when I standardized the income of agents (and in computation of the moments) to be mean zero, it was able to match the moment precisely.

This is a bit beyond the code and the package, but could you help me understand why standardizing one variable such that matching on the "scale-less" moments performs better? Is this some sign of model specification and my aggregate shares are wrong?

[jeffgortmaker]

Ah, okay. I think this is about what you're implicitly assuming for different preferences for the outside good by income.

When you only have a nonlinear parameter on income x price, you're implicitly setting the intercept of this relationship (typically modelled with a second nonlinear parameter on income alone) to zero. This intercept shifts how much income affects consumers' preferences for the outside good.

When you de-mean or otherwise shift income, you're changing the interpretation of this intercept, and hence the implications of implicitly setting it to zero. Probably de-meaning income fits the data better because the economic content of what it implies about this intercept matches reality better.

Model fit aside, it's still important to understand that by not having a parameter on income alone, you're making an implicit assumption. If you have the data to do so, it can usually help to relax this assumption by matching a moment that pins down this parameter instead of setting it to zero -- this is precisely the motivation for the first part of the third mixtape exercise that I recently put together. If you don't have the data to do this, I'd still probably recommend trying out different values for the parameter to see how it changes your results.

[giheungkim]

Thank you so much - that makes a lot of sense!

In that spirit, is there any way to let the micromoment part of pyBLP to match the choice patterns of consumers with a broader product set to recover the heterogeneous parameters, while the shares are matching the inside good shares?

For example, suppose I am interested in estimating the demand for apartments, which is a very specific segment of the housing market.

The ACS individual renter-level data is an almost ideal micro dataset for this market because it links individuals demographics to their choice of housing characteristics (# of bedrooms, age of bld, rent, etc). This arguably is the "market" for renters, but potentially too big of a definition to fit the demand for apartment with it.

Nonetheless, there is valuable information in the ACS data. For example, if I estimate ACS renter's preference in the MLE framework on observable characteristics of dwelling, I can recover the heterogeneous parameters, such as income_i x rent_j or hhsize_i x bedrooms_j. And because the apartments under study are just a subset of what these renters had chosen, these parameters can be generalized to individuals choosing across apartments, in theory.

The only thing that'd be missing would be the vertical quality of these apartments, or the "deltas", which can be inverted from the observed shares given the heterogeneous parameters.

So in this situation where I want to not shoe-horn the moments from ACS individual data into the market for apartment rentals plus one "composite" outside good, what would be the best way to go about it within the package's framework?

I hope this makes sense.. let me know if it does/doesnt!

[jeffgortmaker]

Is the issue that your micro data for just apartment choices fairly limited, so your parameter estimates aren't very precise? If so, it sounds like this is a classic trade-off between statistical power and internal validity.

Perhaps the best-case scenario is you precisely estimate some subset of your structural parameters from your ACS data (say with MLE or whatever estimator seems reasonable for that dataset), and when doing BLP estimation on your apartments data, just fix those same parameters to their previously-estimated values (by setting their upper and lower bounds equal PyBLP). Of course, this requires that you believe these structural parameters are externally valid across the two different settings.

If you're unable to credibly estimate structural parameters with the ACS data, what might be more feasible and a second-best alternative is to construct some carefully-chosen summary statistics from the ACS data, and match them when doing micro BLP estimation with your apartments data. Statistics like elasticities or correlations are often targeted in, for example, macro calibration studies. You may be less confident that these reduced form statistics are as externally valid as structural parameters, but doing this may be more practically feasible.

Does that help?

[giheungkim]

Thank you for the insight Jeff! This has helped a lot.

What has worked is somewhat similar to your idea but a slightly easier way. I ended up making a prediction on who is likely to be living in an inside good apartment. Then I constructed micro-moments such as E[income_i x price_j | \hat{j > 0}] and fit the data such way. This worked quite well and I think this is a potentially interesting approach of utilizing micro data that cannot be linked directly to market-level data at hand.

Thank you!

[jeffgortmaker]

Great! That seems reasonable to me -- I like it.