Useful comments from various seminars

[jackblun]

The goal of this issue is to summarize feedback we've had from various seminars

[jackblun]

LSE 6th June 2018

Stephen Jenkins email

_"Interesting paper and approach, though I'm not sure I'm persuaded by your argument for using multiple indicators on the RHS. Put differently, perhaps this means more work on the justification of the "proportion of variance of the outcome variable explained" for measuring the degree of intergen mobility. Of course in a bivariate (and linear) world, there is a close relationship between the standard "beta" coefficient and the "correlation". Why might one use multiple indicators on RHS? I suppose one reason might be that the 'income' data for the parental generation are too poor quality, but multiple indicators are available. Some social scientists would create a composite index from these variables and then shove it the regression. A bit reminiscent of IV? I was thinking that your approach might be useful for providing a link between the commonly-used bivariate model used to provide summary measures like Beta, and fully fledged 'models' in which additional explanatory variables are added to the RHS to provide a fuller "explanation" of the offspring's outcome. But then, if so, this presumably constrains the sorts of variables that one would want to include in the additional variable set. (You want to avoid giving the impression of a kitchen sink approach!)

By the way, one of my pet peeves is the weight given to Becker and Tomes's work and the (over) emphasis on "economic models". (I say this because of one of your slides.) Many people were doing a lot of important and interesting work in addition using "mechanical" models, and it's a shame this is not acknowledged. I'm right with Goldberger on this. [Goldberger, A.S.(1989) Economic and mechanical models of intergenerational transmission, American Economic Review Volume 79, Issue 3, Pages 504-513.] Rant over.

I think you might be on stronger ground with your approach using the NO admin data than using the 2 birth cohort surveys. It's not simply the issue of how to define ranks when you're subsampling and selecting so much. It's also that these data sets are of poorer quality than is often acknowledged by economists. E.g. not the same measures of "income" available in each generation; item non-response and attrition; how to handle banded data. Jo and Steve probably don't agree with me! And I think there is something in Goldthorpe's point about social class providing multiple estimates so that don't have to conclude about trends from 2 data points (both very old now!)

BTW One of the reasons for using a rank-based measure is that it measures pure "exchange" mobility, by contrast with "beta" which is a rather odd amalgam reflecting exchange and structural changes. (Strictly speaking; regression to the log mean.) You might be interested in the attached - section on indices , and also the review of intergen estimates._ "

Others / general

• Use elastic net as a tool to introduce idea of overfit / regularization
• Why not find a low-probability event, such as ending up at bottom of income distribution?
• Use income at 42 from BCS / NCDS, later literature tends to do this
• Need to frame exactly what our estimate is getting at better

[jackblun]

Oxford 8th June 2018:

• Boosting and LASSO actually come from the same idea and in regression frameworks tend to give similar results - check out LARS
• Not clear how we should interpret our measure

[jackblun]

Also Oxford 8th June 2018:

There are many other measures of fit we could use, which might better pick up differences between ML and non-ML. I think using some kind of absolute deviations measure would probably make a difference. Essentially we could use any loss function - perhaps we can find something better.

[jackblun]

I just got off a very useful call with Jann Spiess. Main takeaways: 1) Jann, Sendhil and others have work ongoing and mentioned in their JEP article about how aggregate R2 'masks' big improvements at certain parts of the distribution. I think this is very relevant for us. 2) A suggestion from Jann is that we could use a 'sensible' OLS model, say with just income ranks and parent education, or just income ranks on its own as a benchmark, then look at improvements on that. This would kind of be thinking about where the standard models fall down, and be useful for guiding new models to explain certain bits of the distribution. Ways to do this could be: a) Boosting the residuals from the benchmark model b) fitting a bunch of models then using an ensemble to bring them together. The weighting would kind of tell us how well the benchmark model does relative to other models. c) Estimate separately then inspect R2 at different parts of the (child) income distribution. As we know there are non-linearities at the top, there will probably be quite a big improvement there.

In sum, he was actually very enthusiastic about what we have done with the continuous prediction stuff... So even more to think about!

[jackblun]

Talking to Stantcheva at the Bonn summer school - she thinks there is value to this but we have to be very careful with our interpretation, in particular how we interpret what we learn for mechanisms.

[jackblun]

Alan Manning suggests DFL re-weighting as a way to separate mechanisms from different distributions.

[jackblun]

Some suggestions from yesterday:

• We need to relate this better to the notion of completeness from statistics
• We should try to pitch this so that we get citations from both sides of the rank-rank argument
• Develop more on how our method is informative on the validity of different ways of coding variables
• Detailed data section needed with summary statistics, graphs
• We should talk to guido about how to do inference here. How do we interpret the CV bands?
• Should discuss other datasets on which this exercise could be performed.
• Need to be very clear on how ranks are constructed.

[jackblun]

Comments from Guido

• Using some transformation of the ranks would result in more focus on the tails. For example log(R)/(1-R). This would be useful as we are not particularly interested in the middle of the distribution.
• Is there a way to demonstrate quite where we are getting any improvement from ML?
• Regional differences the most interesting. Is there a way for us to determine why these differ so much?
• What does geographical mobility look like between these areas?

[jackblun]

So we have seminars this year where everyone gives anonymous feedback on your presentation. This is actually very useful. I presented our paper and got the attached comments. Theres a lot in here thats quite valuable to us. Its worth looking over

feedback.pdf