Making sense of results across states

[misea]

After looking at the code for a bit, I got somewhat confused about predictions for results across states.

If I estimate race across states with very different overall racial makeup, I get results I was not expecting. For example, if I compare Auburn, ME (~95% White, 2.5% Black) with Batesville, MS (52% White, 46% Black). I find someone named SMITH is more likely to be White in MS.

library(tidyverse)
library(wru)

vf <- data.frame(
  state = c('ME', 'MS'),
  surname = 'SMITH',
  place = c('02060', '03620'),
  placename = c('Auburn, ME', 'Batesville, MS')
)

census_geo_api(Sys.getenv("CENSUS_API_KEY"), vf$state[1], geo = 'place') %>% filter(place == vf$place[1])
census_geo_api(Sys.getenv("CENSUS_API_KEY"), vf$state[2], geo = 'place') %>% filter(place == vf$place[2])
predictions <- predict_race(vf, census.geo = 'place', census.key = Sys.getenv('CENSUS_API_KEY'))
predictions

yields

  state surname place      placename  pred.whi  pred.bla   pred.his    pred.asi   pred.oth
1    ME   SMITH 02060     Auburn, ME 0.6836567 0.2634975 0.02069956 0.002964457 0.02918179
2    MS   SMITH 03620 Batesville, MS 0.7290747 0.2312587 0.01421624 0.002244195 0.02320616

What I don't get: The values for r_whi etc, are calculated on a per-state basis ignoring national demographics, but the census surname distributions are national. Are these reconciled somewhere or is this just based on an assumption that name distributions are independent of states (something to do with note 4 in your paper)? I admit to being a Bayesian newbie, so I'm likely just missing something or my expectations may be wrong.

[misea]

For comparison, in Diamondhead MS (93% White, 2.9% Black), someone with the name SMITH is has a predicted probability of being White of .95 .

[kosukeimai]

The assumption of the method is that conditional on race, surname and geolocation are independent of one another. Please see https://imai.fas.harvard.edu/research/files/race.pdf for details. Also note that the method is trying to get things correct on average, not for each specific name.

[misea]

Yes, thank you - that's the paper I was looking at. I was wondering whether, given the very different underlying race distributions by state, the concrete implementation of the algorithm (as opposed to the description in the paper) may be introducing avoidable bias. If the set G included all census divisions, not just the ones in a particular state, wouldn't results likely be more comparable across states?

I didn't see anything in the paper that indicated that G should be per-state, though I realize that with certain assumptions the results would match.

[kosukeimai]

You are correct in that the bias is introduced to the extent the assumption is violated. In our experience, a finer census area can give you a better prediction. Unfortunately, the statistics on names and race are only available at the national level although you could collect additional data from southern states, for example, where voter files ask people about their race. You can incorporate these and other additional pieces of information to improve the accuracy of the predictions. In the paper, for example, we use partisanship to improve the prediction.