spatialmodel / inmap

InMAP reduced-form air quality model for fine particulate matter (PM2.5)
GNU General Public License v3.0
59 stars 41 forks source link

Issue with reading Mortality shapefile when using SR Predict #107

Open julialromero opened 2 years ago

julialromero commented 2 years ago

This issue is the same as the previously submitted one.

What are you trying to do?

I am trying to run inmap sr predict with mortality-related output variables.

What did you do?

I used inmap run and inmap sr predict. First, I tried each using our own input data files and config file. Second, I tried each using the github repo data at the path cmd/inmap/. I used exampleConfig.toml, the provided test data at the default paths, and the only thing I changed in the config file was the SR.OutputFile path to be the path to the IRSM matrix downloaded from Zenodo linked here.

Here are the results for the four attempts to run with mortality output:

  1. Using inmap run with our data/config file along with some example data needed as input to inmap run. This did not throw an error and it outputted the target mortality data. Success
  2. Using inmap sr predict with our data/config file. This threw the error message "inmap: undefined variable name 'AllCause'" and terminated.
  3. Using inmap run with only example data/config files from the github. This did not throw an error and it outputted the target mortality data. Success
  4. Using inmap sr predict with the same example data/config files. This threw the same error of "inmap: undefined variable name 'AllCause'" and terminated.

If I remove the mortality-related output variables then inmap sr predict works, but we don't get these mortality outputs that we want. Other things I tried are: playing with the capitalization of AllCause to allcause in the config file and moving the Mortality Rate and Census related fields in the config file to the top.

What did you expect to happen?

We expected inmap sr predict to successfully run and compute all the output variables, but when we use AllCause in the formulas for output variables, the previously mentioned error message is thrown. It is the same error message every time.

What actually happened?

With the exact same setup (except that inmap sr predict uses the IRSM file and inmap run does not), inmap run works correctly while inmap sr predict throws the error "inmap: undefined variable name 'AllCause'" and terminates.

What version of Go and InMAP are you using?

Using inMAP version: inmap-v1.9.2-linux-amd64 We haven't used Go at all. To run inmap we use the commands: ./inmap run steady --config="configfile.toml" and ./inmap sr predict --config="configfile.toml"

Does this issue reproduce with the current master?

Yes

vxhuang commented 2 years ago

Hi Julia,

I am trying to use inmap srpredict and run into the same issue. However, since I only need air pollution results but not mortality results, I am curious if I can work around this issue. In your post, you said "If I remove the mortality-related output variables then inmap sr predict works", could you please let me know how you removed the mortality-related output variables? Did you just remove them in the [OutputVariables] part? If yes, do I need to modify the [VarGrid.MortalityRateColumns] part? Thank you!

Best, Xinyuan

ctessum commented 2 years ago

I think the issue here might be that the mortality rate variable in the SR matrix might just have a different name. If I remember correctly it might be MortalityR or something like that. So the fix might just be to replace all instances of allcause in the configuration file with MortalityR. However, keep in mind that the mortality rate data in that file is probably relatively old, maybe from 2005 or so.

vxhuang commented 2 years ago

Thank you, Chris! I will try this today and see if it works.

vxhuang commented 2 years ago

I changed it to MortalityRate following this post: https://www.inmap.run/blog/2019/04/20/sr/ it seems to work in the first step, and got following error:

Screen Shot 2022-04-19 at 14 32 02 Screen Shot 2022-04-19 at 14 35 04
acwatt commented 9 months ago

Regarding @vxhuang's index out of range error, I think I tracked it down to the Baseline variables. Try excluding any baseline variables in the OutputVariables section/command-line flag, and see if it runs.

I get a similar error (with a different index that is out of range) when using any one (or combination) of the baseline variables, and it doesn't occur when using any one (or combination) of the other variables included in the 2005neiConfig file. I don't think it's a problem of the variables being spelled differently (like with AllCause and MortalityRate), because that should throw an undefined variable error at the beginning. This index out of range error is different because the error only occurs at the very end of the simulation (I think) when inmap is attempting to access cell values to build the output variables. I am guessing that this occurs because the baseline variables are not actually produced using the SR matrix, but the CheckOutputVars function is using a list of variable names that includes the baseline variables.

I think that I can get away with only using the output PM variables without using any baseline variables. But that may be a sizeable limitation to other projects.

@ctessum, would it help if I submit a different issue for the index out of range error?