All code for Ricardian Land Value estimation paper. Authors: Gold, Binder, and Nolte.
Note: This README is an ongoing documentation of our work and should not be taken as final. I've tried to include caveats where I expect something to change significantly as we finalize data and analysis strategies.
My general strategy here is to load, clean, and model the data all in one pipeline, using magrittr's pipe (%>%), which allows you to make fewer copies of your data as you go along.
First, we load the packages and specify where the data file (or files) reside. I want to keep this part vague for now, in case we want to utilize remote database storage. If so, RSQLite, Google BigQuery, Hadoop, and associated packages like DBI should be considered. Here's a helpful post on how to integrate remote database connections with RStudio analysis workflows. I assume Google Drive won't be sufficient, but that's also worth a shot, as the googledrive package allows for seamless integration with Drive directories.
Next, we specify the names of all the variables we want to include in our model, both the outcome y (ls_price) and the vector of explanatory variables X.
The general setup is as follows:
ols_model <- arrow::read_parquet(sample) %>%
dplyr::select(all_of(model_variables)) %>%
stats::lm(ls_price ~ ., .) %>%
summary(.) %>%
stats::coef(.) %>%
as.data.frame(.)
Here's a brief run-down:
Note: the periods in the code above are used in pipe-based workflows to specify the object created up to that point in the pipe.
randomForest.R contains a ton of code for a basic random forest implementation with the base randomForest package. Unfortunately, randomForest scales poorly, quickly getting bogged down with larger datasets and more intensive forest modeling (e.g., greater numbers of features tried at each split, commonly denoted as "mtry").
To fix this problem, I've moved to the ranger package, which vastly improves performance for big data use compared to randomForest or other older packages like Rborist.