VisionEval / VisionEval-Dev

Development version of VisionEval framework
https://visioneval.github.io/
Apache License 2.0
6 stars 32 forks source link

Incorrect Household Table Measures #172

Open sylvan-sh opened 2 years ago

sylvan-sh commented 2 years ago

Running VE-State with this release. Of relevance to this issue are the Household table fields:

jrawbits commented 2 years ago

I agree the computations are obtuse. The actual values for AveGPM and AveKWHPM that show up in the Datastore are computed in CalculateVehicleOperatingCost.R, circa lines 1314 to 1319, by using the tapply function to add up the proportion of each vehicle type times either the NetGPM or NetKWHPM for each vehicle type (none of which have anything to do with a number of miles each vehicle is driven, but rather the proportion of miles driven by each vehicle type)

As written (and I think observed in Sylvan's report), that function works fine to identify gas versus electric DVMT, as long as each household has only one type of vehicle.

When both types of vehicle are present, the proportion is still applied to all the vehicles in the household (so we get GPM for electric as well as non-electric vehicles and likewise for KWHPM - both vehicle types participate in both numbers). The net result is exactly what Sylvan observes: that AveGPM and AveKWHPM are computed against total household DVMT, not against the DVMT for the respective vehicle types, so when you back-compute DVMT you get total DVMT not DVMT by vehicle type.

I suspect what is happening is, in fact, the intended behavior: we are expected to apply AveGPM to the TOTAL household DVMT, not DVMT by vehicle powertrain type. And it is intended because the interesting policy question those numbers inform is how a household's daily travel might get allocated to KWH or G under different circumstances. Essentially, the code responds to this question: for each total mile driven by the household in any vehicle type, what is the average number of gallons used and what is the average number of KWH used. All of which is highly relevant as shorthand for total household energy consumption.

If you want the DVMT by vehicle powertrain type, you could go directly to the Dvmt field in the Vehicles table and tabulate it by the Powertrain field in the Vehicles table - those numbers are computed in CalculateTravelDemand. Or compute household values by tabulating against the vehicle's HhId? The pitfall I see is that all the household Dvmt adjustments made due to various cost policies in modules that run after CalculateTravelDemand may never be back-propagated to the Vehicle$Dvmt field: they're computed at the total household level. So VisionEval may just be using the Vehicle Dvmt as a starting proportion, and all the subsequent computations revolve around adjusting and evaluating total household Dvmt and the consequent total household energy consumption (expressed as energy per mile of household travel).

I'll leave this issue open for a while in case anyone wants to inject further insights, or to marvel at how doing sensible things in a model can still seem entirely perverse if you're coming at it with a different set of needs. Or we can treat this as a documentation issue rather than a "broken code" issue.

jrawbits commented 2 years ago

Since these numbers are misleading, it is proposed to remove AveGPM and AveKWHPM entirely since they are not computed usefully (per the original issue). Any thoughts?