search

bicyclingplus / atp-bc-tool-analysis

Analyzing inputs/outputs from the CTC Active Transportation Benefits/Costs tool, to identify and investigate potential issues (analysis notebooks only - input/output data is not included)
0 stars 0 forks source link

Physical Activity (MMET) Calculations #9

Open mRaffill opened 4 months ago

mRaffill commented 4 months ago

The total physical activity benefits from the project are several billion added "marginal metabolic equivalent tasks." Is it really supposed to be this large or is there some error in the calculations?

Some initial thoughts/notes:

mRaffill commented 4 months ago

I tried calculating it this afternoon (c373244) and I did get much, much smaller results than the most recent output.

Using the general equation: Increase in Daily MMET = Increase in Active Travel % not route shift MMET per hour/miles per hour Everything except the active travel is a fixed constant, so it should be possible to take the total active travel across all projects and then multiply by these constants to get the MMET. I just multiplied the total increase in BMT and total increase in WMT by the corresponding constants, and then added them together to get the overall increase in MMET for both modes. (The report just has one column for MMET, not separated by mode)

Result from the report: 387,666,171 (looks like what Peter got, although 388 million, not billion) Result from my quick calculations: 75,039 (WAY smaller, only 75 thousand) The ratio between these is 43,345 This seems really bizarre and I have no idea what could make that big of a difference in the output. I could have missed some additional factor in my calculations or misunderstood what the column means, but based on the documentation it looks like it should just be this simple.

Also, units: I notice the documentation calls this Increase in "Daily" MMET, but the increase in active miles traveled aren't daily, right? The tool calculates the increase in active miles traveled over one year and then 20 years. If the MMET output is calculated as some multiple of the active miles traveled output, wouldn't it also be in terms of years? Is there another factor to convert it into days? And Dillon also mentioned potentially having errors with hours vs minutes. Definitely need to figure out what units and time scales the inputs and outputs are in.

mRaffill commented 4 months ago

Graphing this across all of the individual projects, it looks like the tool MMET and my calculation are in fact constant multiples:

my calculation tool calculation
image image

The ratio between these is a constant 5158.887881.

My first guess is that part of this difference is that the tool output is scaled by the 20 year project time frame. After applying the 20 year time frame, the ratio between the results is exactly 365. Which is obviously supposed to convert from MMET per day to MMET per year.

This is just from some informed guesses, but it looks like if I

  1. calculate the MMET using the equation from the documentation ($\text{πΌπ‘›π‘π‘Ÿπ‘’π‘Žπ‘ π‘’ 𝑖𝑛 π‘‘π‘Žπ‘–π‘™π‘¦ 𝑀𝑀𝐸𝑇}_π‘š = 𝐴𝑇𝐼_π‘š βˆ— \frac{𝑀_π‘š}{𝑆_π‘š}$)
  2. multiply by 365 days per year
  3. multiply by 20 years (with the discount rate)

then I do get the same increase in MMET as the tool.

mRaffill commented 4 months ago

The question now is whether this is how it the calculation is supposed to work. This depends on what units of time the BMT and WMT columns are using.

If the BMT and WMT columns here are the increase in miles traveled per day, not per year or per 20 years, then I think the calculation is correct, because these conversion factors are needed to report the full benefit over 20 years. But if the increase in BMT and WMT are already reported over one year or 20 years, then the tool is scaling the outputs multiple times, which would make the results much larger than they should be.

According to the documentation:

All benefits and intermediate estimates are first calculated at either the daily or annual level. Daily values are assumed to be daily annual averages and multiplied by 365 to be comparable to annual estimates (e.g., daily bike miles traveled for a project is multiplied by 365). Once all values are at the annual level, depending on the user defined project time frame, benefits are calculated as a net present benefit (currently only 1 year and 20 years are allowed) and with a 4% discount rate.

dtfitch commented 4 months ago

Thanks for figuring this out! Can you email Matt to ask about the scaling. 388 million seems way more reasonable to me than 388 billion :)

Dillon Fitch-Polse 530.601.7624 @.** Co-Director BicyclingPlus* Research Collaborative https://bicyclingplus.ucdavis.edu/ Research Faculty, UC Davis Institute of Transportation Studies

On Thu, Apr 25, 2024 at 2:28β€―AM mRaffill @.***> wrote:

The question now is whether this is how it the calculation is supposed to work. This depends on what units of time the BMT and WMT columns are using.

If the BMT and WMT columns here are the increase in miles traveled per day, not per year or per 20 years, then this is the correct way to report the benefits over 20 years. But if the increase in BMT and WMT are already reported over one year or 20 years, then the tool is scaling the outputs twice, which would make the results much larger than they should be.

According to the documentation:

All benefits and intermediate estimates are first calculated at either the daily or annual level. Daily values are assumed to be daily annual averages and multiplied by 365 to be comparable to annual estimates (e.g., daily bike miles traveled for a project is multiplied by 365). Once all values are at the annual level, depending on the user defined project time frame, benefits are calculated as a net present benefit (currently only 1 year and 20 years are allowed) and with a 4% discount rate.

β€” Reply to this email directly, view it on GitHub https://github.com/mRaffill/atp-bc-tool-analysis/issues/9#issuecomment-2076757813, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZ3LZ6DEO4BVH6S2QMAAQ3Y7DEFFAVCNFSM6AAAAABGUSCGYOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANZWG42TOOBRGM . You are receiving this because you are subscribed to this thread.Message ID: @.***>

mRaffill commented 4 months ago

Confirmed by Matt: the BMT and WMT are reported as daily numbers. MMET is reported over 20 years (the full project timeframe).

So the miles traveled are multiplied by 365 to go from days -> years, then inputted to the MMET equation in the documentation to calculate MMET per year, then projected to the 20 year project timeframe.

This means the tool outputs match what I get by calculating manually, so ~388 million total MMET should be the correct result!

dtfitch commented 4 months ago

Thank you for doing this check. So, somehow there was some misreading of the magnitude and 388 million was reported as 388 billion. Phew!

On Tue, Apr 30, 2024 at 3:36β€―PM mRaffill @.***> wrote:

Confirmed by Matt: the BMT and WMT are reported as daily numbers. MMET is reported over 20 years (the full project timeframe).

So the miles traveled are multiplied by 365 to go from days -> years, then inputted to the MMET equation in the documentation to calculate MMET per year, then projected to the 20 year project timeframe.

This means the tool outputs match what I get by calculating manually, so ~388 million total MMET should be the correct result!

β€” Reply to this email directly, view it on GitHub https://github.com/mRaffill/atp-bc-tool-analysis/issues/9#issuecomment-2087614264, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZ3LZ76OOSB27OHCNXBDUDZAAMFJAVCNFSM6AAAAABGUSCGYOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBXGYYTIMRWGQ . You are receiving this because you commented.Message ID: @.***>

-- Dillon Fitch-Polse 530.601.7624 @.** Co-Director BicyclingPlus* Research Collaborative https://bicyclingplus.ucdavis.edu/ Research Faculty, UC Davis Institute of Transportation Studies