search

impactlab / eemeter

‼️ MOVED TO https://github.com/openeemeter/eemeter - Core computation engine for the Open Energy Efficiency Meter
https://eemeter.readthedocs.io/
MIT License
25 stars 13 forks source link

No model fitness criteria. #53

Closed philngo closed 9 years ago

philngo commented 9 years ago

One of the reasons the PRISM implementation doesn't check for model fitness is that we haven't yet implemented any model fitness criteria. We might start with implementing the Coefficient of Variation of Root Mean Square Error (CVRMSE).

philngo commented 9 years ago

Added CVRMSE

jgagliano commented 9 years ago

Great! Also should have R2 and the flatness index that PRISM defines. Then we can use a combination of these metrics to determine quality of regressions following PRISM. Do you have the PRISM documentation?

From: Phil Ngo [mailto:notifications@github.com] Sent: Friday, March 20, 2015 1:54 PM To: impactlab/eemeter Subject: Re: [eemeter] No model fitness criteria. (#53)

Added CVRMSE

— Reply to this email directly or view it on GitHubhttps://github.com/impactlab/eemeter/issues/53#issuecomment-84087902.

IMPORTANT NOTICE: This email message is intended to be received only by persons entitled to receive the confidential information it may contain. Email messages to clients of Performance Systems Development may contain information that is confidential and legally privileged. Please do not read, copy, forward, or store this message unless you are an intended recipient of it. If you have received this message in error, please forward it to the sender and delete it completely from your computer system.

philngo commented 9 years ago

This is the main doc I've been referencing http://www.marean.mycpanel.princeton.edu/~marean/images/prism_intro.pdf

There are probably others I should be aware of - are you thinking of a specific document?

jgagliano commented 9 years ago

Yeah the actual PRISM reference documentation only exists as a paperback book. We should figure out how to get ours scanned in and make available.

From: Phil Ngo [mailto:notifications@github.com] Sent: Friday, March 20, 2015 2:03 PM To: impactlab/eemeter Cc: Jerone Gagliano Subject: Re: [eemeter] No model fitness criteria. (#53)

This is the main doc I've been referencing http://www.marean.mycpanel.princeton.edu/~marean/images/prism_intro.pdfhttp://www.marean.mycpanel.princeton.edu/%7Emarean/images/prism_intro.pdf

There are probably others I should be aware of - are you thinking of a specific document?

— Reply to this email directly or view it on GitHubhttps://github.com/impactlab/eemeter/issues/53#issuecomment-84089935.

IMPORTANT NOTICE: This email message is intended to be received only by persons entitled to receive the confidential information it may contain. Email messages to clients of Performance Systems Development may contain information that is confidential and legally privileged. Please do not read, copy, forward, or store this message unless you are an intended recipient of it. If you have received this message in error, please forward it to the sender and delete it completely from your computer system.

philngo commented 9 years ago

That would be extraordinarily helpful. Let me know if you're able to do that

potash commented 9 years ago

I've put a request in to Interlibrary Loan for a copy of Miriam Goldberg's dissertation "A Geometrical Approach to Non-differentiable Regression Models as Related to Methods for Assessing Residential Energy Conservation." Based on the citations of it in the Robust PRISM paper it should be quite useful to us.