search

thieu1995 / permetrics

Artificial intelligence (AI, ML, DL) performance metrics implemented in Python
https://permetrics.readthedocs.io/en/latest/
GNU General Public License v3.0
68 stars 17 forks source link

Difference between documentation and implementation for SMAPE #8

Open chris-fj opened 2 months ago

chris-fj commented 2 months ago

Hello, everyone

I was reviewing the SMAPE for v2.0 and I found something odd that may be a mistake in the documentation or a misunderstanding on my end.

The documentation uses the formula that includes the 2 in the numerator, which, according to this wikipedia page (that is linked into the docs), means that the SMAPE will range between 0 and 2 (or 0% and 200%). However, the docs specify that the SMAPE will range between 0 and 1, which is not totally wrong, because in the implementation (see here) one can see that the 2 is missing and as such, the SMAPE, as calculated by the library, will range between 0 and 1.

The issue here is that the docs shows one formula (the one whose results range 0 and 2) and implements the other one (the one whose results range between 0 and 1), which is a bit inconsistent. Maybe dropping the 2 from the formula will be the easiest solution, although it would be nice to be able to parametrize and have both metrics.

thieu1995 commented 2 months ago

@chris-fj,

Thanks for pointing it out. Basically, I took the commonly used equations as an actual implementation, but in my documentation, I forgot to remove the 2. I will remove it to fit the actual implementation code.