search

cybergreen-net / pm

Tech project management repo (issue tracker only)
2 stars 1 forks source link

bug: rank was calculated per risk instead of per country / ASN #99

Open aaronkaplan opened 7 years ago

aaronkaplan commented 7 years ago

If you check out the detailed description of the metric 2.1, you will see that it has a rank() function over all risks (open NTP, open recursive DNS, etc) . Not a country within a risk category. So, the rank was implemented wrongly! We must sum up all risks and weigh by AF (for 2.1) and then calculate the rank of that.

image

However, if you go to the country detail page, you get for example: image

Please re-inspect the rank() calculation and check if it really conforms with the intention of the designers.

Note: this will also solve issues #97,

rufuspollock commented 7 years ago

@aaronkaplan this choice here is deliberate and not a bug - the rank in the UI is clearly associated with a given risk. Whilst the new "overall metric" does allow us, in theory, to rank countries overall we have refrained from doing so explicitly in the UI because of the various outstanding issues (essentially it is a DDOS metric and we have risks like Mirai that aren't included, we may get other risks like spam etc etc).

I would therefore proposing closing this issue as WONTFIX / INVALID as there is no bug in the way we have calculated the ranks here or displayed them on the site.

There may be a feature missing: i.e. overall rank but that never got flagged in the repeated UX reviews and the lack of it was not an oversight but a choice :wink: (I definitely agree it is something to discuss and happy to have a new issue or a rename of this issue to reflect a request for it: e.g. "Display Overall Rank based on new 2.1 Metric (basically DDOS)")

aaronkaplan commented 7 years ago

re: "this choice here is deliberate and not a bug " -> then it is implemented wrongly. Please check the original paper.

I will definitely REOPEN this indefinitely (in case it gets closed) until it is fixed (or until Yurie says "okay, we can omit that"). But from a TD perspective, even though it was not noticed, it's software not written according to specs. @YurieIto any comments?

YurieIto commented 7 years ago

please implement the rank as v.2 metrics paper (by Dan Geer and others) specifies. also like to see the description of rank (calculation method) on the glossary page after implemented.

On Tue, Mar 21, 2017 at 7:05 AM, AaronK notifications@github.com wrote:

re: "this choice here is deliberate and not a bug " -> then it is implemented wrongly. Please check the original paper.

I will definitely REOPEN this indefinitely (in case it gets closed) until it is fixed (or until Yurie says "okay, we can omit that"). But from a TD perspective, even though it was not noticed, it's software not written according to specs. @YurieIto https://github.com/YurieIto any comments?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cybergreen-net/pm/issues/99#issuecomment-288045759, or mute the thread https://github.com/notifications/unsubscribe-auth/AS4bvijfvEwbfhMzurXK9mj78scu82MKks5rn68RgaJpZM4MiMr1 .