Fair Resource Usage per User

[en0ma]

Not sure where this will lead, but I think we need to start looking is fair usage per user accounts. Some users are heavy users and flood the system for smaller users.

I noticed this because I uploaded a small file and it took so much time before it shows up in the staging area - the toCheck queue is probably occupied. I am hoping the issue will be a start for us to discuss how to improve UX by managing resources fairly amongst users.

So far I can think of certain places where UX is not so good due to no fair scheduling;

• the tocheck queue can be flooded by heavy users
• heavy users tend to make too many deals within a period
• too many deals are sent too same miners within a period

To improve the above issues, we need to;

• figure out how to allow fair usage of the tocheck
• figure out fair usage of deal-making amongst users
• figure fair usage for selecting miners for deals

[alvin-reyes]

Just opening up some discussions here.

Some ideas:

• we can create a BucketQueue for the ContentManager. We pre-assign buckets in range of sizes (this sorts it by request size). Each bucket will then have a priority queue to calculate the "fairness" in the context of that bucket only. This includes deal making, and miner selection.
• miner-selection - this is currently round-robin (i think) so I think we should create a "reputation" system - either by accumulating internal points for each miner that finalizes as a deal + the loyalty (how long is the miner been available) - still need to quantify this.

[corinne-antonia]

@whyrusleeping @alvin-reyes @en0ma Do we want to have a meeting to discuss if we should work on this?