chrisokasaki / TradeMaximizer

Tool for finding the largest possible set of trades in BoardGameGeek-style "math trades".
MIT License
39 stars 14 forks source link

Improving wishlist rankings (proposition, discussion) #2

Open mrbrdo opened 3 years ago

mrbrdo commented 3 years ago

Hello,

Not sure if you still follow this project, but I have a possible PR with some changes that could improve the "happiness" of the choices with the penalty of number of trades. I am interested in your opinion if this would be useful in the real world. I already did the code modifications and came up with the following results, where avg. fulfilled want position means the average position of the received item on the receiver's wantlist (#1 being most wanted):

Meaning that originally the people that traded got their 15th most desired item on average, while with the updated algo they got their 10th most desired item on average. However the median - most often - received desired item only changed from number 8 to number 7 in this case.

Explanation of the thought process and changes:

The original algorithm's core goal is to maximize the number of trades, hence the weights of non-trades is set very high. In the case with no wishlist ranking (preferential ordering, let's call it wishlist ranking), this is probably the optimal way. However, when there is a wishlist ranking, depending on the input data, this can mean that many people will get the less desired items on their wishlist, because the algorithm values the number of trades much more than the wishlist ranking. By lowering the weight on the non-trade, it is possible to achieve better fulfilled wishlist rankings (people getting the items they want more), but the number of trades will be less. Of course, this then becomes a parameter question of where the trade-off is worth it. The parameters of this trade-off are, in my opinion, the percent of total trades willing to sacrifice, for the percentage of average wishlist ranking improvement. This could be done through iterative repetitions and bisection (if needed) to find the desired balance between total trades and best wishlist rankings. I only did a quick modification and test on the sample data to confirm this idea. What do you think about this? In the ask.txt sample, we got 35% a better fulfilled average wishlist rank for the cost of 12% less trades - 12% of people that would otherwise trade, did not trade in this case, but the ones that did trade got their 10th favorite item instead of their 15th favorite, on average.

Another thing I was thinking about is the weighting of the first and last item on the wishlist. I think generally, people will much prefer the first item in the wishlist, and at least in case of a shorter wishlist, much dislike the last item in the wishlist (still better then a no trade of course). So I also experimented with adjusting the weights of the first and last item in the wishlist, by simply adding a fixed penalty to non-first items and twice that for the last item. In the case of the same ask.txt example, with some value I tested I got 60% more people getting first-choices (16 instead of 10 people), at the expense of the average wishlist ranking going from 9.8 to 9.96 and getting 2 trades less (171 instead of 173).

My thinking is to create a simple parameter in the way of a small range of preference (e.g. number 1-5), which would determine how much trades to sacrifice for better average wishlist ranking (e.g. sacrifice up to 10% trades for at least 30% better rankings), and also how much trades and average ranking to sacrifice for getting more "number 1" picks and less "last picks".

Any opinion on this?

chrisokasaki commented 3 years ago

Early on, I thought something like this would be a popular choice. Somebody on BoardGameGeek even ran a couple of experimental trades based on maximizing happiness rather than number of trades. But it never caught on. Instead, the community settled on maximizing number of trades, often with maximizing number of distinct users as a secondary criterion.

In fact, many trades abandoned priorities altogether, at least partly because managing those priorities became unwieldy as the sizes of trades grew, especially when those wantlists were created by third-party tools where the user was clicking on the items they wanted.

I think the fundamental issue is that most people would prefer to have a greater chance to get something on their list, even if it was the least wanted item on their list, than have a greater chance of getting nothing (even if that meant that if you did luck out, you might end up with something better than you otherwise would have).

tjhighley commented 3 years ago

I think the community might prefer something like this:

When a participant goes through all the work to list a game and choose wants, it stinks to walk away with nothing at all, so it makes sense that most users want to maximize the total number of trades in order to minimize their own chances of getting nothing. However, I suspect most users would prefer one higher-want item to two lower-want items. They just don't want to increase the chances of leaving empty-handed. There are a number of ways to define "overall happiness", but they would mostly require ranked lists from the participants, which isn't reasonable for extremely large trade lists.

T.J.

Timothy L. Highley, Jr., Ph.D. Associate Professor Department of Mathematics and Computer Science La Salle University


From: Chris Okasaki @.> Sent: Thursday, March 18, 2021 10:25 AM To: chrisokasaki/TradeMaximizer @.> Cc: Subscribed @.***> Subject: Re: [chrisokasaki/TradeMaximizer] Improving wishlist rankings (proposition, discussion) (#2)

External Email: Use caution and do not click links or open attachments from unknown senders.

Early on, I thought something like this would be a popular choice. Somebody on BoardGameGeek even ran a couple of experimental trades based on maximizing happiness rather than number of trades. But it never caught on. Instead, the community settled on maximizing number of trades, often with maximizing number of distinct users as a secondary criterion.

In fact, many trades abandoned priorities altogether, at least partly because managing those priorities became unwieldy as the sizes of trades grew, especially when those wantlists were created by third-party tools where the user was clicking on the items they wanted.

I think the fundamental issue is that most people would prefer to have a greater chance to get something on their list, even if it was the least wanted item on their list, than have a greater chance of getting nothing (even if that meant that if you did luck out, you might end up with something better than you otherwise would have).

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fchrisokasaki%2FTradeMaximizer%2Fissues%2F2%23issuecomment-801971007&data=04%7C01%7C%7C7b77c751747f41bd654b08d8ea19a1e9%7C1036f37b0d59489ca8801a6aebd2bd07%7C0%7C0%7C637516743115116442%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2BwK%2FyQWDB7Sf1ebZm3ems%2BA0TstvXazT5M29lmvJN3M%3D&reserved=0, or unsubscribehttps://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABF4D6GLHBQRXKRA4YGCKSLTEIEMHANCNFSM4ZL32CHA&data=04%7C01%7C%7C7b77c751747f41bd654b08d8ea19a1e9%7C1036f37b0d59489ca8801a6aebd2bd07%7C0%7C0%7C637516743115126432%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=XOXV6x%2FxaLiT2Kd9OWf3qLmzs1T%2BXEqaqG2DDdpf%2BqA%3D&reserved=0.

chrisokasaki commented 3 years ago

I think the community might prefer something like this:

  • First: maximize number of individuals trading
  • Second: maximize total overall happiness

Yes, I think that ordering would be appealing. The catch is that we don't know how to maximize the number of individuals trading efficiently. Right now, we try it a bunch of times and keep the one that was best, but with no guarantee that it's the best possible. That, of course, is a compromise that seems to me okay as a secondary criterion, but less so as the primary criterion. (Opinions could certainly vary...)

Even assuming that technical hurdle was overcome, if I put on my unintended-consequences hat, I have two potential concerns:

First is the extent to which maximizing number of individuals trading encourages people to game the system by offering a single item with a ridiculous shoot-for-the-moon want list. Of course, they could do that now, but I think the chances of success are small enough to not be worth the effort. Changing the primary criterion would increase the chances of success, but it's impossible to say how much without data. Regardless of the actual likelihood of success, if this came to be perceived as a good strategy, I think the impact on the community would be negative.

Second is the extent to which it would encourage people to make puppet accounts so that they could make a number of entries that look like they are coming from separate individuals. Again, they could do that already, but manipulating a secondary criterion is less likely to be effective than manipulating the primary criterion.

mrbrdo commented 3 years ago

Very happy for such a quick response, much appreciated.

I've seen mention on the bgg forums about ranked wishlists not being commonly used, although I don't understand the reason for this. My experience with math trades is limited, though, but for me the wishlist order is important. For me, getting my least wanted item is not that much worse than not trading, definitely not a billion times worse. Maybe twice as worse, sure, but not a billion times worse. If you factor in shipping costs, it may just be a 50-50 I don't care scenario. Surely from an UX perspective and a dev with any JS experience, it would be extremely easy to make an in-browser sortable list with "move to top" and "move to bottom" buttons. I think this is quite useful enough and would work fine for larger lists too. Alternatively you could do sorting into 3 groups (or 3 wishlists), like "really want", "want" and "want less". I think most users will have a few games that they'd really want and a few games they'd be fine with just to make the trade happen. It doesn't mean that most people won't trade instead of getting their least favorite item, it means just a very small number will not trade but those that do trade will get quite more desirable stuff. It doesn't mean that someone won't trade instead of getting their top pick, it means someone won't trade instead of getting their least desirable pick(s).

In the current algo (or more accurately algo weights), the penalty for no trade is so high, that the actual ranking matters much less, but I do think it can have a big impact on happiness. Just with ranked wishlists you can get a bit better results for sure, but sacrificing a few trades can give much better results. Of course there is a trade-off at some point, but in my test going from 173 trades to 171 trades (1.1% less) makes 16 people get their first pick instead of 10 people (60% more). I would say that trade-off is quite worth it. If nothing else it makes 6 people more happy and 2 people less happy. In that example I put extra weight only the first and last pick more heavily, the rest are just square ranks, and I substantially lowered the weight of non-trade. I'd have to do more trial and error but I'm pretty sure that even for a few percent less trades you can get substantially better results (guesstimate of 10-30%).

More or less the question is if people would use ranked wishlists at all or not. I'm worried that the assumption that people don't want this could be based simply on what readily-available web tools were used for wishlist creation and what was simply easier for less-techy users. Many people will just use the default setting or copy the settings from some other guy without really understanding it, if the choices are not presented in a really newbie-friendly way and easy to change.

chrisokasaki commented 3 years ago

When I first made TradeMaximizer, I was in much the same boat I think you're in now -- I thought that the "happiness" parts would be much more important than they actually turned out to be in practice.

I'll note that if you want to run trades where happiness plays a greater role, you can already do that by using the NONTRADE-COST option (with a smaller than normal cost) along with one of the priority schemes. This will allow trade cycles that generate a lot of happiness even if that means fewer total trades. However, this is all based on the total cost, not on being within some percentage of the maximum number of trades.