Allow users to "correct" Vocabulary feature

[RyckRichards]

The problem

Many users have been using this feature to learn a language, to get their sentences translated, etc. However, some of them are not using this properly by choosing a wrong flag/language or not writing the word correctly

Maybe it would help if admins and CMs were allowed to correct them.

[Guybrush88]

related to #1238 ?

[trang]

Related Wall thread: https://tatoeba.org/eng/wall/show_message/31919#message_31919

[LBeaudoux]

I also encountered this cluttering issue when cleaning the Tatoeba search log for Tatominer. I fixed it by building a lexicon for each supported language: when a search query can not be found in the lexicon, it does not appear in the Tatominer lists.

With its own lexicon, Tatoeba would be able to classify the vocabulary queries into 2 categories Then it could return the vocabulary requests found in the lexicon by default, while allowing contributors to switch to the lower quality category thanks to the filter settings.

[jiru]

Maybe the vocabulary feature needs to be designed differently to avoid such problems. How about decorrelating the number of search results from a vocabulary request? Instead, a vocabulary request would work a bit like a collaborative list. The requester fills a free-form text with a description of the wanted word and eventually the context. The goal is to have people link existing sentences to the request, or create new sentences directly from within the request page that are automatically linked to the request. Once the target number of sentences is reached, the request automatically disappear from the list of pending requests.

[LBeaudoux]

@jiru While I believe that collaborative lists can be useful for pushing complex vocabulary requests, I also think it's important to keep a place where contributors can regularly scroll for inspiration. For this to be possible, the flow of new vocabulary requests needs to be high, and thus the effort required to add one needs to be as low as possible.

Getting back to the topic, I feel that the problem we face is similar to that of question-and-answer applications. Perhaps we could take our cue from Quora, which allows contributors to pass or downvote a question:

Applied to the Tatoeba context, when a contributor clicks on the 'downvote' button, it would subtract one from the counter of the targeted vocabulary request. And as long as the counter of a vocabulary request is less than zero, it would not appear in the ranking. Incorrect requests would be the first to be penalized by this system.

As a first step in this direction, I propose that we sort by default the vocabulary requests by descending numAdded and then by descending id.

[jiru]

@LBeaudoux Just to clarify, I am not suggest to replace vocabulary requests by collaborative lists. I am just mentioning collaborative lists for the sake of explaining in which way vocabulary requests could be changed.

For this to be possible, the flow of new vocabulary requests needs to be high, and thus the effort required to add one needs to be as low as possible.

I agree.

Applied to the Tatoeba context, when a contributor clicks on the 'downvote' button, it would subtract one from the counter of the targeted vocabulary request

I am skeptical about your suggestion of downvoting. It solves one problem, but at the same time it brings a new kind of potential problem: users downvoting for personal interests (such as "I hate this user or this language so let’s downvote everything"), possibly worsened by sockpuppetry. Big corporation like Quora, with hundreds of employees, probably have other means of tackling this kind of problems (for example account verification by SMS, bot patterns recognition mechanisms, web of trust among users), but we don’t and probably never will. I am afraid letting users "downvote" (or "upvote") things around could open up a whole can of worms. I’d rather provide additional means to let users filter irrelevant vocabulary requests.

As a first step in this direction, I propose that we sort by default the vocabulary requests by descending numAdded and then by descending id.

I am not sure I understand. In that case, won’t new vocabulary requests (with zero sentences) always be far away in the list and never brought up to the attention of contributors?

[LBeaudoux]

I am afraid letting users "downvote" (or "upvote") things around could open up a whole can of worms.

Allowing users to rate other contributors' sentences would certainly be a very bad idea. But assessing the relevance of a vocabulary request seems less controversial to me.

Besides, we can't always catch a malicious behavior, but we can limit its effects:

• A user could only request (or downvote) a vocabulary item once.
• The downvotes would remain anonymous and the "relevance scores" would not be published at any time, which should largely spare everyone's sensitivities and therefore mitigate the risk of fights.
• Without minimum score at which a request is no longer displayed, the maximum harm a user could do is relegating a phrase to the very bottom of the (default) ranking. I doubt the stakes are high enough to make many users go to great lengths.

won’t new vocabulary requests (with zero sentences) always be far away in the list and never brought up to the attention of contributors?

The goal of the "relevance score" is to progressively relegate the requests that are too technical, or incorrect, or in the wrong language, or difficult to match exactly, while promoting the requests that have the most chance of success.

When two requests are equally popular, I think the more recent one should be favored in order to increase the turnover at the top. For now, it seems that the number of requests for the same word is never very high. The new requests would therefore appear at a fairly high rank.

For contributors who would like to easily identify words with zero sentences, I propose to add a button to optionally sort requests by ascending number of sentences.

I’d rather provide additional means to let users filter irrelevant vocabulary requests.

A system closer to the Quora "pass button" perhaps? If so, I feel that the two approaches would complement each other perfectly:

• The relevance score allows showing bad queries to fewer contributors.
• The "pass button" allows a contributor to no longer see the queries he has decided to ignore.

[jiru]

Besides, we can't always catch a malicious behavior, but we can limit its effects:

I agree with your comments about potential malicious behaviors.

won’t new vocabulary requests (with zero sentences) always be far away in the list and never brought up to the attention of contributors?

The goal of the "relevance score" is to progressively relegate the requests that are too technical, or incorrect, or in the wrong language, or difficult to match exactly, while promoting the requests that have the most chance of success.

When you mentioned "numAdded", I assumed with you were talking about the number of sentences matching the request (actually numSentences). Now I understand better your suggestion and I think it makes sense. Note that about 97% of the requests have numAdded = 1, so

as long as the counter of a vocabulary request is less than zero, it would not appear in the ranking

this might be a bit too strict because only two downvotes would remove the request. I’m in favor of pushing down the requests lower in the list instead.

A system closer to the Quora "pass button" perhaps?

The personal pass button sounds like a good idea to me. If we are to add such a button, can’t we simply use it in a similar way as your proposed downvote button? I mean using the number of users "passing" a given request to calculate its ranking.

The goal of the "relevance score" is to progressively relegate the requests that are too technical, or incorrect, or in the wrong language, or difficult to match exactly, while promoting the requests that have the most chance of success.

While I understand the benefits, I am afraid this might have the effect of relegating requests that are difficult to answer yet relevant. Some requests might require a high level in the language (both as literacy level and as your capacity to come up with good example sentences). If you are being showed easy ones all the time, why would you care about the difficult ones lower in the list? That being said, the current system, by showing unanswered requests first, is the opposite of that as it "promotes" difficult requests, which can turn down many users.

[LBeaudoux]

I am afraid this might have the effect of relegating requests that are difficult to answer yet relevant.

"pass" means "I am just not inspired by this word" whereas "downvote" means "this request is irrelevant here". Using the "pass" button for ranking would amplify the relegation of difficult yet relevant vocabulary requests. The "downvote" button is better for driving bad queries away from most users while "pass" is meaningful at a personal level. Implementing both of them seems to be a more comprehensive solution.

But in my opinion, the number one priority would be to increase the renewal at the top by favoring by default the latest requests at the expense of the least well served. Ultimately, all the searches that get less than 10 results need more examples anyway.

[ckjpn]

Related Wall post by Seael.

https://tatoeba.org/en/wall/show_message/40051#message_40051