search

umputun / remark42

comment engine
https://remark42.com
MIT License
4.92k stars 386 forks source link

Add support of duplicates prevention #566

Open umputun opened 4 years ago

umputun commented 4 years ago

The idea is to add an api and both backend and frontend support notifying the user if a similar comment already posted to the thread. This thing should be optional.

Probably on the backend side, we may have a new method similar to createCommentCtrl but checking for possible dups.

paskal commented 4 years ago

In my opinion, it's too specific for a particular use-case and I am against increasing the complexity of the whole system for everyone because of a single customer, even the biggest one.

Reddit solves such tasks by the community, I think we are not that special and can rely on the voting system as well.

umputun commented 4 years ago

well, github is trying smth similar as well. Generally, it is not an easy task and I don't think this is a high priority, but I do see a value in this. Regardless of the use cases in a long discussion, people tend to post the same links and if we can notify them (not reject, just notify and allow ok/cancel) it could be helpful

paskal commented 4 years ago

I see the complexity and predictable amount of errors around trying to understand if the links are the same, but it might indeed be useful.

sleshJdev commented 4 years ago

Seems that boltdb doesn't provide any built-in support for any more or less strong ad-hoc text processing. My vision is that we need some additional storage which has this functionality to replicate data from boltdb into that storage and run ad-hoc text analysis there. It also could be used for full-text search in the future #734

Another less popular option is to implement it in go: pull the related comment from boltdb into memory and perform text processing. It could work if the amount of data is not big, but in general, it is not reliable at all. To speed it a bit up we could pre-process comments and keep k-grams in boltdb and use them later. But it seems like inventing the wheel.

@umputun, what do you think?