kol3x
commented
3 months ago I finished and tested the whole thing. These are the end results for different text inputs:
A little more on the implementation:
-
GitHub Actions workflow finds the first .md file in the PR and sends it to a Cloudflare worker. Then it formats the result to a proper markdown for a GitHub PR comment. Lastly, it posts the results comment. There is one little hack I used here - for line breaks I literally pressed "Enter" in the .yml file, because neither \n or < b r > did the job. (\n didn't do anything at all, and < b r > was creating a new line visually, but was still treating the whole comment as a single line, so for example if I had triple hash for my first line, everything in the comment got treated like a header)
-
Cloudflare worker checks the text one sentence at a time (Anything more than 10 characters and ending with a dot is considered a sentence) Now, this is not a super advanced way for checking plagiarism, but since Evgeny pointed out that we are looking for the simple copy-pasting of text from the open web - it does the job.
I am gonna work on a README later with instructions on setting up the service and adding all necessary secrets
Looking forward to hear some feedback on my implementation!
Closes #424
Hi, I am starting this as draft request for now, because I still need to test the GitHub workflow script properly. I've never worked with one before, so it's ruff. The one I have now is work in progress.
However, the plagiarism-checker service is ready and working. Since Evgeny gave me a hint that this problem has a better solution than just using a paid third-party API, I started researching on how I might implement it otherwise. One old open-source library drew my attention, as it was using google search API to check text for plagiarism in the open web. I found that method better for given problem, but I didn't want to use a library that hasn't been updated for 10+ years, so I developed a little service that uses the same technic.
Additional costs with this method should be 0, since Google lets you make up to 100 queries a day for free. My service is implemented in a way, that it checks every sentence for a 100% match on the internet, while Crypto Attack Wiki article has less than 30 sentences on the average (Rough estimate). And since new articles are released not that often, we can have 2-3 free checks for a new article in a single day.