Research Article re-writer

[gpseal]

We are in need of a good option to re-write our articles before adding them to the database.

I have started looking at an option for re-writing the articles and found this one to be the most popular on hugging face using the search term "paraphraser"

https://huggingface.co/prithivida/parrot_paraphraser_on_T5

It is a couple of years old but seems to provide what we are looking for and seems quite easy to implement. I have start testing it on a branch called 'gseal-parrot-experiment' and have it working in a bare bones sort of way.

Of note, it requires an old version of python to install so works if VS code is opened via the anaconda navigator.

Another one I was looking to try is here: https://huggingface.co/humarin/chatgpt_paraphraser_on_T5_base

This one looks quite new but employs chagpt by the look of things so may provide advantages.

[gpseal]

David has suggested LLama as a possible alternative for this https://ai.facebook.com/blog/large-language-model-llama-meta-ai/

After looking into it, Llama seems to be available only if you are appropriately qualified:

https://huggingface.co/docs/transformers/main/en/model_doc/llama takes me to: https://docs.google.com/forms/d/e/1FAIpQLSfqNECQnMkycAp2jP4Z9TFX0cGR4uf7b_fBxjY_OjhJILlKGA/viewform

This form requires a short submission that includes a compulsory "previous related publications" field. Perhaps David may have something to help us to apply for its usage?

[gpseal]

I have tried running https://huggingface.co/humarin/chatgpt_paraphraser_on_T5_base on my home PC but ran into problems with "cuda". I installed it but didn't get anywhere. I am not familiar so not really sure where to go from here with that. It took up a painful 15 gig of disk space on my home pc, so I have removed, will try on campus.

[gpseal]

I briefly played with the chat GPT api and have it working in VS code (pretty easy to set up compared to the hugging face resources). It basically does everything we need, at an extremely high level, but will require $$. It will be a last resort option if we can't find anything open source that is comparable. Hugging face transformers seem to re-write the articles, sentence by sentence, where as chat gpt seems to have a better understanding of the article contents, which it can then re produce in its own words. Seems to be the best, easiest option at this stage, only issue is the funding required....

[gpseal]

I successfully used parrot paraphraser on campus, with questionable results. The results were incredibly inconsistent, with the exact phrase being given as a result on multiple occasions, and a habit of crashing halfway through the process. Not a great start really but may continue to look at it if we cannot find a better alternative.

I also successfully installed and used this one: https://huggingface.co/ramsrigouthamg/t5_paraphraser

This is another popular hugging face option, but unfortunately produced results even less impressive than parrot. It was faster, and always completed its output, but ended up producing nonsense, often making little to no sense in the context of the article, Not ideal

[gpseal]

PEGASUS PARAPHRASER

Had some positive results with a couple of transformers: https://huggingface.co/tuner007/pegasus_paraphrase Was able to complete an entire article without a problem. Only problem is it can only re-write one sentence at a time, which is not ideal for cohesive writing, as sentences can sometimes not relate to the previous line.

I have experimented with the "beams" settings and conclude that the higher the number, the slower the process, but I am unsure if the effect it is having warrants the extra processing power it requires, see samples below:

PEGASUS PARAPHRASER

ORIGINAL: "But Mr Putin said the plan could be put forward only \"when they are ready for it in the West and in Kyiv\". ", "The Russian leader met Chinese President Xi Jinping on Tuesday in Moscow to discuss the conflict, and relations between the two countries.", "China's plan, published last month, does not explicitly call for Russia to leave Ukraine.", "Listing ", ", it calls for peace talks and respect for national sovereignty, without specific proposals.", "But Ukraine has insisted on Russia withdrawing from its territory as a condition for any talks - and there is no sign that Russia is ready to do that.", "The US Secretary of State Antony Blinken said on Monday that calling for a ceasefire before Russia withdrew \"would effectively be supporting the ratification of Russian conquest\".", "In a joint news conference after talks with Mr Xi ended, Mr Putin said: \"Many provisions of the Chinese peace plan can be taken as the basis for settling of the conflict in Ukraine, whenever the West and Kyiv are ready for it.\"", "But Russia had yet to see such \"readiness\" from the other side, he added.", "Standing alongside the Russian leader, Mr Xi said his government was in favour of peace and dialogue and that China was on the \"right side of history\". ", "He again claimed that China had an \"impartial position\" on the conflict in Ukraine, seeking to cast Beijing as the potential peace-maker. ", "The pair also discussed growing trade, energy and political ties between the two nations. ", "\"China is the leading foreign trade partner of Russia,\" President Putin said, pledging to keep up and surpass the \"high level\" of trade achieved last year. ", "According to Russian state media, the two leaders also:"

BEAMS = 100 When they are ready for it in the West, the plan can be put forward, according to Mr Putin. The Russian leader met the Chinese president in Moscow to discuss relations between the two countries. China's plan doesn't explicitly call for Russia to leave. There is a listing. It calls for peace talks and respect for national sovereignty. There is no sign that Russia is willing to withdraw from its territory as a condition for talks. The US Secretary of State said on Monday that calling for a ceasefire before Russia withdrew would be supporting the Russian conquest. "Many provisions of the Chinese peace plan can be taken as the basis for settling of the conflict in Ukraine whenever the West and Kyiv are ready for it," Mr Putin said in a joint news conference. He said that Russia had yet to see such "readiness" from the other side. He said that China was on the right side of history and that his government was in favor of peace and dialogue. He claimed that China had an "impartial position" on the conflict in Ukraine, in order to cast Beijing as the potential peace-maker. Growing trade, energy and political ties between the two nations were discussed by the pair. President Putin said that China is the leading foreign trade partner of Russia. The two leaders were also reported by Russian state media.

BEAMS = 5 Mr Putin said the plan could only be put forward when it is ready in the West. On Tuesday, the Russian leader met the Chinese president in Moscow to discuss relations between the two countries. China's plan doesn't explicitly call for Russia to leave. There is a listing. It calls for peace talks and respect for national sovereignty. There is no sign that Russia is willing to withdraw from its territory as a condition for talks. The US Secretary of State said on Monday that calling for a ceasefire before Russia withdrew would be supporting the Russian conquest. "Many provisions of the Chinese peace plan can be taken as the basis for settling of the conflict in Ukraine whenever the West and Kyiv are ready for it," Mr Putin said in a joint news conference. He said that Russia had yet to see such "readiness" from the other side. The Chinese leader said his government was in favor of peace and dialogue and that China was on the right side of history. He claimed that China had an "impartial position" on the conflict in Ukraine, in order to cast Beijing as the potential peace-maker. Growing trade, energy and political ties between the two nations were discussed by the pair. President Putin said that China is the leading foreign trade partner of Russia. The two leaders were also reported by Russian state media.

BEAMS = 1 The plan could only be put forward when the West and the Ukrainians are ready for it, Mr Putin said. The Russian leader met the Chinese president in Moscow to discuss the conflict. China's plan does not explicitly call for Russia to leave. There is a listing. It calls for peace talks and respect for national sovereignty. There is no sign that Russia is ready to withdraw from its territory as a condition for talks. The US Secretary of State Antony Blinken said on Monday that calling for a ceasefire before Russia withdrew would be supporting the Russian conquest. The basis for the conflict in Ukraine can be found in the Chinese peace plan, according to Mr Putin. He said that Russia had yet to see such "readiness" from the other side. Mr. Xi said that China was on the right side of history and that his government was in favor of peace and dialogue. He claimed that China had an "impartial position" on the conflict in Ukraine, and that he wanted to cast Beijing as the potential peace-maker. The pair discussed the growing trade between the two nations. President Putin said that China is the leading foreign trade partner of Russia. The two leaders are also mentioned in the state media.

[gpseal]

humarin/chatgpt_paraphraser_on_T5_base

The second positive reult was from this transformer: https://huggingface.co/humarin/chatgpt_paraphraser_on_T5_base

This one seems to have alot of potential, as I was able to input multpile sentences, though the result was overly condensed into a couple of sentences rather than a few paragraphs. I haven't tried it sentence by sentence yet, but will continue to experiment with this one.

[gpseal]

I have successfully posted re-written articles using both pegasus and chatgpt by adding code into the scraper pipeline. Both resulted in re-written articles, though chatgpt is faster and gives more realistic results as it re-writes the entire article rather than one sentence at a time, which removes the risk of sentences not flowing together properly.

These two seem to produce the best results, the next step is to remove them from the scraper pipelines and create version that collects all necessary data from the database, re-writes it, and posts it to a new table.