Gemini API : The recitation problem is annoying

[ImadSaddik]

When attempting to use the Gemini API in Android, I encountered the error Content generation stopped. Reason: RECITATION. This error is particularly frustrating as it occurs randomly without any apparent cause.

The most recent occurrence of this issue was when I raised #249, regarding streaming not functioning properly in Android. David Motson provided temporary assistance in resolving this problem. I used a prompt to test if the streaming would work - Give me a Java program that uses Jsoup -. Initially, I was successfully receiving chunks of data, but then suddenly encountered the RECITATION problem.

Below are the logs:

2024-02-11 08:15:05.409 13579-13741 GeminiPro               com.example.streaminggeminiapp       D  onNext: .nodes.Document;
                                                                                                    import org.jsoup.nodes.Element;
                                                                                                    import org.jsoup.select.Elements;

                                                                                                    public class JsoupExample {
2024-02-11 08:15:06.284 13579-13741 GeminiPro               com.example.streaminggeminiapp       D  onNext: 

                                                                                                        public static void main(String[] args) {
                                                                                                            // Parse HTML from a URL
                                                                                                            String url = "https://www.example.com";
                                                                                                            Document doc = Jsoup.connect(url).get();

                                                                                                            // Get the title of the page
                                                                                                            String title = doc.
2024-02-11 08:15:08.039 13579-13741 GeminiPro               com.example.streaminggeminiapp       D  onError: Content generation stopped. Reason: RECITATION
--------- beginning of crash
2024-02-11 08:15:08.055 13579-13741 AndroidRuntime          com.example.streaminggeminiapp       E  FATAL EXCEPTION: DefaultDispatcher-worker-2
                                                                                                    Process: com.example.streaminggeminiapp, PID: 13579
                                                                                                    com.google.ai.client.generativeai.type.ResponseStoppedException: Content generation stopped. Reason: RECITATION
                                                                                                        at com.google.ai.client.generativeai.GenerativeModel.validate(GenerativeModel.kt:182)
                                                                                                        at com.google.ai.client.generativeai.GenerativeModel.access$validate(GenerativeModel.kt:49)
                                                                                                        at com.google.ai.client.generativeai.GenerativeModel$generateContentStream$$inlined$map$1$2.emit(Emitters.kt:224)
                                                                                                        at kotlinx.coroutines.flow.FlowKt__ChannelsKt.emitAllImpl$FlowKt__ChannelsKt(Channels.kt:37)
                                                                                                        at kotlinx.coroutines.flow.FlowKt__ChannelsKt.access$emitAllImpl$FlowKt__ChannelsKt(Channels.kt:1)
                                                                                                        at kotlinx.coroutines.flow.FlowKt__ChannelsKt$emitAllImpl$1.invokeSuspend(Unknown Source:14)
                                                                                                        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
                                                                                                        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
                                                                                                        at kotlinx.coroutines.EventLoop.processUnconfinedEvent(EventLoop.common.kt:68)
                                                                                                        at kotlinx.coroutines.internal.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:347)
                                                                                                        at io.ktor.utils.io.internal.CancellableReusableContinuation.resumeWith(CancellableReusableContinuation.kt:93)
                                                                                                        at io.ktor.utils.io.ByteBufferChannel.resumeReadOp(ByteBufferChannel.kt:2098)
                                                                                                        at io.ktor.utils.io.ByteBufferChannel.flushImpl(ByteBufferChannel.kt:186)
                                                                                                        at io.ktor.utils.io.ByteBufferChannel.flush(ByteBufferChannel.kt:196)
                                                                                                        at io.ktor.utils.io.ByteBufferChannel.copyDirect$ktor_io(ByteBufferChannel.kt:1262)
                                                                                                        at io.ktor.utils.io.ByteBufferChannel$copyDirect$1.invokeSuspend(Unknown Source:18)
                                                                                                        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
                                                                                                        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
                                                                                                        at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115)
                                                                                                        at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103)
                                                                                                        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
                                                                                                        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
                                                                                                        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
                                                                                                        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)
                                                                                                        Suppressed: java.lang.NullPointerException: Can't toast on a thread that has not called Looper.prepare()
                                                                                                            at com.android.internal.util.Preconditions.checkNotNull(Preconditions.java:167)
                                                                                                            at android.widget.Toast.getLooper(Toast.java:212)
                                                                                                            at android.widget.Toast.<init>(Toast.java:197)
                                                                                                            at android.widget.Toast.makeText(Toast.java:583)
                                                                                                            at android.widget.Toast.makeText(Toast.java:570)
                                                                                                            at android.widget.Toast.makeText(Toast.java:544)
                                                                                                            at com.example.streaminggeminiapp.MainActivity$1.onError(MainActivity.java:57)
                                                                                                            at com.example.streaminggeminiapp.GeminiPro$1.onError(GeminiPro.java:56)
                                                                                                            at kotlinx.coroutines.reactive.FlowSubscription.flowProcessing(ReactiveFlow.kt:215)
                                                                                                            at kotlinx.coroutines.reactive.FlowSubscription.access$flowProcessing(ReactiveFlow.kt:187)
                                                                                                            at kotlinx.coroutines.reactive.FlowSubscription$flowProcessing$1.invokeSuspend(Unknown Source:14)
                                                                                                            at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
                                                                                                            at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
                                                                                                            ... 16 more
                                                                                                        Suppressed: kotlinx.coroutines.internal.DiagnosticCoroutineContextException: [FlowSubscription{Active}@c97cec2, Dispatchers.Unconfined]

[hsuperman]

I'm getting this as well

[natapol2547]

Me too. I am getting RECITATION error while implementing RAG with gemini api. I am guessing that the generation will stop if the system detects that the model is repeating the prompt. In my case the model basically repeats part of the retrieved documents and the generation stops. I might be totally wrong about the cause though. In any case, it would be wonderful if the api has a parameter to turn off recitation checking.

[davidmotson]

The recitation stop reason is used whenever the model begins to recite training data, especially copyrighted material.

For example, if you ask the model to tell you the first chapter of harry potter you will run into this error.

By its nature, when it pops up is difficult to predict, since its based on model output more than prompt

[rakotomandimby]

I also face this problem, reported here: https://www.googlecloudcommunity.com/gc/AI-ML/Gemini-API-finishReason-RECITATION/m-p/709738 (still held for moderation)

[rakotomandimby]

If I prompt Gemini via the API with this, I hit the RECITATION issue:

There is a database that can be reached at the following address:
host: localhost
username: root
password: root
database: test
Write a PHP function that connects to that database and create a table called "users" with the following fields:
id (int, auto increment, primary key)
username (varchar, 255)
password (varchar, 255)
email (varchar, 255)

But if I paste it in the Gemini website, I get my answer : https://gemini.google.com/app/11a4c6b2bdcf6b7f

I think this is unfair...

[hsuperman]

Not sure how this is being considered as closed. To be clear, it happens often when not requesting training data, or anything remotely resembling copyrighted or training data. In my case, I'm analyzing and classifying text.

[marlinspike]

I'm seeing this sporadically as well.

[JeevansSP]

Was using vision api to do some ocr on some basic guideline documents (publically available) and I would randomly get this error too on pages like preface, etc.

[kevsjh]

i'm getting this issue as well when implementing RAG with gemini-pro. palm2 (chat-bison) seems to work well without such issue

[naourass]

Have you guys found any workaround for this?

[BeibinLi]

From the documentation, I can tell the RECITATION error occurs when the model detects that it is repeating parts of the prompt or the training documents.

However, I am still not sure why I would encounter this problem in many of my calls.

It is a significant problem in many areas. Here are some of my observations and hypotheses, including (of course, not limited to):

1. RAG: when the retrieved document is unfortunately in the training data of Gemini.
2. RAG: when the answer is very similar to the extracted content (from the retrieved document) in the prompt.
3. Multiple agents (maybe): for instance, "Reply TERMINATE when done" might repeat many times for different agents, which may cause an error.
4. Writing code: for instance, debugging code may involve writing similar code over and over again and can cause this issue.
5. Reproduction: I can run my notebook (for tutorial purposes) without a problem this week, but next week I may encounter RECITATION error (maybe my API calls were added to the training data of Gemini, not sure).

Yeah, just my hypotheses, because I really do not know the root cause of this error.

[omarnahdi]

I am also performing RAG and using load_summarize_chain function from langchain and when I passed the chain_type="map_reduce" it worked but when I didn't or pass something else it just throws the same damn error. Any solutions for this?

[emreaiservices]

For my case I found a workaround for it to work but I did not test it throughly. Basically I added a "try except" block and a while loop on top of it. If it throws recitation error it calls the gemini model again with the same prompt and tries again until the model actually returns a response.

NOTE: writer() is a function that returns the gemini model.

def run_without_recitation(): convo = writer().start_chat(history = []) get_out = 1 while(get_out): try: payload = "You should write approximately 500 words. Topic is:" "only return the output without further explanation" convo.send_message(payload) get_out = 0 except: pass return convo.last.text

def throws_recitation(): convo = writer().start_chat(history = []) payload = "You should write approximately 500 words. Topic is:" "only return the output without further explanation" convo.send_message(payload) return convo.last.text

[simonff]

FYI, there is a new public bug on this topic in a more central place: https://issuetracker.google.com/issues/331677495

[Sameera2001Perera]

Not sure how to consider this as closed. Im also getting this issue when implementing RAG with gemini-1.0-pro-001. Increasing the temperature can be a solution.

[simonff]

@Sameera2001Perera : could you please comment on https://issuetracker.google.com/issues/331677495 instead? That is an active bug.

[itsaaiml]

I'm also encountering the block reason 4 error when using gemini-1.0-pro-001; setting: BOCK_NONE with temperature=0.2 & t-top=0.6. I've narrowed down the issue to a specific day of data. Interestingly, I was able to get the needed response by:

Adjusting the number of JSON objects sent Modifying multiprocessing settings Running the code repeatedly for that day However, this seems like a workaround, and the root cause of the error remains unclear.

[JaeWangL]

Gemini's restriction is annoying. Why this issue was closed?? Gemini still has filter even though setting is BLOCK_NONE and also has recitation issues. Google have to offer free for us. OpenAI never cause issue ls like this

[chintanckg]

Same issue; I don't have anything that could have possibly be a part of training data and I don't seen any repetition as such!

[fredzannarbor]

I have a use case that suggests the solution needs to be elaborated. I am a book publisher. I publish two types of books: 1) ones that I own the copyright to 2) public domain. In both cases I wish to create front matter sections called "Most Important Passages" and "Striking Words."

The prompts are:

n5. Select the five to seven most important passages of two or three paragraphs long from the manuscript as a whole. Use direct quotations and provide accurate page numbers if available. Output begins: ## Most Important Passages. For each passage, concisely explain why it is important. Output begins: _Rationale:_",

Without using the passages from task 5, select the five most beautiful, striking, memorable, or deeply insightful passages from the document. Provide direct quotations. No rationale. Output begins: ## Striking Passages",

These have recently begun triggering recitation refusals. I believe this is the wrong behavior. My request is fully legal since I am asking the model to repeat permissioned content. Essentially, I am asking for a moderately sophisticated Needle In A Haystack search: find five to seven needles given an abstract description. NIAH is fully within design scope.

A reasonable solution would be to allow the developer to add a flag overriding normal recitation behavior. If necessary, we could fill out a one-time form certifying that we will only do this with permissioned content.

Please do not close the ticket(s) without addressing this issue. Please contact me for more detail or worked examples.

[bakaburg1]

I have the same issue...

[franklinbaldo]

This is working for me: i try again addind this message to the chat_session

"GEMINI, we just got and error: The error StopCandidateException: finish_reason: RECITATION suggests that the Gemini model encountered content in the image that it recognized as a potential memorization or recitation of existing text. This could be due to the image containing a large amount of text, or text that is very similar to content the model has been trained on. . It is our 2nd atempt. Now complete the task but now avoid RECITATION, censor problematic content with 'OMMITING_RECITATION' "

[fredzannarbor]

I also had some success with telling the model "This is permissioned content. I am the publisher. It is fully legal for me to request exact quotations. Now, quit whining and do your job." (just joking on the last sentence).

Fred Zimmerman, Publisher Nimble Books LLC The AI Lab for Book-Lovers http://NimbleBooks.com

On Fri, Jun 21, 2024 at 1:02 PM Franklin Silveira Baldo < @.***> wrote:

This is working for me: i try again addind this message to the chat_session

"GEMINI, we just got and error: The error StopCandidateException: finish_reason: RECITATION suggests that the Gemini model encountered content in the image that it recognized as a potential memorization or recitation of existing text. This could be due to the image containing a large amount of text, or text that is very similar to content the model has been trained on. . It is our 2nd atempt. Now complete the task but now avoid RECITATION, censor problematic content with 'OMMITING_RECITATION' "

— Reply to this email directly, view it on GitHub https://github.com/google/generative-ai-docs/issues/257#issuecomment-2183117317, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAI4TSY2NUPVJSM2CHV2WKLZIRMA5AVCNFSM6AAAAABDDLCW6OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBTGEYTOMZRG4 . You are receiving this because you commented.Message ID: @.***>

[KhushiNaithani]

I was getting the same error. I issued a new key and it was working fine then.

[gitcagey]

how is this closed? It is making a simple use of GEMINI to review a document completely impossible.