Gemini API : The recitation problem is annoying

[ImadSaddik]

When attempting to use the Gemini API in Android, I encountered the error Content generation stopped. Reason: RECITATION. This error is particularly frustrating as it occurs randomly without any apparent cause.

The most recent occurrence of this issue was when I raised #249, regarding streaming not functioning properly in Android. David Motson provided temporary assistance in resolving this problem. I used a prompt to test if the streaming would work - Give me a Java program that uses Jsoup -. Initially, I was successfully receiving chunks of data, but then suddenly encountered the RECITATION problem.

Below are the logs:

2024-02-11 08:15:05.409 13579-13741 GeminiPro               com.example.streaminggeminiapp       D  onNext: .nodes.Document;
                                                                                                    import org.jsoup.nodes.Element;
                                                                                                    import org.jsoup.select.Elements;

                                                                                                    public class JsoupExample {
2024-02-11 08:15:06.284 13579-13741 GeminiPro               com.example.streaminggeminiapp       D  onNext: 

                                                                                                        public static void main(String[] args) {
                                                                                                            // Parse HTML from a URL
                                                                                                            String url = "https://www.example.com";
                                                                                                            Document doc = Jsoup.connect(url).get();

                                                                                                            // Get the title of the page
                                                                                                            String title = doc.
2024-02-11 08:15:08.039 13579-13741 GeminiPro               com.example.streaminggeminiapp       D  onError: Content generation stopped. Reason: RECITATION
--------- beginning of crash
2024-02-11 08:15:08.055 13579-13741 AndroidRuntime          com.example.streaminggeminiapp       E  FATAL EXCEPTION: DefaultDispatcher-worker-2
                                                                                                    Process: com.example.streaminggeminiapp, PID: 13579
                                                                                                    com.google.ai.client.generativeai.type.ResponseStoppedException: Content generation stopped. Reason: RECITATION
                                                                                                        at com.google.ai.client.generativeai.GenerativeModel.validate(GenerativeModel.kt:182)
                                                                                                        at com.google.ai.client.generativeai.GenerativeModel.access$validate(GenerativeModel.kt:49)
                                                                                                        at com.google.ai.client.generativeai.GenerativeModel$generateContentStream$$inlined$map$1$2.emit(Emitters.kt:224)
                                                                                                        at kotlinx.coroutines.flow.FlowKt__ChannelsKt.emitAllImpl$FlowKt__ChannelsKt(Channels.kt:37)
                                                                                                        at kotlinx.coroutines.flow.FlowKt__ChannelsKt.access$emitAllImpl$FlowKt__ChannelsKt(Channels.kt:1)
                                                                                                        at kotlinx.coroutines.flow.FlowKt__ChannelsKt$emitAllImpl$1.invokeSuspend(Unknown Source:14)
                                                                                                        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
                                                                                                        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
                                                                                                        at kotlinx.coroutines.EventLoop.processUnconfinedEvent(EventLoop.common.kt:68)
                                                                                                        at kotlinx.coroutines.internal.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:347)
                                                                                                        at io.ktor.utils.io.internal.CancellableReusableContinuation.resumeWith(CancellableReusableContinuation.kt:93)
                                                                                                        at io.ktor.utils.io.ByteBufferChannel.resumeReadOp(ByteBufferChannel.kt:2098)
                                                                                                        at io.ktor.utils.io.ByteBufferChannel.flushImpl(ByteBufferChannel.kt:186)
                                                                                                        at io.ktor.utils.io.ByteBufferChannel.flush(ByteBufferChannel.kt:196)
                                                                                                        at io.ktor.utils.io.ByteBufferChannel.copyDirect$ktor_io(ByteBufferChannel.kt:1262)
                                                                                                        at io.ktor.utils.io.ByteBufferChannel$copyDirect$1.invokeSuspend(Unknown Source:18)
                                                                                                        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
                                                                                                        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
                                                                                                        at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115)
                                                                                                        at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103)
                                                                                                        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
                                                                                                        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
                                                                                                        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
                                                                                                        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)
                                                                                                        Suppressed: java.lang.NullPointerException: Can't toast on a thread that has not called Looper.prepare()
                                                                                                            at com.android.internal.util.Preconditions.checkNotNull(Preconditions.java:167)
                                                                                                            at android.widget.Toast.getLooper(Toast.java:212)
                                                                                                            at android.widget.Toast.<init>(Toast.java:197)
                                                                                                            at android.widget.Toast.makeText(Toast.java:583)
                                                                                                            at android.widget.Toast.makeText(Toast.java:570)
                                                                                                            at android.widget.Toast.makeText(Toast.java:544)
                                                                                                            at com.example.streaminggeminiapp.MainActivity$1.onError(MainActivity.java:57)
                                                                                                            at com.example.streaminggeminiapp.GeminiPro$1.onError(GeminiPro.java:56)
                                                                                                            at kotlinx.coroutines.reactive.FlowSubscription.flowProcessing(ReactiveFlow.kt:215)
                                                                                                            at kotlinx.coroutines.reactive.FlowSubscription.access$flowProcessing(ReactiveFlow.kt:187)
                                                                                                            at kotlinx.coroutines.reactive.FlowSubscription$flowProcessing$1.invokeSuspend(Unknown Source:14)
                                                                                                            at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
                                                                                                            at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
                                                                                                            ... 16 more
                                                                                                        Suppressed: kotlinx.coroutines.internal.DiagnosticCoroutineContextException: [FlowSubscription{Active}@c97cec2, Dispatchers.Unconfined]

[hsuperman]

I'm getting this as well

[natapol2547]

Me too. I am getting RECITATION error while implementing RAG with gemini api. I am guessing that the generation will stop if the system detects that the model is repeating the prompt. In my case the model basically repeats part of the retrieved documents and the generation stops. I might be totally wrong about the cause though. In any case, it would be wonderful if the api has a parameter to turn off recitation checking.

[davidmotson]

The recitation stop reason is used whenever the model begins to recite training data, especially copyrighted material.

For example, if you ask the model to tell you the first chapter of harry potter you will run into this error.

By its nature, when it pops up is difficult to predict, since its based on model output more than prompt

[rakotomandimby]

I also face this problem, reported here: https://www.googlecloudcommunity.com/gc/AI-ML/Gemini-API-finishReason-RECITATION/m-p/709738 (still held for moderation)

[rakotomandimby]

If I prompt Gemini via the API with this, I hit the RECITATION issue:

There is a database that can be reached at the following address:
host: localhost
username: root
password: root
database: test
Write a PHP function that connects to that database and create a table called "users" with the following fields:
id (int, auto increment, primary key)
username (varchar, 255)
password (varchar, 255)
email (varchar, 255)

But if I paste it in the Gemini website, I get my answer : https://gemini.google.com/app/11a4c6b2bdcf6b7f

I think this is unfair...

[hsuperman]

Not sure how this is being considered as closed. To be clear, it happens often when not requesting training data, or anything remotely resembling copyrighted or training data. In my case, I'm analyzing and classifying text.

[marlinspike]

I'm seeing this sporadically as well.

[JeevansSP]

Was using vision api to do some ocr on some basic guideline documents (publically available) and I would randomly get this error too on pages like preface, etc.

[kevsjh]

i'm getting this issue as well when implementing RAG with gemini-pro. palm2 (chat-bison) seems to work well without such issue

[naourass]

Have you guys found any workaround for this?

[BeibinLi]

From the documentation, I can tell the RECITATION error occurs when the model detects that it is repeating parts of the prompt or the training documents.

However, I am still not sure why I would encounter this problem in many of my calls.

It is a significant problem in many areas. Here are some of my observations and hypotheses, including (of course, not limited to):

1. RAG: when the retrieved document is unfortunately in the training data of Gemini.
2. RAG: when the answer is very similar to the extracted content (from the retrieved document) in the prompt.
3. Multiple agents (maybe): for instance, "Reply TERMINATE when done" might repeat many times for different agents, which may cause an error.
4. Writing code: for instance, debugging code may involve writing similar code over and over again and can cause this issue.
5. Reproduction: I can run my notebook (for tutorial purposes) without a problem this week, but next week I may encounter RECITATION error (maybe my API calls were added to the training data of Gemini, not sure).

Yeah, just my hypotheses, because I really do not know the root cause of this error.

[omarnahdi]

I am also performing RAG and using load_summarize_chain function from langchain and when I passed the chain_type="map_reduce" it worked but when I didn't or pass something else it just throws the same damn error. Any solutions for this?

[emreaiservices]

For my case I found a workaround for it to work but I did not test it throughly. Basically I added a "try except" block and a while loop on top of it. If it throws recitation error it calls the gemini model again with the same prompt and tries again until the model actually returns a response.

NOTE: writer() is a function that returns the gemini model.

def run_without_recitation(): convo = writer().start_chat(history = []) get_out = 1 while(get_out): try: payload = "You should write approximately 500 words. Topic is:" "only return the output without further explanation" convo.send_message(payload) get_out = 0 except: pass return convo.last.text

def throws_recitation(): convo = writer().start_chat(history = []) payload = "You should write approximately 500 words. Topic is:" "only return the output without further explanation" convo.send_message(payload) return convo.last.text

[simonff]

FYI, there is a new public bug on this topic in a more central place: https://issuetracker.google.com/issues/331677495

[Sameera2001Perera]

Not sure how to consider this as closed. Im also getting this issue when implementing RAG with gemini-1.0-pro-001. Increasing the temperature can be a solution.

[simonff]

@Sameera2001Perera : could you please comment on https://issuetracker.google.com/issues/331677495 instead? That is an active bug.

[itsaaiml]

I'm also encountering the block reason 4 error when using gemini-1.0-pro-001; setting: BOCK_NONE with temperature=0.2 & t-top=0.6. I've narrowed down the issue to a specific day of data. Interestingly, I was able to get the needed response by:

Adjusting the number of JSON objects sent Modifying multiprocessing settings Running the code repeatedly for that day However, this seems like a workaround, and the root cause of the error remains unclear.

[JaeWangL]

Gemini's restriction is annoying. Why this issue was closed?? Gemini still has filter even though setting is BLOCK_NONE and also has recitation issues. Google have to offer free for us. OpenAI never cause issue ls like this

[chintanckg]

Same issue; I don't have anything that could have possibly be a part of training data and I don't seen any repetition as such!

[fredzannarbor]

I have a use case that suggests the solution needs to be elaborated. I am a book publisher. I publish two types of books: 1) ones that I own the copyright to 2) public domain. In both cases I wish to create front matter sections called "Most Important Passages" and "Striking Words."

The prompts are:

n5. Select the five to seven most important passages of two or three paragraphs long from the manuscript as a whole. Use direct quotations and provide accurate page numbers if available. Output begins: ## Most Important Passages. For each passage, concisely explain why it is important. Output begins: _Rationale:_",

Without using the passages from task 5, select the five most beautiful, striking, memorable, or deeply insightful passages from the document. Provide direct quotations. No rationale. Output begins: ## Striking Passages",

These have recently begun triggering recitation refusals. I believe this is the wrong behavior. My request is fully legal since I am asking the model to repeat permissioned content. Essentially, I am asking for a moderately sophisticated Needle In A Haystack search: find five to seven needles given an abstract description. NIAH is fully within design scope.

A reasonable solution would be to allow the developer to add a flag overriding normal recitation behavior. If necessary, we could fill out a one-time form certifying that we will only do this with permissioned content.

Please do not close the ticket(s) without addressing this issue. Please contact me for more detail or worked examples.

[bakaburg1]

I have the same issue...

[franklinbaldo]

This is working for me: i try again addind this message to the chat_session

"GEMINI, we just got and error: The error StopCandidateException: finish_reason: RECITATION suggests that the Gemini model encountered content in the image that it recognized as a potential memorization or recitation of existing text. This could be due to the image containing a large amount of text, or text that is very similar to content the model has been trained on. . It is our 2nd atempt. Now complete the task but now avoid RECITATION, censor problematic content with 'OMMITING_RECITATION' "

[fredzannarbor]

I also had some success with telling the model "This is permissioned content. I am the publisher. It is fully legal for me to request exact quotations. Now, quit whining and do your job." (just joking on the last sentence).

Fred Zimmerman, Publisher Nimble Books LLC The AI Lab for Book-Lovers http://NimbleBooks.com

On Fri, Jun 21, 2024 at 1:02 PM Franklin Silveira Baldo < @.***> wrote:

This is working for me: i try again addind this message to the chat_session

"GEMINI, we just got and error: The error StopCandidateException: finish_reason: RECITATION suggests that the Gemini model encountered content in the image that it recognized as a potential memorization or recitation of existing text. This could be due to the image containing a large amount of text, or text that is very similar to content the model has been trained on. . It is our 2nd atempt. Now complete the task but now avoid RECITATION, censor problematic content with 'OMMITING_RECITATION' "

— Reply to this email directly, view it on GitHub https://github.com/google/generative-ai-docs/issues/257#issuecomment-2183117317, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAI4TSY2NUPVJSM2CHV2WKLZIRMA5AVCNFSM6AAAAABDDLCW6OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBTGEYTOMZRG4 . You are receiving this because you commented.Message ID: @.***>

[KhushiNaithani]

I was getting the same error. I issued a new key and it was working fine then.

[gitcagey]

how is this closed? It is making a simple use of GEMINI to review a document completely impossible.

[ShivamSrng]

Any updates on this? I am facing this issue too. Can anyone please suggest some way to resolve it?

[ImadSaddik]

Sadly, they didn't clarify nor solve this issue.

[ShivamSrng]

Sadly, they didn't clarify nor solve this issue.

That's something irritating, like I want to use this Gemini API in the Google Gemini API Competition and this error keeps popping on some data every time. I can't keep out a project with this RECITATION error in the competition.

[keropudding]

+1 super annoying error -- running into this sporadically also.

[altbdoor]

This is getting a litle absurd. gemini-1.5-flash-exp-0827, BLOCK_NONE on all safety settings.

> hello gemini. what are the 12 star signs?

< Okay, here are the 12 signs of the zodiac:
< 
< 1. Aries (March 21 - April 19)
< 1. Taurus (April 20 - May 20)
< 1. Gemini (May 21 - June 20)
< 1. Cancer (June 21 - July 22)
< 1. Leo (July 23 - August 22
< Error: [GoogleGenerativeAI Error]: Candidate was blocked due to RECITATION

[naourass]

I was checking the public issue tracker for this one at least once each week hoping for it to be fixed or at least updated. Today to my surprise I've found that the issue was closed as "Won't Fix (Obsolete)". Obsolete? Can't believe that!

[ImadSaddik]

Yeah, it is so sad that they did not provide a convincing answer to this issue.

[gitcagey]

I think it has been addressed. Lots of issues logged on it, but this one has a response from google. I've not encountered but maybe 1 time in our production app in last few weeks, and upon a retry it worked fine.

https://github.com/google-gemini/generative-ai-js/issues/138

[ImadSaddik]

Glad to hear that. I myself did not test it extensively. I need to do that to confirm that it is now fine.

[iliane5]

It's better now but the recitation filter is still fundamentally broken and will randomly block requests in an unpredictable manner. Just ran into another case a minute ago while doing some testing.

[fredzannarbor]

Is there any deterministic way to confirm that a bug like this is "fixed"? You can discourage the behavior by modifying the reward function & training, but if you allow any repetition of permissioned content, you open yourself up to a lot of different problems.

Fred Zimmerman, Publisher Nimble Books LLC The AI Lab for Book-Lovers http://NimbleBooks.com

On Sat, Sep 28, 2024 at 5:56 PM Iliane @.***> wrote:

It's better now but the recitation filter is still fundamentally broken and will randomly block requests in an unpredictable manner. Just ran into another case a minute ago while doing some testing.

— Reply to this email directly, view it on GitHub https://github.com/google/generative-ai-docs/issues/257#issuecomment-2381004585, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAI4TS3SD2QZZD4UQIOGSJLZY4QYJAVCNFSM6AAAAABDDLCW6OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBRGAYDINJYGU . You are receiving this because you commented.Message ID: @.***>

[naourass]

I just did some more or less extended tests. It doesn't look like the recitation has been fixed. I'm still getting tons of recitation errors when asking the model to extract data from my input. All the requested output is already in the input (law texts and attachments), so nothing with copyright or trademark.

[anishjain123]

Anybody have any update on this? This is a terrible bug

[sergeyk]

Why is this issue closed? This is a way overzealous filter and not really acceptable. I'd like to switch from gpt-4o-mini to gemini-1.5-flash for a web page summarization use case, but cannot due to this bug.

[adboio]

+1, i am using gemini (currently 1.5 flash) to summarize web page content, getting this error

[franklinbaldo]

I've conducted some experiments with the Gemini API and discovered that switching to direct HTTP requests can bypass the "RECITATION" error. When using the HTTP approach, the model outputs the content as expected, whereas the issue persists with the generative library.

You can simply switch to HTTP instead of the library. Here’s an example in Python:

import requests

url = "https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash:generateContent"
api_key = "YOUR_GOOGLE_API_KEY"

headers = {
    "Content-Type": "application/json"
}

data = {
    "system_instruction": {
        "parts": {
            "text": "You are a cat. Your name is Neko."
        }
    },
    "contents": {
        "parts": {
            "text": "Hello there"
        }
    }
}

response = requests.post(f"{url}?key={api_key}", headers=headers, json=data)
print(response.json())

Hope this helps!

[ImadSaddik]

Thank you, @franklinbaldo, for the info. It's great to hear that using HTTP requests can help bypass the Recitation error. If that's working, they should definitely consider fixing the library as well.

I raised this issue while using the JAVA client, but I wonder if people face the Recitation error in the other languages too.

[fredzannarbor]

I wonder if it’s possible that some of the prompts submitted via the generative library are malformed in a way that produces recitations, either by the client library or by (gah) the submitting program.

On Oct 10, 2024, at 3:15 PM, Franklin Silveira Baldo @.***> wrote:

I've conducted some experiments with the Gemini API and discovered that switching to direct HTTP requests can bypass the "RECITATION" error. When using the HTTP approach, the model outputs the content as expected, whereas the issue persists with the generative library.

You can simply switch to HTTP instead of the library. Here’s an example in Python:

import requests

url = "https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash:generateContent" api_key = "YOUR_GOOGLE_API_KEY"

headers = { "Content-Type": "application/json" }

data = { "system_instruction": { "parts": { "text": "You are a cat. Your name is Neko." } }, "contents": { "parts": { "text": "Hello there" } } }

response = requests.post(f"{url}?key={api_key}", headers=headers, json=data) print(response.json()) Hope this helps!

— Reply to this email directly, view it on GitHub https://github.com/google/generative-ai-docs/issues/257#issuecomment-2405857855, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAI4TSYB3W5ZKQJZ662JDFDZ23G5JAVCNFSM6AAAAABDDLCW6OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMBVHA2TOOBVGU. You are receiving this because you commented.

[iliane5]

@franklinbaldo I've tried using the REST API directly, but unfortunately I'm still getting a recitation error - on the specific example I was testing with at least. I looked through the source of the JS SDK quickly but there doesn't seem to be any weird stuff on that side.

@ImadSaddik:

I raised this issue while using the JAVA client, but I wonder if people face the Recitation error in the other languages too.

Can confirm I've had the issue from the javascript and python libraries as well as with direct HTTP requests. It's definitely coming from the API itself.

There isn't a ton of back and forth from Google here nor on the other issue trackers but I've talked with @logankilpatrick and Mat Veloso about it and they're definitely aware & working on it. A lot of people would really appreciate it if we could get a definitive fix though!

---

For what it's worth: if you set "candidate_count" to n>1, there is - in my experience - a good chance that one of the outputs won't have triggered the recitation filter.

Kinda hate this "fix" but I guess if you look at it from a glass half-full perspective, gemini flash is very cheap & you get to have best of (n-blocked candidates) sampling :)

[ImadSaddik]

Thank you, @iliane5, for confirming that the problem occurs with the other languages too. Hopefully, we will get a fix one day, and thank you for the tip.

[sswam]

For what it's worth: if you set "candidate_count" to n>1, there is - in my experience - a good chance that one of the outputs won't have triggered the recitation filter.

That's probably why the web app doesn't trigger the RECITATION error very often, it always generates three candidates. (And often selects a worse one, but that's a different issue.)

Kinda hate this "fix" but I guess if you look at it from a glass half-full perspective, gemini flash is very cheap & you get to have best of (n-blocked candidates) sampling :)

To reduce costs, you can opt to retry upon encountering an error, rather than paying for multiple candidates upfront. This will increase total processing time in the case that there is a recitation error.

[njoppi2]

Hi, I had this same problem, almost half of my request were returning the 'RECITATION' finish reason. I noticed I was using the model "gemini-1.5-flash", and just switching to the model "gemini-1.5-flash-002" seems to have solved the problem.

Hope that helps other people!! :)

[naourass]

Just tested both gemini-1.5-flash-002 and gemini-1.5-pro-002, unfortunately still getting recitation error (finish_reason: RECITATION) for the same prompts that have nothing to do with copyrighted material, just asking to extract data from my own provided input.

[adboio]

sharing an approach that’s working for me:

i’m using gemini to summarize web page content. if i pass an HTML file, or the HTML file contents as a string, i often see the RECITATION error.

however, if i extract the text of the HTML page (removing all tags, head, etc) and pass that to gemini, i no longer see the issue

[natapol2547]

Here's an idea. I haven't have time to test it with RAG Engine API on Google Cloud. Maybe this could eliminate the Recitation error as information is passed as Function Response instead.