Not supporting Farsi Text

[hosseinmirzapur]

As I try to send Farsi (Persian) text to the Gemini API I get this error:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x965cb0]

goroutine 34 [running]:
github.com/google/generative-ai-go/genai.(*ChatSession).addToHistory(...)
        /home/ubuntu/go/pkg/mod/github.com/google/generative-ai-go@v0.7.0/genai/chat.go:63
github.com/google/generative-ai-go/genai.(*GenerateContentResponseIterator).Next(0xc00038eb40)
        /home/ubuntu/go/pkg/mod/github.com/google/generative-ai-go@v0.7.0/genai/client.go:162 +0xd0
github.com/hosseinmirzapur/golangchain/pkg.outputResponse(0xc00038eb40, 0xc00046d3e0)
        /home/ubuntu/hgai/pkg/model.go:135 +0x39
created by github.com/hosseinmirzapur/golangchain/pkg.getModelResponse in goroutine 1
        /home/ubuntu/hgai/pkg/model.go:125 +0x158

I've even printed out the server response and it seems like it doesn't understand the Farsi text unless I put it between quotes by using fmt.Sprintf("%q", text) which then again panics on more than one word. I'm using Gemini for my Telegram Chatbot. I've checked the telegram update message and the problem is not from there and just when I was trying to send raw text to API, this happened.

[eliben]

Could you provide a self-contained code sample that demonstrates your issue?

Does your example of Farsi text work in the web UI of https://aistudio.google.com/ ?

[hosseinmirzapur]

Could you provide a self-contained code sample that demonstrates your issue?

Does your example of Farsi text work in the web UI of https://aistudio.google.com/ ?

As I checked the link you mentioned, It seems like Gemini API does not support Persian yet, although supporting it via the https://gemini.google.com/ Thanks for the answer!

[eliben]

Thanks for checking. Even so, a panic of the sort you described is probably not the best behavior for our SDK. If you can provide a simple reproducer we can try to improve the handling of this case in terms of errors/panics.

[hosseinmirzapur]

Sure. Let me describe the error more in details. I initialize the data which is going to be sent to API like this:

prompt := []genai.Part{
        genai.Text("سلام چطوری حالت خوبه؟"),     // simple meaning: Hey, how you doing, are you doing ok?
}

And what I get back is not a 400< statusCode < 600 error, but it actually passes through with a 200 status code. Then I try to retrieve the candidates and parts after it, so I carefully check each part for not being nil. Like below:

for {
        resp, err := iter.Next()
        if errors.Is(err, iterator.Done) {
            break
        }
        if err != nil {
            log.Println(err.Error())
            output <- err.Error()
            break
        }
        if resp != nil && len(resp.Candidates) > 0 {
            firstCandidate := resp.Candidates[0]
            if firstCandidate.Content != nil && len(firstCandidate.Content.Parts) > 0 {
                part := fmt.Sprint(firstCandidate.Content.Parts[0])
                output <- part
            } else {
                output <- "no content in response"
            }
        } else {
            output <- "response is empty"
        }
    }
    close(output)

And surprisingly it passes the len(resp.Candidates) condition but no value is actually available inside the candidates, it's just empty nil values. What I request is, that you handle the structure and even if google does not support a language, at least put empty strings in the content returned, not letting nil enter the conditionals, because the response structure shouldn't fail on not supporting a language

[jba]

I believe #57 should fix this. If you go get github.com/google/generative-ai-go/genai@main you should be able to try it.

[hosseinmirzapur]

I believe #57 should fix this. If you go get github.com/google/generative-ai-go/genai@main you should be able to try it.

thanks for your answer, i'll check it

UPDATE: It's working fine now, thanks again!