mochi-neko / clust

An unofficial Rust client for the Anthropic/Claude API.
Apache License 2.0
22 stars 11 forks source link

create_a_message returns Content::MultipleBlocks with StreamOption::ReturnOnce #3

Open ShelbyJenkins opened 4 months ago

ShelbyJenkins commented 4 months ago

It might be an API issue, but I thought I should raise the issue.

I'm able to parse the response any how.

  pub async fn text_generation_request(&self, req_config: &RequestConfig) -> Result<String> {
        let prompt = req_config.default_formatted_prompt.as_ref().unwrap();

        let request = MessagesRequestBuilder::new(model_id_to_enum(&self.model.model_id))
            .stream(StreamOption::ReturnOnce)
            .messages(vec![Message::user(prompt["user"]["content"].clone())])
            .system(SystemPrompt::new(prompt["system"]["content"].clone()))
            .max_tokens(
                MaxTokens::new(
                    req_config.actual_request_tokens.unwrap(),
                    model_id_to_enum(&self.model.model_id),

                )
                .map_err(|e| anyhow!(e))?,
            )
            .temperature(Temperature::new(req_config.temperature).map_err(|e| anyhow!(e))?)
            .top_p(TopP::new(req_config.top_p).map_err(|e| anyhow!(e))?)
            .build();

        match self.client().create_a_message(request).await {
            Ok(response) => match &response.content {
                Content::SingleText(content) => {
                    if self.logging_enabled {
                        tracing::info!(?response);
                    }
                    Ok(content.to_owned())
                }
                Content::MultipleBlocks(blocks) => {
                    if self.logging_enabled {
                        tracing::info!(?response);
                    }
                    if blocks.len() == 1 {
                        match &blocks[0] {
                            ContentBlock::Text(content) => Ok(content.text.to_owned()),
                            _ => panic!("Images not supported"),
                        }
                    } else {
                        panic!("MultipleBlocks not supported")
                    }
                }
            },
            Err(e) => {
                let error =
                    anyhow::format_err!("AnthropicBackend text_generation_request error: {}", e);

                if self.logging_enabled {
                    tracing::info!(?error);
                }
                Err(error)
            }
        }
    }
mochi-neko commented 4 months ago

@ShelbyJenkins

ResponseBody.content is multiple content blocks (but usually may be single length of text content block) on non-streaming API.

This is an array of content blocks, each of which has a type that determines its shape. Currently, the only type in responses is "text".

https://docs.anthropic.com/en/api/messages#:~:text=This%20is%20an%20array%20of%20content%20blocks%2C%20each%20of%20which%20has%20a%20type%20that%20determines%20its%20shape.%20Currently%2C%20the%20only%20type%20in%20responses%20is%20%22text%22.

The text content in response can be excluded by Contentl.flatten_into_text() method with Result

https://github.com/mochi-neko/clust/blob/fe0a62e18e08391c2cedabcc0a0005a174ae4d9e/src/messages/content.rs#L123C12-L123C29

like this example

https://github.com/mochi-neko/clust/blob/main/examples/create_a_message.rs