Context length issues

[vedant-sanil]

Hi,

I am using GPT-3.5-Turbo for the Clinfo API, but I run into a "context length exceeded" error from OpenAI for some queries. Is it possible to control the size of the context being used for these queries?

[Ale9806]

Hello Vendant I can try to fix the error, can you please provide more info so I can re create it? What is your initial query?

[vedant-sanil]

Hi Alejandro, appreciate your quick response. Here is the query I was trying: What tests are needed to diagnose Chronic Neutropenia?

Some more info: I was using the following architecture_path, prompts/PubMed/Architecture_1/master.json. And the model as I mentioned was GPT-3.5-Turbo.

And here is the error: Error code: 400 - {'error': {'message': "This model's maximum context length is 16385 tokens. However, your messages resulted in 24204 tokens. Please reduce the length of the messages.", 'type': 'invalid_request_error', 'param': 'messages', 'code': 'context_length_exceeded'}}

[Ale9806]

Thanks on it!

[Ale9806]

@vedant-sanil Can you provide more context on how you are initializing clinfo.ai? (share code) are you changing anything? I am testing it right now and can't reproduce the error

For instance this is what I get when I submit your question:

`synthesis Literature Summary: Studies on chronic neutropenia in children have highlighted various diagnostic approaches. De Alarcon et al. utilized bone marrow aspiration, biopsy, steroid stimulation, and in vitro assays to categorize patients, finding that a normal bone marrow response to steroids indicated a benign course, while CFU-GM and CSA assays helped classify neutropenia. Antineutrophil antibody quantification in children with chronic neutropenia may aid in understanding the clinical course, as demonstrated by Kobayashi et al. In autoimmune neutropenia of infancy, Lalezari et al. emphasized the importance of immunologic tests for diagnosis. Bux et al. found associations between reduced CFU-GM, circulating hematopoietic progenitors, and severe infections in pediatric patients with anti-neutrophil antibodies. Dale and Welte reviewed chronic idiopathic neutropenia (CIN) and autoimmune neutropenia (AIN), suggesting G-CSF treatment for increasing neutrophils in symptomatic cases. While these studies provide valuable insights, the evidence is limited by small sample sizes, potential selection bias, and the retrospective nature of some studies.

TL;DR: Diagnostic tests for chronic neutropenia in children may include bone marrow evaluation, steroid stimulation, antineutrophil antibody quantification, and immunologic tests, with potential treatment implications, but the evidence is limited by study design and sample sizes.

References:

1. de Alarcon PA, Goldberg J, Nelson DA, Stockman JA. Chronic neutropenia: diagnostic approach and prognosis. The American journal of pediatric hematology/oncology. 1983;5(1):3-9.
2. Kobayashi M, Nakamura K, Kawaguchi H, Sato T, Kihara H, Hiraoka A, Tanihiro M, Taniguchi K, Takata N, Ueda K. Significance of the detection of antineutrophil antibodies in children with chronic neutropenia. Blood. 2002;99(9):3468-71.
3. Lalezari P, Khorshidi M, Petrosova M. Autoimmune neutropenia of infancy. The Journal of pediatrics. 1986;109(5):764-9.
4. Bux J, Behrens G, Jaeger G, Welte K. Diagnosis and clinical course of autoimmune neutropenia in infancy: analysis of 240 cases. Blood. 1998;91:181–186.
5. Dale DC, Welte K. Neutropenia and Neutrophilia. In: Kaushansky K, Lichtman MA, Prchal JT, et al., editors. Williams Hematology. 9th. New York, NY: McGraw-Hill; 2016. pp. 991–1004.`

[vedant-sanil]

Here is the code I am using to initialize the model:

PROMPS_PATH = '<clinfoai_path>/prompts/PubMed/Architecture_1/master.json'
question = "What tests are needed to diagnose Chronic Neutropenia?"
clinfo   = ClinfoAI(llm="gpt-3.5-turbo",
                    openai_key=<OPENAI_API_KEY>, 
                    email=<email>,
                    architecture_path=PROMPS_PATH)
answer   = clinfo.forward(question=question)