[Bug]: no citations return using azure ai search

[yonggang-xiao]

Language

Javascript/Typescript

Version

latest

Description

chat return text with [doc1] [doc2], but no citations in content CHAT PROMPT: [ { "role": "system", "content": "The following is a conversation with an AI assistant, who is an expert on answering questions over the given context.\nResponses should be in a short journalistic style with no more than 80 words." }, { "role": "user", "content": "tell me about PerksPlus" } ] CHAT RESPONSE: duration: 4086ms { "role": "assistant", "content": "Contoso Electronics' PerksPlus Program is a benefits initiative aimed at supporting employees' health and wellness. It allows employees to expense up to $1000 for various fitness-related activities [doc1]. Covered activities include gym memberships, personal training, yoga, Pilates, fitness equipment, sports team fees, health retreats, outdoor adventures, group fitness, and virtual fitness programs [doc1]. Additionally, the program supports lessons such as skiing, scuba diving, surfing, and horseback riding [doc1]. Non-fitness-related expenses, medical treatments, travel not related to fitness, and food supplements are not covered under the program [doc1]." }

Reproduction Steps

1.config.json
{
    "schema": 1.1,
    "description": "Chat with Teams RAG",
    "type": "completion",
    "completion": {
        "model": "gpt-35-turbo",
        "completion_type": "chat",
        "include_history": true,
        "include_input": true,
        "max_input_tokens": 5000,
        "max_tokens": 1000,
        "temperature": 0.9,
        "top_p": 0.0,
        "presence_penalty": 0.6,
        "frequency_penalty": 0.0,
        "stop_sequences": []
    },
    "augmentation": {
        "augmentation_type": "none"
    }
}

2.app.js
const { ActivityTypes, MemoryStorage, MessageFactory } = require("botbuilder");
const path = require("path");
const config = require("../config");

// See https://aka.ms/teams-ai-library to learn more about the Teams AI library.
const { Application, ActionPlanner, OpenAIModel, PromptManager } = require("@microsoft/teams-ai");

// Create AI components
const model = new OpenAIModel({
  azureApiKey: config.azureOpenAIKey,
  azureDefaultDeployment: config.azureOpenAIDeploymentName,
  azureEndpoint: config.azureOpenAIEndpoint,
  azureApiVersion: '2024-05-01-preview',
  useSystemMessages: true,
  logRequests: true,
});
const prompts = new PromptManager({
  promptsFolder: path.join(__dirname, "../prompts"),
});
const planner = new ActionPlanner({
  model,
  prompts,
  defaultPrompt: async () => {
    const prompt = await prompts.getPrompt('chat2');

    prompt.config.completion.model = config.azureOpenAIDeploymentName;

    if (config.azureSearchKey && config.azureSearchEndpoint) {
        prompt.config.completion.data_sources = [
            {
                type: 'azure_search',
                parameters: {
                    endpoint: config.azureSearchEndpoint,
                    index_name: 'my-documents',
                    semantic_configuration: 'default',
                    query_type: 'vector_simple_hybrid',
                    embedding_dependency: {
                        type: 'deployment_name',
                        deployment_name: config.azureOpenAIEmbeddingDeploymentName,
                        dimensions: 3072
                    },
                    fields_mapping: {
                      content_fields: ['description'],
                      vector_fields: ['descriptionVector'],
                      title_field: 'docTitle',
                    },
                    in_scope: false,
                    strictness: 3,
                    top_n_documents: 5,
                    role_information: fs
                        .readFileSync(path.join(__dirname, '../prompts/chat2/skprompt.txt'))
                        .toString('utf-8'),
                    authentication: {
                        type: 'api_key',
                        key: config.azureSearchKey
                    },
                }
            }
        ];
    }

    return prompt;
  }
});

// Define storage and application
const storage = new MemoryStorage();
const app = new Application({
  storage,
  ai: {
    planner,
    enable_feedback_loop: true
  },
});

app.message('reset', async (context, state) => {
  state.deleteConversationState();
  await context.sendActivity("Ok I've deleted the current conversation state.");
});

module.exports = app;

...

[corinagum]

In order to get citations to work with Azure search, you will need to extend SayCommand to handle incoming data that matches the Azure search sources structure. To my knowledge, how that data is structured is dependent on your Azure Search settings so will be different from customer to customer. Please see modifying SAY command for more information.

If you are planning on using enable_feedback_loop, you will also need to create a route handler for app.feedbackLoop in order for that functionality to work in Teams.

[reopening -- currently discussing with @singhk97]

[yonggang-xiao]

app.ai.action(AI.SayCommandActionName, async (context, state, data) => { console.log(data); return ''; }); I got the log: { type: 'SAY', response: { role: 'assistant', content: 'PerksPlus is a benefits program offered by Contoso Electronics, aimed at supporting the health and wellness of employees. It allows employees to expense up to $1000 for various fitness-related activities, ranging from gym memberships and personal training sessions to virtual fitness programs and outdoor adventure activities such as rock climbing and kayaking [doc1]. Additionally, the program covers a variety of lessons that promote health and wellness, including skiing, scuba diving, and horseback riding lessons [doc1]. However, non-fitness-related expenses, medical treatments, and travel expenses unrelated to fitness programs are not covered [doc1].' } } I can not see any citations?

[yonggang-xiao]

POST {endpoint}/openai/deployments/{deployment-id}/chat/completions?api-version={api-version} I test the api url directly with the same data_sources parameters, and got the return json with citations: { "id": "46876dd4-c21d-4fc6-bc4c-2128d1eb3529", "model": "gpt-4o", "created": 1727399822, "object": "extensions.chat.completion", "choices": [ { "index": 0, "finish_reason": "stop", "message": { "role": "assistant", "content": "The PerksPlus program at Contoso Electronics is a comprehensive benefits initiative designed to support the health and wellness of employees. It allows employees to expense up to $1000 for fitness-related programs, making it easier and more affordable to maintain a healthy lifestyle. The program covers a wide range of fitness activities, including:\n\n- Gym memberships\n- Personal training sessions\n- Yoga and Pilates classes\n- Fitness equipment purchases\n- Sports team fees\n- Health retreats and spas\n- Outdoor adventure activities (such as rock climbing, hiking, and kayaking)\n- Group fitness classes (such as dance, martial arts, and cycling)\n- Virtual fitness programs (such as online yoga and workout classes)\n\nAdditionally, PerksPlus covers various lessons and experiences that promote health and wellness, such as:\n\n- Skiing and snowboarding lessons\n- Scuba diving lessons\n- Surfing lessons\n- Horseback riding lessons\n\nThese activities provide employees with opportunities to try new things, challenge themselves, and improve their physical skills, while also helping to relieve stress and have fun [doc1].\n\nHowever, the program does not cover non-fitness related expenses, medical treatments and procedures, travel expenses (unless related to a fitness program), or food and supplements [doc1].", "end_turn": true, "context": { "citations": [ { "content": "# Contoso Electronics PerksPlus Program\n\nDisclaimer: This document contains information generated using a language model (Azure OpenAI). The information contained in this document is only for demonstration purposes and does not reflect the opinions or beliefs of Microsoft. Microsoft makes no representations or warranties of any kind, express or implied, about the completeness, accuracy, reliability, suitability or availability with respect to the information contained in this document. All rights reserved to Microsoft.\n\n## Overview\nIntroducing PerksPlus - the ultimate benefits program designed to support the health and wellness of employees. With PerksPlus, employees have the opportunity to expense up to $1000 for fitness-related programs, making it easier and more affordable to maintain a healthy lifestyle. PerksPlus is not only designed to support employees' physical health, but also their mental health. Regular exercise has been shown to reduce stress, improve mood, and enhance overall well-being. With PerksPlus, employees can invest in their health and wellness, while enjoying the peace of mind that comes with knowing they are getting the support they need to lead a healthy life.\nWhat is Covered?\n\nPerksPlus covers a wide range of fitness activities, including but not limited to:\n Gym memberships\n Personal training sessions\n Yoga and Pilates classes\n Fitness equipment purchases\n Sports team fees\n Health retreats and spas\n Outdoor adventure activities (such as rock climbing, hiking, and kayaking)\n Group fitness classes (such as dance, martial arts, and cycling)\n Virtual fitness programs (such as online yoga and workout classes)\n\nIn addition to the wide range of fitness activities covered by PerksPlus, the program also covers a variety of lessons and experiences that promote health and wellness. Some of the lessons covered under PerksPlus include:\n Skiing and snowboarding lessons\n Scuba diving lessons\n Surfing lessons\n Horseback riding lessons\n\nThese lessons provide employees with the opportunity to try new things, challenge themselves, and improve their physical skills. They are also a great way to relieve stress and have fun while staying active.\n\nWith PerksPlus, employees can choose from a variety of fitness programs to suit their individual needs and preferences. Whether you're looking to improve your physical fitness, reduce stress, or just have some fun, PerksPlus has you covered.\n\n## What is Not Covered?\nIn addition to the wide range of activities covered by PerksPlus, there is also a list of things that are not \ncovered under the program. These include but are not limited to:\n Non-fitness related expenses\n Medical treatments and procedures\n Travel expenses (unless related to a fitness program)\n Food and supplements", "title": "Contoso Electronics_PerkPlus_Program.md", "url": null, "filepath": null, "chunk_id": "0" }, { "content": "# Contoso Electronics Plan and Benefit Packages\n\nDisclaimer: This document contains information generated using a language model (Azure OpenAI). The information contained in this document is only for demonstration purposes and does not reflect the opinions or beliefs of Microsoft. Microsoft makes no representations or warranties of any kind, express or implied, about the completeness, accuracy, reliability, suitability or availability with respect to the information contained in this document. All rights reserved to Microsoft.\n\n## Northwind Health Plus\n\nNorthwind Health Plus is a comprehensive plan that provides comprehensive coverage for medical, vision, and dental services. This plan also offers prescription drug coverage, mental health and substance abuse coverage, and coverage for preventive care services. With Northwind Health Plus, you can choose from a variety of in-network providers, including primary care physicians, specialists, hospitals, and pharmacies. \n\nThis plan also offers coverage for emergency services, both in-network and out-of-network.\n\n## Northwind Standard\n\nNorthwind Standard is a basic plan that provides coverage for medical, vision, and dental services. This plan also offers coverage for preventive care services, as well as prescription drug coverage. With Northwind Standard, you can choose from a variety of in-network providers, including primary care physicians, specialists, hospitals, and pharmacies. This plan does not offer coverage for emergency services, mental health and substance abuse coverage, or out-of-network services.\n\n## Comparison of Plans\n\nBoth plans offer coverage for routine physicals, well-child visits, immunizations, and other preventive care services. The plans also cover preventive care services such as mammograms, colonoscopies, and other cancer screenings. \n\nNorthwind Health Plus offers more comprehensive coverage than Northwind Standard. This plan offers coverage for emergency services, both in-network and out-of-network, as well as mental health and substance abuse coverage. Northwind Standard does not offer coverage for emergency services, mental health and substance abuse coverage, or out-of-network services. \n\nBoth plans offer coverage for prescription drugs. Northwind Health Plus offers a wider range of prescription drug coverage than Northwind Standard. Northwind Health Plus covers generic, brand-name, and specialty drugs, while Northwind Standard only covers generic and brand-name drugs. \n\nBoth plans offer coverage for vision and dental services. Northwind Health Plus offers coverage for vision exams, glasses, and contact lenses, as well as dental exams, cleanings, and fillings. Northwind Standard only offers coverage for vision exams and glasses. \n\nBoth plans offer coverage for medical services. Northwind Health Plus offers coverage for hospital stays, doctor visits, lab tests, and X-rays. Northwind Standard only offers coverage for doctor visits and lab tests. \n\nNorthwind Health Plus is a comprehensive plan that offers more coverage than Northwind Standard. Northwind Health Plus offers coverage for emergency services, mental health and substance abuse coverage, and out-of-network services, while Northwind Standard does not. Northwind Health Plus also offers a wider range of prescription drug coverage than Northwind Standard. Both plans offer coverage for vision and dental services, as well as medical services.\n\n## Cost Comparison\n\nContoso Electronics deducts the employee's portion of the healthcare cost from each paycheck. This means that the cost of the health insurance will be spread out over the course of the year, rather than being paid in one lump sum. The employee's portion of the cost will be calculated based on the selected health plan and the number of people covered by the insurance. The table below shows a cost comparison between the different health plans offered by Contoso Electronics\n\n| | Northwind Standard | NorthWind Health Plus |\n|---------------|----------|---------------------|\n| Employee Only | $45.00 | $55.00 |\n| Employee +1 | $65.00 | $71.00 |\n| Employee +2 or more | $78.00 | $89.00 |", "title": "Contoso_Electronics_Plan_Benefits.md", "url": null, "filepath": null, "chunk_id": "0" }, { "content": "# Contoso Electronics Company Overview\n\nDisclaimer: This document contains information generated using a language model (Azure OpenAI). The information contained in this document is only for demonstration purposes and does not reflect the opinions or beliefs of Microsoft. Microsoft makes no representations or warranties of any kind, express or implied, about the completeness, accuracy, reliability, suitability or availability with respect to the information contained in this document. All rights reserved to Microsoft.*\n\n## History\n\nContoso Electronics, a pioneering force in the tech industry, was founded in 1985 by visionary entrepreneurs with a passion for innovation. Over the years, the company has played a pivotal role in shaping the landscape of consumer electronics.\n\n| Year | Milestone |\n|------|-----------|\n| 1985 | Company founded with a focus on cutting-edge technology |\n| 1990 | Launched the first-ever handheld personal computer |\n| 2000 | Introduced groundbreaking advancements in AI and robotics |\n| 2015 | Expansion into sustainable and eco-friendly product lines |\n\n## Company Overview\n\nAt Contoso Electronics, we take pride in fostering a dynamic and inclusive workplace. Our dedicated team of experts collaborates to create innovative solutions that empower and connect people globally.\n\n### Core Values\n\n- Innovation: Constantly pushing the boundaries of technology.\n- Diversity: Embracing different perspectives for creative excellence.\n- Sustainability: Committed to eco-friendly practices in our products.\n\n## Vacation Perks\n\nWe believe in work-life balance and understand the importance of well-deserved breaks. Our vacation perks are designed to help our employees recharge and return with renewed enthusiasm.\n\n| Vacation Tier | Duration | Additional Benefits |\n|---------------|----------|---------------------|\n| Standard | 2 weeks | Health and wellness stipend |\n| Senior | 4 weeks | Travel vouchers for a dream destination |\n| Executive | 6 weeks | Luxury resort getaway with family |\n\n## Employee Recognition\n\nRecognizing the hard work and dedication of our employees is at the core of our culture. Here are some ways we celebrate achievements:\n\n- Monthly \"Innovator of the Month\" awards\n- Annual gala with awards for outstanding contributions\n- Team-building retreats for high-performing departments\n\n## Join Us!\n\nContoso Electronics is always on the lookout for talented individuals who share our passion for innovation. If you're ready to be part of a dynamic team shaping the future of technology, check out our careers page for exciting opportunities.\n\nLearn more about Contoso Electronics!\n", "title": "Contoso_Electronics_Company_Overview.md", "url": null, "filepath": null, "chunk_id": "0" } ], "intent": "[\"PerksPlus program\", \"PerksPlus benefits\", \"What is PerksPlus?\"]" } } } ], "usage": { "prompt_tokens": 4217, "completion_tokens": 270, "total_tokens": 4487 }, "system_fingerprint": "fp_67802d9a6d" }

[fgodino-milu]

I am having the exact same issue

[corinagum]

My apologies for prematurely closing this issue. Thanks for your patience while I investigate. @yonggang-xiao or @fgodino-milu, would either of you be able to spin up a repo of 'minimum repro' of this scenario? There's quite a few paths to explore.

Also, does the OYD sample work for both of you? Or do you experience the same, citations not getting parsed?

Also was this experience previously working and now not, or are these new scenarios for the both of you?

[fgodino]

I will spin up a repo this evening. The issue is happening with version 1.4.x, but working fine on 1.3.x. Thanks for looking it up

[yonggang-xiao]

I change the version to 1.3.1, and update the SayCommand.js file with abstract: Utilities_1.Utilities.snippet(citation.content, 480), got the correct results:

CHAT PROMPT: [ { "role": "system", "content": "The following is a conversation with an AI assistant, who is an expert on answering questions over the given context.\nResponses should be in a short journalistic style with no more than 80 words." }, { "role": "user", "content": "tell me about PerksPlus" } ] CHAT RESPONSE: status: 200 duration: 3946ms { "id": "09047ece-c27f-41a4-91c4-36b443255295", "model": "gpt-4o", "created": 1728527193, "object": "extensions.chat.completion", "choices": [ { "index": 0, "finish_reason": "stop", "message": { "role": "assistant", "content": "The PerksPlus program from Contoso Electronics is a benefits initiative aimed at supporting employees' health and wellness. It allows employees to expense up to $1000 for fitness-related activities like gym memberships, personal training, yoga classes, and even outdoor adventures such as rock climbing and kayaking [doc1]. Additionally, it covers lessons for activities like skiing, scuba diving, and horseback riding, providing opportunities for stress relief and physical fitness. Certain non-fitness-related expenses and medical treatments are excluded from coverage [doc1].", "end_turn": true, "context": { "citations": [ { "content": "# Contoso Electronics PerksPlus Program\n\nDisclaimer: This document contains information generated using a language model (Azure OpenAI). The information contained in this document is only for demonstration purposes and does not reflect the opinions or beliefs of Microsoft. Microsoft makes no representations or warranties of any kind, express or implied, about the completeness, accuracy, reliability, suitability or availability with respect to the information contained in this document. All rights reserved to Microsoft.\n\n## Overview\nIntroducing PerksPlus - the ultimate benefits program designed to support the health and wellness of employees. With PerksPlus, employees have the opportunity to expense up to $1000 for fitness-related programs, making it easier and more affordable to maintain a healthy lifestyle. PerksPlus is not only designed to support employees' physical health, but also their mental health. Regular exercise has been shown to reduce stress, improve mood, and enhance overall well-being. With PerksPlus, employees can invest in their health and wellness, while enjoying the peace of mind that comes with knowing they are getting the support they need to lead a healthy life.\nWhat is Covered?\n\nPerksPlus covers a wide range of fitness activities, including but not limited to:\n Gym memberships\n Personal training sessions\n Yoga and Pilates classes\n Fitness equipment purchases\n Sports team fees\n Health retreats and spas\n Outdoor adventure activities (such as rock climbing, hiking, and kayaking)\n Group fitness classes (such as dance, martial arts, and cycling)\n Virtual fitness programs (such as online yoga and workout classes)\n\nIn addition to the wide range of fitness activities covered by PerksPlus, the program also covers a variety of lessons and experiences that promote health and wellness. Some of the lessons covered under PerksPlus include:\n Skiing and snowboarding lessons\n Scuba diving lessons\n Surfing lessons\n Horseback riding lessons\n\nThese lessons provide employees with the opportunity to try new things, challenge themselves, and improve their physical skills. They are also a great way to relieve stress and have fun while staying active.\n\nWith PerksPlus, employees can choose from a variety of fitness programs to suit their individual needs and preferences. Whether you're looking to improve your physical fitness, reduce stress, or just have some fun, PerksPlus has you covered.\n\n## What is Not Covered?\nIn addition to the wide range of activities covered by PerksPlus, there is also a list of things that are not \ncovered under the program. These include but are not limited to:\n Non-fitness related expenses\n Medical treatments and procedures\n Travel expenses (unless related to a fitness program)\n Food and supplements", "title": "Contoso Electronics_PerkPlus_Program.md", "url": null, "filepath": null, "chunk_id": "0" }, { "content": "# Contoso Electronics Company Overview\n\nDisclaimer: This document contains information generated using a language model (Azure OpenAI). The information contained in this document is only for demonstration purposes and does not reflect the opinions or beliefs of Microsoft. Microsoft makes no representations or warranties of any kind, express or implied, about the completeness, accuracy, reliability, suitability or availability with respect to the information contained in this document. All rights reserved to Microsoft.\n\n## History\n\nContoso Electronics, a pioneering force in the tech industry, was founded in 1985 by visionary entrepreneurs with a passion for innovation. Over the years, the company has played a pivotal role in shaping the landscape of consumer electronics.\n\n| Year | Milestone |\n|------|-----------|\n| 1985 | Company founded with a focus on cutting-edge technology |\n| 1990 | Launched the first-ever handheld personal computer |\n| 2000 | Introduced groundbreaking advancements in AI and robotics |\n| 2015 | Expansion into sustainable and eco-friendly product lines |\n\n## Company Overview\n\nAt Contoso Electronics, we take pride in fostering a dynamic and inclusive workplace. Our dedicated team of experts collaborates to create innovative solutions that empower and connect people globally.\n\n### Core Values\n\n- Innovation: Constantly pushing the boundaries of technology.\n- Diversity: Embracing different perspectives for creative excellence.\n- Sustainability: Committed to eco-friendly practices in our products.\n\n## Vacation Perks\n\nWe believe in work-life balance and understand the importance of well-deserved breaks. Our vacation perks are designed to help our employees recharge and return with renewed enthusiasm.\n\n| Vacation Tier | Duration | Additional Benefits |\n|---------------|----------|---------------------|\n| Standard | 2 weeks | Health and wellness stipend |\n| Senior | 4 weeks | Travel vouchers for a dream destination |\n| Executive | 6 weeks | Luxury resort getaway with family |\n\n## Employee Recognition\n\nRecognizing the hard work and dedication of our employees is at the core of our culture. Here are some ways we celebrate achievements:\n\n- Monthly \"Innovator of the Month\" awards\n- Annual gala with awards for outstanding contributions\n- Team-building retreats for high-performing departments\n\n## Join Us!\n\nContoso Electronics is always on the lookout for talented individuals who share our passion for innovation. If you're ready to be part of a dynamic team shaping the future of technology, check out our careers page for exciting opportunities.\n\nLearn more about Contoso Electronics!\n", "title": "Contoso_Electronics_Company_Overview.md", "url": null, "filepath": null, "chunk_id": "0" }, { "content": "# Contoso Electronics Plan and Benefit Packages\n\nDisclaimer: This document contains information generated using a language model (Azure OpenAI). The information contained in this document is only for demonstration purposes and does not reflect the opinions or beliefs of Microsoft. Microsoft makes no representations or warranties of any kind, express or implied, about the completeness, accuracy, reliability, suitability or availability with respect to the information contained in this document. All rights reserved to Microsoft.*\n\n## Northwind Health Plus\n\nNorthwind Health Plus is a comprehensive plan that provides comprehensive coverage for medical, vision, and dental services. This plan also offers prescription drug coverage, mental health and substance abuse coverage, and coverage for preventive care services. With Northwind Health Plus, you can choose from a variety of in-network providers, including primary care physicians, specialists, hospitals, and pharmacies. \n\nThis plan also offers coverage for emergency services, both in-network and out-of-network.\n\n## Northwind Standard\n\nNorthwind Standard is a basic plan that provides coverage for medical, vision, and dental services. This plan also offers coverage for preventive care services, as well as prescription drug coverage. With Northwind Standard, you can choose from a variety of in-network providers, including primary care physicians, specialists, hospitals, and pharmacies. This plan does not offer coverage for emergency services, mental health and substance abuse coverage, or out-of-network services.\n\n## Comparison of Plans\n\nBoth plans offer coverage for routine physicals, well-child visits, immunizations, and other preventive care services. The plans also cover preventive care services such as mammograms, colonoscopies, and other cancer screenings. \n\nNorthwind Health Plus offers more comprehensive coverage than Northwind Standard. This plan offers coverage for emergency services, both in-network and out-of-network, as well as mental health and substance abuse coverage. Northwind Standard does not offer coverage for emergency services, mental health and substance abuse coverage, or out-of-network services. \n\nBoth plans offer coverage for prescription drugs. Northwind Health Plus offers a wider range of prescription drug coverage than Northwind Standard. Northwind Health Plus covers generic, brand-name, and specialty drugs, while Northwind Standard only covers generic and brand-name drugs. \n\nBoth plans offer coverage for vision and dental services. Northwind Health Plus offers coverage for vision exams, glasses, and contact lenses, as well as dental exams, cleanings, and fillings. Northwind Standard only offers coverage for vision exams and glasses. \n\nBoth plans offer coverage for medical services. Northwind Health Plus offers coverage for hospital stays, doctor visits, lab tests, and X-rays. Northwind Standard only offers coverage for doctor visits and lab tests. \n\nNorthwind Health Plus is a comprehensive plan that offers more coverage than Northwind Standard. Northwind Health Plus offers coverage for emergency services, mental health and substance abuse coverage, and out-of-network services, while Northwind Standard does not. Northwind Health Plus also offers a wider range of prescription drug coverage than Northwind Standard. Both plans offer coverage for vision and dental services, as well as medical services.\n\n## Cost Comparison\n\nContoso Electronics deducts the employee's portion of the healthcare cost from each paycheck. This means that the cost of the health insurance will be spread out over the course of the year, rather than being paid in one lump sum. The employee's portion of the cost will be calculated based on the selected health plan and the number of people covered by the insurance. The table below shows a cost comparison between the different health plans offered by Contoso Electronics\n\n| | Northwind Standard | NorthWind Health Plus |\n|---------------|----------|---------------------|\n| Employee Only | $45.00 | $55.00 |\n| Employee +1 | $65.00 | $71.00 |\n| Employee +2 or more | $78.00 | $89.00 |", "title": "Contoso_Electronics_Plan_Benefits.md", "url": null, "filepath": null, "chunk_id": "0" } ], "intent": "[\"PerksPlus\", \"What is PerksPlus?\", \"PerksPlus benefits and features\"]" } } } ], "usage": { "prompt_tokens": 4632, "completion_tokens": 123, "total_tokens": 4755 }, "system_fingerprint": "fp_67802d9a6d" }

[rbertucat]

@corinagum using 1.5.1, somehow, there are no more citations in the answer (it shows [doc 1], [doc 2], etc. but no citations are displayed for those), and using 1.3.1, there is the #2080 bug. Any suggestions on a workaround?

[corinagum]

@yonggang-xiao I just released version 1.5.3 which includes the citation fixes. Could you please bump that package (and openai to 4.68.2) to verify the fix applies to your case as well? Thank you!