microsoft / BCApps

Repository for collaboration on Microsoft Dynamics 365 Business Central applications.
https://microsoft.github.io/BCApps/
MIT License
318 stars 158 forks source link

[Bug]: [Number Series Copilot] Bug fixes #2106

Open DmitryKatson opened 1 month ago

DmitryKatson commented 1 month ago

Describe the issue

Bugs

  1. Bug 1: Incorrect Intent Mapping for "Generate" Command

    • Problem: When users intended to create a new Number Series using the word “generate,” the system often mistakenly interpreted the command as a request to prepare Number Series for the next period (e.g., the next fiscal year). This is a misinterpretation of user intent.
    • Expected Behavior: The LLM should correctly distinguish between creating a new Number Series (using terms like "create" or "generate") and preparing a Number Series for the next year/period (using more specific time-related phrases).
    • Proposed Fix: Update the tools definition and prompt to clarify the distinction between creating new Number Series and generating Number Series for the next period. Ensure that the system identifies the intent based on contextual clues rather than assuming a time-based action when the word "generate" is used.
  2. Bug 2: Incorrect Field Check for Existing Number Series

    • Problem: When a user generated Number Series and some already existed in the database, the LLM incorrectly performed checks on all fields (e.g., warning number, increment number, ending number), which should not be the case for pre-existing series. Only new Number Series should undergo a full check.
    • Expected Behavior: The system should skip validation of non-mandatory fields (such as warning number and increment number) for existing Number Series and only perform these checks on newly created series.
    • Proposed Fix: Modify the behavior to exclude pre-existing Number Series from the field validation checks. Ensure that only mandatory fields (such as Number Series code, description, and series) are validated for new series.
  3. Bug 3: Blank Fields Causing Validation Errors

    • Problem: Sometimes the LLM generated Number Series without values for optional fields like the warning number or increment number. If the user's environment had blank fields in an existing Number Series, the system would replicate this and then fail validation when mandatory fields were checked.
    • Expected Behavior: The system should only validate mandatory fields, such as the Number Series code and description, and allow optional fields to remain blank if necessary. This would prevent errors from being thrown unnecessarily.
    • Proposed Fix: Refine the validation logic so that only mandatory fields are checked. Optional fields, such as warning number and increment number, should not cause validation failures if left blank.

Expected behavior

Steps to Implement:

  1. Bug Fixes:
    • Refactor the LLM prompt and tools definition to improve the understanding of user intent between creating and preparing Number Series for different periods.
    • Update the logic to exclude field checks for existing Number Series and restrict checks to mandatory fields.

Steps to reproduce

Acceptance Criteria:

Additional context

No response

I will provide a fix for a bug

JesperSchulz commented 6 days ago

Also backporting to 25.x. Reopening.