Mintplex-Labs / anything-llm

The all-in-one Desktop & Docker AI application with built-in RAG, AI agents, and more.
https://anythingllm.com
MIT License
22.79k stars 2.3k forks source link

Poor results compared to DocsBot #325

Closed wimjongman closed 10 months ago

wimjongman commented 10 months ago

I am trying out this code, and kudos for the YouTube introduction and the effortless onboarding. I have installed, configured, and uploaded one 90-page manual in around 20 minutes.

I am hooked into a paid OpenAI key, and I am using the default vector db.

Then I ask a question, but the response is incoherent. DocsBot OTOH got it spot on.

Q: How do I create a task

Docsbot answer (perfect) To create a task in TDOMS, you can follow these steps: Start the New Task wizard by choosing "New -> Task" from the context menu in any TD/OMS view or by clicking the "New Task" button in the toolbar. In the wizard, enter the required information for the task, such as the application, task type, release, priority code, programmer, short task description, and extended task description. Note that fields marked with an asterisk (*) are required. If you don't need to enter task project information, you can directly press the "Create Task" button. Otherwise, press the "Next" button. On the next page, enter the task project information if needed, such as the expected start date, expected development end date, expected completion date, and expected hours. After entering the task project information, press the "Create Task" button. The wizard will then show the Choice screen, which informs you about the action performed and allows you to initiate the next step (connecting a new request) or repeat the current activity (creating a task). You can press the "Stop" button to end the wizard. Please note that if any errors occur during the task creation process, you will receive an error message.
Anythingllm (incoherent) To create a task, follow these steps: Switch to the TD/OMS Work Management view. Select the task you want to add to the project. **>> Wrong** Right-click on the task and select “Add task to Project” from the context menu. **>> Wrong** Select the project to which you want to add the task and press the OK button. **>> Wrong** Fill in the required fields in the New Task wizard, such as application, task type, release, priority code, programmer, short task description, and extended task description. **>> Good but out of context** If needed, enter the Task Project information. **>> Good but out of context** Press the Create Task button to create the task. **>> Good but out of context** Note: You can also access the New Task wizard by using the quick-access box in the toolbar and typing “Task”. **>> Good**
This is the paragraph from the manual titled: 'Creating a Task' (please ignore mediawiki markup) Creating a Task You can start creating a new task by choosing New->Task from the context-menu in any TD/OMS view or by pressing the New Task Ewm imgf1.png button in the toolbar. This will start the New Task wizard. Page 1 - Enter Remote System This page will not be shown when you have started the wizard and the context could have been determined from the current selected element. The Host (Remote system) name is already filled in and can not be changed when a detail line of a Remote system has been selected within the TD/OMS Work Management view. Select the Host name when this is not the case and press the Next-button. Ewm imgf2.png Page 2 - Enter Task Attributes All fields indicated with an asterisk are required. Specifying *GEN for the Task will instruct TD/OMS to generate the Task number based of the application specific settings. Enter a long description (if needed). Press the Create Task button if there is no requirement to enter Task Project information or press the Next-button if there is. Ewm imgf3.png Fields [Application](https://remainsoftware.com/wiki/index.php/APPC001) [Task](https://remainsoftware.com/wiki/index.php/FIX_001) [Transfer Path](https://remainsoftware.com/wiki/index.php/PATH001) [Task Type](https://remainsoftware.com/wiki/index.php/FIXT001) [Release](https://remainsoftware.com/wiki/index.php/RLS_001) [Priority Code](https://remainsoftware.com/wiki/index.php/PRIC001) [Programmer](https://remainsoftware.com/wiki/index.php/PGMR001) [Short Task Description](https://remainsoftware.com/wiki/index.php/SHFD001) [Extended Task Description](https://remainsoftware.com/wiki/index.php/SORD003) Page 3 - Enter Project Dates Enter the Task Project information (if needed) and press the Create Task button. You can press the question mark button in the lower left corner to obtain context sensitive help. Please note that you can also use the planning view to manipulate the project dates. Fields [Expected Start Date](https://remainsoftware.com/wiki/index.php/ESDT001) [Expected Development End Date](https://remainsoftware.com/wiki/index.php/EDDT001) [Excepted Completion Date](https://remainsoftware.com/wiki/index.php/ECDT001) [Expected Hours](https://remainsoftware.com/wiki/index.php/EPH_001) Page 4 - Continue Adding a Task may fail in which case you will receive an error message. After generating a Task the wizard will show the Choice screen. This screen informs you about the action performed and enables you to immediately initiate the next step (Connecting a (new) Request) or repeat the current activity (Creating a Task). You can press the Stop button to end the wizard.
wimjongman commented 10 months ago

I converted the PDF to a txt document and cleaned it up as much as possible.

With gpt3 the response was a bit better, and with gpt4 it was spot on:

anythingllm + txt + gpt4 was perfect You can create a new task by choosing New->Task from the context-menu in any TD/OMS view or by pressing the New Task button in the toolbar. This will start the New Task wizard. In the wizard, you will need to enter various task attributes. If you started the wizard and the context could have been determined from the current selected element, the Host (Remote system) name is already filled in and cannot be changed. If this is not the case, you need to select the Host name and press the Next-button. You will then need to fill in fields such as Application, Task, Transfer Path, Task Type, Release, Priority Code, Programmer, Short Task Description, and Extended Task Description. All fields indicated with an asterisk are required. If there is a requirement to enter Task Project information, press the Next-button. If there is no such requirement, press the Create Task button. If you pressed the Next-button, you will need to enter the Task Project information and press the Create Task button. You can also use the planning view to manipulate the project dates. After generating a Task, the wizard will show the Choice screen. This screen informs you about the action performed and enables you to immediately initiate the next step or repeat the current activity. You can press the Stop button to end the wizard.
timothycarambat commented 10 months ago

Well now its no mystery what Docsbot is using!!

Good to know :)

wimjongman commented 10 months ago

Well now its no mystery what Docsbot is using!!

Good to know :)

Can you elaborate on that a bit?