Dynamically adding tools does not properly handle Share Context

[cjellick]

Best example of this is the knowledge tool, which has this line:

Share Context: github.com/gptscript-ai/context/workspace

Grant put together tests here: 'https://github.com/g-linville/context-testing

Follow up question: does Share Tools work? seems like it does, but we should verify

[sangee2004]

This is one of the workflows were we see the Share Context not working when knowledge tool is added using Add Knowledge option.

1. Create an assistant with no tools.
2. Chat with the assistant and use Add Knowledge to add a knowledge file.
3. Try to ask any question relating to knowledge file like say `How to raise money to start a Business "
4. Notice that knowledge tool will not get called unless we explicitly ask it to use knowledge like - How to raise money to start a Business . Use knowledge."

This behavior of needing to ask it to explicitly use knowledge tool is not needed if I have workspace tool added to the assistant. This should not needed since we already have Share Context: github.com/gptscript-ai/context/workspace included in knowledge tool.

[cjellick]

I'm assigning to myself because I guess the best way forward with this is, as @thedadams mentioned, to support dynamic tool adding in core gptscript. So, I'll talk to @ibuildthecloud about this on monday and see where he wants to go with it.

[cjellick]

fixed in core gptscript here: https://github.com/gptscript-ai/gptscript/pull/827

[sangee2004]

Testing desktop app with gptscript version - v0.0.0-dev+f2bfa9f3

I am still able to reproduce the issue with having to ask Knowledge tool to used explicitly when testing the scenario outlined in https://github.com/gptscript-ai/desktop/issues/259#issuecomment-2305824276 . This might be relating to issue - https://github.com/gptscript-ai/desktop/issues/272.

When I tried again with another knowledge file, I was able to get the Knowledge tool to get invoked without having to ask it to use knowledge explicitly. Also I see the workspace context being used in the stack trace view.

I will also test cases mentioned in https://github.com/g-linville/context-testing

[sangee2004]

When testing scenarios by adding the tools dynamically to a thread using "Add Tool" from https://github.com/g-linville/context-testing , following are the observations made:

1. github.com/g-linville/context-testing/direct - Fail - Not able to see the context tool
2. github.com/g-linville/context-testing/indirect - Fail - Not able to see the context tool
3. github.com/g-linville/context-testing/sharedirect - Pass
4. github.com/g-linville/context-testing/sharedirect - Pass

When adding tool - github.com/g-linville/context-testing/direct

When adding tool - github.com/g-linville/context-testing/indirect

[thedadams]

In the case of github.com/g-linville/context-testing/direct, the assistant doesn't have "direct" access to the context. Only the tool blue does because the context is not shared. Therefore, the context is only run when the blue tool is run.

For example:

Similarly for the https://github.com/g-linville/context-testing/indirect tool:

Let me know what you think about these results.

[sangee2004]

Closing this issue as per testing based on https://github.com/gptscript-ai/desktop/issues/259#issuecomment-2320021127.