How to simplify tool use?

[monarchwadia]

Does anybody have any suggestions for how to improve this?

Right now, the only way to capture tool use inputs and outputs is in the handler.

const capture = (payload) => console.log(payload);

const tool = new RaggedTool()
.name("theTool")
.handler((payload) => capture(payload))

This is problematic because it discourages separation of concerns. Ideally, the return value of the handler would hand off the value to ragged which would then emit it as an event.

Maybe something like this....? Although I feel like this needs to be improved too.

// for example... i actually dont like this too much
subject.subscribe(e => {
  if (e.eventType === "toolUse" && e.toolName === "theTool") {
    console.log(e.payload);
  }
})

Or maybe something more fluid. I want to remove rxjs as a dependency anyway. Maybe the returnValue could be something like this...

const stream = r.predictStream("Call the adder tool with 123 + 456");

stream
  .onToolUse("adder", (nums) => {
     console.log(nums[0], nums[1]); // 123, 456
     return nums[0] + nums[1];
  });

[CoderDill]

Hi Monarch,

I've been reviewing the challenges associated with separating concerns within Ragged's tool handling and I'd like to suggest an approach that might simplify implementation while enhancing modularity and maintainability.

Suggestion: Event Emitter Pattern

I propose using the Event Emitter Pattern to effectively decouple the handling of tool outputs from their operational logic. This pattern facilitates a clear separation between the generation of data (through tool operations) and subsequent actions taken on this data (such as logging or further processing).

Implementation Sketch

Here's how you could integrate this pattern into Ragged:

class EventEmitter {
    constructor() {
        this.events = {};
    }

    on(eventName, fn) {
        this.events[eventName] = this.events[eventName] || [];
        this.events[eventName].push(fn);
    }

    emit(eventName, data) {
        const event = this.events[eventName];
        if (event) {
            event.forEach(fn => {
                fn(data);
            });
        }
    }
}

// Usage within Ragged
const emitter = new EventEmitter();

const tool = new RaggedTool()
    .name("theTool")
    .handler((payload) => {
        emitter.emit("toolUse", payload);
    });

// Listening for tool use
emitter.on("toolUse", (payload) => {
    console.log(payload); // Handle payload separately from the tool logic
});

Benefits

• Modularity: The event emitter allows for a clean separation between computational logic and output handling, aligning with principles of good architecture.
• Flexibility: It provides the flexibility to add or modify event listeners without changing the core operational logic of your tools.
• Scalability: Adapts easily to more complex scenarios or new features with minimal changes.

Considerations

• Library Dependency: This can be implemented using a lightweight event emitter library, or through a minimal custom implementation as shown, minimizing new dependencies.
• Alternatives to RxJS: Implementing this pattern could facilitate a shift away from RxJS by replacing observables with simple event handling, aligning with your goal to potentially remove this dependency.

This event-driven model provides a robust framework for achieving the flexibility and simplicity needed in Ragged. It supports scalability and clean separation of concerns—key for maintaining and extending your library.

I hope this proposal assists in your development process, and I'm eager to see how it evolves!

Matthew

[monarchwadia]

@CoderDill thanks a lot. This is very interesting, you seem to be onto something... your chain of thought sparked something... I wonder what you think of the following code which follows your model:

const theTool = new RaggedTool()
    .name("theTool");
// no more handler

and then accessed as below

const stream = r.predictStream("Call theTool", {tools: [theTool]});

stream
  .onToolUse("theTool", (payload, emitter) => {
     console.log(payload);
     return null; // gets sent to the LLM as a response
  });

One downside of this approach is that the toolcall should only be responded to exactly once... so if multiple responses are sent because of multiple handlers, then an error will be thrown... this is the true advantage of the tool.handler

[CoderDill]

Hey Monarch,

Really appreciate your development on this! Stripping the handler from RaggedTool and using predictStream for dynamic handling sounds promising. It definitely opens up flexibility and makes the tool’s use more fluid.

Regarding the concern about multiple responses, a possible solution is to use a flag within the tool instance to ensure each tool responds just once per stream. Something like this might work:

stream.onToolUse("theTool", (payload, emitter) => {
    if (!emitter.hasResponded) {
        console.log(payload);
        emitter.hasResponded = true; // Prevent further responses
        return null; // Send this back to the LLM
    }
});

This would prevent multiple responses from cluttering up the process, keeping the operation clean and error-free.

Looking forward to your thoughts on this.

Matthew

[monarchwadia]

@CoderDill what do you think of the current API in 0.2.0 ? (i just pushed this)

non-tool-use example

const answer: string = await r
  .chat("Say 'Hello World!'") 
  .firstText();

console.log(answer); // Hello World!

tool-use example

const adder = t
  .tool()
  .title("adder")
  .description("Add two numbers together")
  .inputs({
    a: t.number().description("first number").isRequired(),
    b: t.number().description("second number").isRequired(),
  })
  .handler((input: { a: number; b: number }) => input.a + input.b);

const answer: string = await r
  .chat("Add 1 + 2", { tools: [adder] }) 
  .firstToolResult();

console.log(`answer: ${answer.result}`) // answer: 3

[CoderDill]

Hey Monarch,

I've reviewed the new API in v0.2.0, and I must say, I'm impressed with the updates. The clear distinction between chat and tool-use functionalities not only simplifies interaction but also enhances the system’s manageability—excellent work there!

Quick Thoughts:

• API Design: The separation between direct chats and tool operations is brilliantly clear, making the system both user-friendly and easier to maintain.
• Integration and Extensibility: The current setup promises seamless integration with existing frameworks and offers great potential for incorporating more sophisticated tools in the future.

Suggestion:

• Handling Multiple Responses: The addition of emitter.hasResponded to manage response control is clever. It might be beneficial to automate this within the tool management system to streamline code and avoid redundancy.

Thank you for pushing the envelope on this project. I'm excited about its direction and am eager to contribute further to its success.

Best regards, Matthew

[monarchwadia]

Thanks @CoderDill for the feedback & suggestions! After working on it for a bit, I think I'm happy with where this interface is -- it's not perfect, and won't ever be. But it's a good enough starting point. Closing this issue.