search

Tudor44 / assistant-api-java

Lightweight Java client for easy integration with OpenAI Assistant API, enabling Generative AI in Java applications
8 stars 6 forks source link

Simple promt results in tens of responses #1

Open PawelSzpyt opened 6 months ago

PawelSzpyt commented 6 months ago

I wrote a prompt that basically said something like "scrap me the header of this site: sitename_here", and I got TENS of responses. There is no way to control that as far as I know, and you are billed for every token returned. And its not a small bill, it can add up fast. Why you can't instruct assistant that a request of this specific type should not take more than 3 responses, for example? You can limit tokens as far as I know but I don't want to do that, but I also don't like the idea of getting literally TENS of short responses with similar content for a simple question, and getting billed for every token TENS of times (that also includes billing tens of times for prompt and for instructions). It is horrific design from OpenAI. I don't see use of it for me, as a software developer, in the current state. I will look into Chat Completions API now, as at first glance that is something more reliable.

Tudor44 commented 5 months ago

Hello @PawelSzpyt ,

I create this project for demo purpose using it for a project in my company. I will review in a short time your PR.

Warm regards and thank you for the contribution, Gaetano.

PawelSzpyt commented 5 months ago

Hi @Tudor44 , Did you end up using Assistant API in your company? No issues?

I don't see myself using Assistant API for any task in it's current state. I was also very disappointed with Autogen, and not satisfied with CrewAI. Now I go with chat completions and some small coding to perform multi-step tasks, I think it's much better to rely on your own coding and prompting.