AI 128k context would allow us to send few-shots prompts

adRn-s commented 3 months ago

I'd like to send some of my saved queries (optionally, with their results) to the AI assistant, to give some examples accompanying the question. I believe this would greatly improve its answer...

PS. Llama 3.1 has a 128k context, and would be suitable for this.

chrisclark commented 3 months ago

Good idea. I'm planning on doing a v2 of the assistant that will incorporate some features like this.

For the few-shot, would you want to always include the same handful of queries every time you use the assitant, or a UI to quckly select an example to query to send, which would vary from request to request? How would you like to see it work? i'd love to add this.

chrisclark commented 3 months ago

Would you use e.g. Openrouter to send things to Llama 3.1? What kind of abstraction would be useful to allow you to plug in the LLM of your choice?

Openrouter seems to make a lot of sense to me (and is drop-in compatible with the openai SDK) but definitely would like to learn more.

adRn-s commented 3 months ago

Good idea. I'm planning on doing a v2 of the assistant that will incorporate some features like this.

For the few-shot, would you want to always include the same handful of queries every time you use the assitant, or a UI to quckly select an example to query to send, which would vary from request to request? How would you like to see it work? i'd love to add this.

Yes, I was thinking of a UI. Currently, we can select some columns (the effect of this is unclear to me). But something similar, would be helpful, to give the AI a sense of which are the relationships available in the DB (not only schema, but practical examples.)

Would you use e.g. Openrouter to send things to Llama 3.1? What kind of abstraction would be useful to allow you to plug in the LLM of your choice?

Openrouter seems to make a lot of sense to me (and is drop-in compatible with the openai SDK) but definitely would like to learn more.

Yes, I'm using OpenRouter. But some of the alternatives (Groq, fireworks, glhf.chat, etc.) are in sight too.

As for the abstraction, I'm not sure that I understand. I was just thinking of giving some examples, and see if I get anything that really runs... so far, my experience with it was rather poor. But that's probably on the schema we are using at work. Many column names are a bit ambiguous and there's a lot of bioinformatic specific terms here.

chrisclark commented 3 months ago

Currently, we can select some columns (the effect of this is unclear to me).

It's actually selecting a table. If selected, the schema of the table, as well as 10 random sample rows, will get send to the LLM as additional context. Basically, if you are prompting "write a query showing me sales from the last 10 days" -- and there's nothing to go on in the current query window, you can select the tables you would like the LLM to consider.

If there is already a query in the SQL editor, then a parser will extract any tables in the query and do the same thing (extract schema and sample rows and add them to the prompt). If you have SQL editor content, you add additional tables via the checkboxes.

I'm thinking I'll add two-ish features:

Ability to include specific queries in context to the LLM. Perhaps a flag on queries that is like "eligible for assistant context" and then maybe a simple heuristic that can automatically include relevant queries for relevant requests (with an option of making it explicit -- like how the table includes work).
Ability to annotate tables. A simple way of storing text data associated with each table. You can put whatever you want in here. For example you could just brain-dump some context about what a particular table includes. And that additional data would be included whenever that table is sent up to the LLM.
- 2a. I'd also like to make this inputtable as voice. I think it's way easier to just "talk about the table for a minute" then try to e.g. properly document everything. It dramatically lowers the bar.

chrisclark commented 2 months ago

@adRn-s I have implemented this in a branch here, along with a bunch of other improvements. It's simple, but very effective.

https://github.com/explorerhq/sql-explorer/pull/664

Please let me know if you have any feedback.

adRn-s commented 2 months ago

Great, I will give it a spin and come back to you, probably tomorrow.

chrisclark commented 2 months ago

right on. Let me know if you'd like me to cut a beta release onto pypi or something, or if you are comfortable installing from source. I really do think it's going to work much better, if you but a few minutes of time into adding some annotations, and marking a few example queries. Look forward to hearing!

adRn-s commented 2 months ago

I am having some difficulty with the UI... it's empty (the assistant panel only, I am able to run queries and everything else). The web browser's network panel revealed a 404 for this file/ url: static/explorer/main.5.2.js. Of course, I did run collect static... but it didn't copy anything this time (after installing from assistant-3 branch)... any ideas?

adRn-s commented 2 months ago

I am having some difficulty with the UI... it's empty (the assistant panel only, I am able to run queries and everything else). The web browser's network panel revealed a 404 for this file/ url: static/explorer/main.5.2.js. Of course, I did run collect static... but it didn't copy anything this time (after installing from assistant-3 branch)... any ideas?

Nevermind. Something was wrong from my side. I just re-run the installation from the top, and now collectstatic did copy some files.. :)

adRn-s commented 2 months ago

The UI still seems off... I don't get the new elements. pip show reports Version: 5.1.1

I have added django-sql-explorer @ git+https://github.com/explorerhq/django-sql-explorer@assistant-3 to requirements.txt

adRn-s commented 2 months ago

chrisclark commented 2 months ago

I think you are looking at master. I haven’t merged the changes. Please pull down the code from the branch assistant-3

On Wed, Aug 28, 2024 at 6:28 AM A.s. @.***> wrote:

image.png (view on web) https://github.com/user-attachments/assets/5cc784f4-c9d2-4b33-90d0-384bc5ba770a

— Reply to this email directly, view it on GitHub https://github.com/explorerhq/sql-explorer/issues/657#issuecomment-2314935204, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFJLRG2BYKACM5PK3Z3G4TZTWQ6JAVCNFSM6AAAAABL76WOLCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMJUHEZTKMRQGQ . You are receiving this because you commented.Message ID: @.***>

adRn-s commented 2 months ago

Yeah... I'm back to getting 404 on <URL>/static/explorer/main.5.2.js...

This is the last output I got while installing and activating the app...

  Applying explorer.0026_tabledescription... OK
  Applying explorer.0027_query_few_shot... OK
  Applying explorer.0028_promptlog_database_connection_promptlog_user_request... OK

0 static files copied to '/usr/src/app/staticfiles', 7888 unmodified.

chrisclark commented 2 months ago

For simplicity, I cut a 5.3 beta release. Please try with that:

https://pypi.org/project/django-sql-explorer/5.3b1/

Should "just work".

adRn-s commented 2 months ago

Great, that worked fine. The new beta looks even more powerful!

Regarding the "assistant examples" checkbox, it's a bit misplaced, since it's after the data and that's too far away from the assistant UI. And it's not really clear how it works (I understood it after reading the merged PR description).

In any case, I think the whole design could do better if we get the examples as suggestions (e.g. via the tables that are in common to old queries, specially those that were marked as favorites).

I think a button to pop new UI where you have the history, and add or remove all queries would be nice to use those suggested examples for few-shot prompting.

OTOH, I noticed two details on the UI that could improve:

the history button works, but the UI that it's opened is a bit misaligned (at least in my browser, I can show a screenshot if needed)
clicking the add all tables has no undo, I had to remove all then copy and re-paste the whole query to get the auto-detect. Perhaps this can do a better if there's a button for to auto-detect again.
- I clicked the add all because I was actually expecting a small pop-up window where I could drag and drop columns or something like that... but that's probably just me poking around. What I mean to say, is that a button to add individual columns would be helpful, perhaps just a button that puts the cursor in the textbox with the dropdown open there and ready..
- sorry... this is a bit like bike-shedding. The point is, the textbox area with the pre-selected tables is not immediately clear as the proper way to add more tables. Since it's pre-filled, people might go directly to the "+" button. That's what I did, at least.

chrisclark commented 2 months ago

This is incredible feedback - thank you! I will respond in more detail later today, and work on some of this for a beta 2 this weekend. Cheers!

p.s. The "trash" icon should remove all selected tables. Was that 1) Unclear or 2) Not working? I agree that that UI is not super obvious.

adRn-s commented 2 months ago

Oh, I wasn't clear. The trash icon did work, and then I went with copy and pasting the query again to get the auto-detection which is just great. The confusing part was the "+" sign adding every table when I wanted to just get pop-up UI or something like the thing we get when placing the cursor in the text area of these.

p.s. Glad to provide feedback, this is an amazing app! 🥇

chrisclark commented 2 months ago

I made a bunch of improvements to the UI that should make this much more obvious for users.

The "select all" icon should be more obvious, and that 'refresh' icon will re-sync the autodetect of relevant tables. There is now also helptext in the table selection area. Additionally, I made the autodetect case-insensitive which should help.

The 'assistant example' checkbox has been related, and I added a tooltip with additional help text.

I have not dealt with auto-suggesting few-shot examples. I like the idea, but don't want to hold up this version. I will likely add it in the future.

Thanks again for the great feedback! I'll get a beta2 out soon, and then hopefully a 5.3 release in a day or two.

adRn-s commented 2 months ago

Great, those changes indeed look neat! I will give beta2 a spin, (and come back if there're any serious issues that I deem relevant.) Thanks a lot :)

chrisclark commented 2 months ago

5.3b2 is released!

explorerhq / sql-explorer

AI 128k context would allow us to send few-shots prompts #657