search

continuedev / continue

⏩ Continue is the leading open-source AI code assistant. You can connect any models and any context to build custom autocomplete and chat experiences inside VS Code and JetBrains
https://docs.continue.dev/
Apache License 2.0
19.5k stars 1.69k forks source link

@Docs not works properly #2977

Open dalisoft opened 6 days ago

dalisoft commented 6 days ago

Before submitting your bug report

Relevant environment info

- OS: macOS 15.1
- Continue version: v0.8.57
- IDE version: VSCode v1.95.3
- Model: `Qwen-2.5-Coder-32B-Instruct`
{
  "docs": [
    {
      "title": "Drizzle ORM",
      "startUrl": "https://orm.drizzle.team/docs/overview",
      "rootUrl": "https://orm.drizzle.team/docs/",
      "favicon": "https://orm.drizzle.team/favicon.ico",
      "maxDepth": 10
    },
    {
      "title": "LibSQL",
      "startUrl": "https://docs.turso.tech/libsql",
      "rootUrl": "https://docs.turso.tech/",
      "favicon": "https://mintlify.s3-us-west-1.amazonaws.com/turso/_generated/favicon/favicon.ico?v=3",
      "maxDepth": 10
    },
    {
      "title": "LibSQL SDK",
      "startUrl": "https://docs.turso.tech/sdk/introduction",
      "rootUrl": "https://docs.turso.tech/sdk/",
      "favicon": "https://mintlify.s3-us-west-1.amazonaws.com/turso/_generated/favicon/favicon.ico?v=3",
      "maxDepth": 10
    },
    {
      "title": "Turso CLI",
      "startUrl": "https://docs.turso.tech/cli/introduction",
      "rootUrl": "https://docs.turso.tech/cli/",
      "favicon": "https://mintlify.s3-us-west-1.amazonaws.com/turso/_generated/favicon/favicon.ico?v=3",
      "maxDepth": 10
    },
    {
      "title": "sqlite-vec",
      "startUrl": "https://alexgarcia.xyz/sqlite-vec/introduction.html",
      "rootUrl": "https://alexgarcia.xyz/sqlite-vec/",
      "favicon": "https://alexgarcia.xyz/sqlite-vec/logo.light.svg",
      "maxDepth": 10
    }
  ],
  "mergeBehavior": "merge"
}

Description

It should re-index, even after manual re-index it does not work. When adding docs to config.json, it removes each entry but does not adds to context on request

To reproduce

  1. Add file .continuerc.json to not overwrite your config.json
  2. Press CMD+Shift+P
  3. Enter Continue: Force Docs Re-Index
  4. Try use chat with contexts
  5. Try sample query: How i can extend Drizzle to support Turso’s native vector in TypeScript? @Docs LibSQL SDK

It should response similar response from https://docs.turso.tech/sdk/ts/orm/drizzle#vector-embeddings

Log output

==========================================================================
==========================================================================
##### Completion options #####
{
  "contextLength": 131072,
  "stream": true,
  "temperature": 0.7,
  "maxTokens": 8192,
  "keepAlive": 300,
  "useMmap": true,
  "top_p": 0.8,
  "repetition_penalty": 1.05,
  "model": "Qwen/Qwen2.5-Coder-32B-Instruct"
}

##### Request options #####
{}

##### Prompt #####
<user>
Turso SDKs are fully compatible with libSQL, so you can use the same SDK to connect to a local database (SQLite), libSQL server, a remote database, or an embedded replica.These SDKs are community-driven and don’t come with official Turso support.

Use the above documentation to answer the following question. You should not reference anything outside of what is shown, unless it is a commonly known concept. Reference URLs whenever possible using markdown formatting. If there isn't enough information to answer the question, suggest where the user might look to learn more.

How i can extend Drizzle to support Turso’s native vector in TypeScript?
LibSQL SDK

VSCode Developer Tools

Extension Host] Creating embeddings for 1 articles
console.ts:137 [Extension Host] Adding 7 embeddings to db
console.ts:137 [Extension Host] Deleting old embeddings
console.ts:137 [Extension Host] Indexing new doc: https://docs.continue.dev/intro
console.ts:137 [Extension Host] Successfully indexed: https://orm.drizzle.team/docs/overview
console.ts:137 [Extension Host] Creating embeddings for 1 articles
log.ts:439   ERR [Extension Host] No embeddings were created for site: https://docs.continue.dev/intro
 Num chunks: 0
console.ts:137 [Extension Host] No embeddings were created for site: https://docs.continue.dev/intro
 Num chunks: 0
uGi @ console.ts:137
$logExtensionHostMessage @ mainThreadConsole.ts:39
S @ rpcProtocol.ts:458
Q @ rpcProtocol.ts:443
M @ rpcProtocol.ts:373
L @ rpcProtocol.ts:299
(anonymous) @ rpcProtocol.ts:161
B @ event.ts:1243
fire @ event.ts:1274
fire @ ipc.net.ts:652
l.onmessage @ localProcessExtensionHost.ts:378
console.ts:137 [Extension Host] Indexing new doc: https://orm.drizzle.team/docs/overview
console.ts:137 [Extension Host] Indexing new doc: https://docs.turso.tech/cli/introduction
console.ts:137 [Extension Host] Creating embeddings for 1 articles
console.ts:137 [Extension Host] Adding 5 embeddings to db
console.ts:137 [Extension Host] Deleting old embeddings
console.ts:137 [Extension Host] Creating embeddings for 1 articles
log.ts:439   ERR [Extension Host] No embeddings were created for site: https://docs.turso.tech/cli/introduction
 Num chunks: 0
console.ts:137 [Extension Host] No embeddings were created for site: https://docs.turso.tech/cli/introduction
 Num chunks: 0
uGi @ console.ts:137
$logExtensionHostMessage @ mainThreadConsole.ts:39
S @ rpcProtocol.ts:458
Q @ rpcProtocol.ts:443
M @ rpcProtocol.ts:373
L @ rpcProtocol.ts:299
(anonymous) @ rpcProtocol.ts:161
B @ event.ts:1243
fire @ event.ts:1274
fire @ ipc.net.ts:652
l.onmessage @ localProcessExtensionHost.ts:378
console.ts:137 [Extension Host] Indexing new doc: https://alexgarcia.xyz/sqlite-vec/introduction.html
console.ts:137 [Extension Host] Indexing new doc: https://docs.continue.dev/intro
console.ts:137 [Extension Host] Creating embeddings for 1 articles
log.ts:439   ERR [Extension Host] No embeddings were created for site: https://alexgarcia.xyz/sqlite-vec/introduction.html
 Num chunks: 0
console.ts:137 [Extension Host] No embeddings were created for site: https://alexgarcia.xyz/sqlite-vec/introduction.html
 Num chunks: 0
uGi @ console.ts:137
$logExtensionHostMessage @ mainThreadConsole.ts:39
S @ rpcProtocol.ts:458
Q @ rpcProtocol.ts:443
M @ rpcProtocol.ts:373
L @ rpcProtocol.ts:299
(anonymous) @ rpcProtocol.ts:161
B @ event.ts:1243
fire @ event.ts:1274
fire @ ipc.net.ts:652
l.onmessage @ localProcessExtensionHost.ts:378
console.ts:137 [Extension Host] Creating embeddings for 1 articles
log.ts:439   ERR [Extension Host] No embeddings were created for site: https://docs.continue.dev/intro
 Num chunks: 0
console.ts:137 [Extension Host] No embeddings were created for site: https://docs.continue.dev/intro
 Num chunks: 0
uGi @ console.ts:137
$logExtensionHostMessage @ mainThreadConsole.ts:39
S @ rpcProtocol.ts:458
Q @ rpcProtocol.ts:443
M @ rpcProtocol.ts:373
L @ rpcProtocol.ts:299
(anonymous) @ rpcProtocol.ts:161
B @ event.ts:1243
fire @ event.ts:1274
fire @ ipc.net.ts:652
l.onmessage @ localProcessExtensionHost.ts:378
console.ts:137 [Extension Host] Indexing new doc: https://docs.turso.tech/libsql
console.ts:137 [Extension Host] Already in queue
console.ts:137 [Extension Host] Indexing new doc: https://docs.turso.tech/cli/introduction
console.ts:137 [Extension Host] Creating embeddings for 1 articles
log.ts:439   ERR [Extension Host] No embeddings were created for site: https://docs.turso.tech/cli/introduction
 Num chunks: 0
console.ts:137 [Extension Host] No embeddings were created for site: https://docs.turso.tech/cli/introduction
 Num chunks: 0
uGi @ console.ts:137
$logExtensionHostMessage @ mainThreadConsole.ts:39
S @ rpcProtocol.ts:458
Q @ rpcProtocol.ts:443
M @ rpcProtocol.ts:373
L @ rpcProtocol.ts:299
(anonymous) @ rpcProtocol.ts:161
B @ event.ts:1243
fire @ event.ts:1274
fire @ ipc.net.ts:652
l.onmessage @ localProcessExtensionHost.ts:378
console.ts:137 [Extension Host] Indexing new doc: https://alexgarcia.xyz/sqlite-vec/introduction.html
console.ts:137 [Extension Host] Successfully indexed: https://docs.turso.tech/libsql
console.ts:137 [Extension Host] Creating embeddings for 1 articles
log.ts:439   ERR [Extension Host] No embeddings were created for site: https://alexgarcia.xyz/sqlite-vec/introduction.html
 Num chunks: 0
console.ts:137 [Extension Host] No embeddings were created for site: https://alexgarcia.xyz/sqlite-vec/introduction.html
 Num chunks: 0
uGi @ console.ts:137
$logExtensionHostMessage @ mainThreadConsole.ts:39
S @ rpcProtocol.ts:458
Q @ rpcProtocol.ts:443
M @ rpcProtocol.ts:373
L @ rpcProtocol.ts:299
(anonymous) @ rpcProtocol.ts:161
B @ event.ts:1243
fire @ event.ts:1274
fire @ ipc.net.ts:652
l.onmessage @ localProcessExtensionHost.ts:378
console.ts:137 [Extension Host] Creating embeddings for 1 articles
console.ts:137 [Extension Host] Adding 1 embeddings to db
console.ts:137 [Extension Host] Deleting old embeddings
console.ts:137 [Extension Host] Indexing new doc: https://docs.continue.dev/intro
console.ts:137 [Extension Host] Successfully indexed: https://docs.turso.tech/sdk/introduction
console.ts:137 [Extension Host] Creating embeddings for 1 articles
log.ts:439   ERR [Extension Host] No embeddings were created for site: https://docs.continue.dev/intro
 Num chunks: 0
console.ts:137 [Extension Host] No embeddings were created for site: https://docs.continue.dev/intro
 Num chunks: 0
uGi @ console.ts:137
$logExtensionHostMessage @ mainThreadConsole.ts:39
S @ rpcProtocol.ts:458
Q @ rpcProtocol.ts:443
M @ rpcProtocol.ts:373
L @ rpcProtocol.ts:299
(anonymous) @ rpcProtocol.ts:161
B @ event.ts:1243
fire @ event.ts:1274
fire @ ipc.net.ts:652
l.onmessage @ localProcessExtensionHost.ts:378
console.ts:137 [Extension Host] Indexing new doc: https://docs.turso.tech/sdk/introduction
console.ts:137 [Extension Host] Indexing new doc: https://docs.turso.tech/cli/introduction
console.ts:137 [Extension Host] Creating embeddings for 1 articles
log.ts:439   ERR [Extension Host] No embeddings were created for site: https://docs.turso.tech/cli/introduction
 Num chunks: 0
console.ts:137 [Extension Host] No embeddings were created for site: https://docs.turso.tech/cli/introduction
 Num chunks: 0
uGi @ console.ts:137
$logExtensionHostMessage @ mainThreadConsole.ts:39
S @ rpcProtocol.ts:458
Q @ rpcProtocol.ts:443
M @ rpcProtocol.ts:373
L @ rpcProtocol.ts:299
(anonymous) @ rpcProtocol.ts:161
B @ event.ts:1243
fire @ event.ts:1274
fire @ ipc.net.ts:652
l.onmessage @ localProcessExtensionHost.ts:378
console.ts:137 [Extension Host] Indexing new doc: https://alexgarcia.xyz/sqlite-vec/introduction.html
console.ts:137 [Extension Host] Creating embeddings for 1 articles
log.ts:439   ERR [Extension Host] No embeddings were created for site: https://alexgarcia.xyz/sqlite-vec/introduction.html
 Num chunks: 0
console.ts:137 [Extension Host] No embeddings were created for site: https://alexgarcia.xyz/sqlite-vec/introduction.html
 Num chunks: 0
uGi @ console.ts:137
$logExtensionHostMessage @ mainThreadConsole.ts:39
S @ rpcProtocol.ts:458
Q @ rpcProtocol.ts:443
M @ rpcProtocol.ts:373
L @ rpcProtocol.ts:299
(anonymous) @ rpcProtocol.ts:161
B @ event.ts:1243
fire @ event.ts:1274
fire @ ipc.net.ts:652
l.onmessage @ localProcessExtensionHost.ts:378

Similar issues

RomneyDa commented 6 days ago

Hi @dalisoft !

Big docs UI and indexing makeover PR just opened here https://github.com/continuedev/continue/pull/2972

Believe this should fix, check next prerelease!

dalisoft commented 3 days ago

@RomneyDa I tested in pre-release and same bug. Bug i found a workaround

{
  "docs": [
    {
      "title": "Drizzle ORM",
-     "startUrl": "https://orm.drizzle.team/docs/overview",
+     "startUrl": "https://orm.drizzle.team/docs",
      "rootUrl": "https://orm.drizzle.team/docs",
      "favicon": "https://orm.drizzle.team/favicon.ico",
      "maxDepth": 10
    },
    {
      "title": "LibSQL",
      "startUrl": "https://docs.turso.tech/libsql",
      "rootUrl": "https://docs.turso.tech/",
      "favicon": "https://mintlify.s3-us-west-1.amazonaws.com/turso/_generated/favicon/favicon.ico?v=3",
      "maxDepth": 10
    },
    {
      "title": "LibSQL SDK",
-     "startUrl": "https://docs.turso.tech/sdk/introduction",
+     "startUrl": "https://docs.turso.tech/sdk",
      "rootUrl": "https://docs.turso.tech/sdk",
      "favicon": "https://mintlify.s3-us-west-1.amazonaws.com/turso/_generated/favicon/favicon.ico?v=3",
      "maxDepth": 10
    },
    {
      "title": "Turso CLI",
      "startUrl": "https://docs.turso.tech/cli/introduction",
      "rootUrl": "https://docs.turso.tech/cli",
      "favicon": "https://mintlify.s3-us-west-1.amazonaws.com/turso/_generated/favicon/favicon.ico?v=3",
      "maxDepth": 10
    },
    {
      "title": "sqlite-vec",
-     "startUrl": "https://alexgarcia.xyz/sqlite-vec/introduction.html",
+     "startUrl": "https://alexgarcia.xyz/sqlite-vec",
      "rootUrl": "https://alexgarcia.xyz/sqlite-vec",
      "favicon": "https://alexgarcia.xyz/sqlite-vec/logo.light.svg",
      "maxDepth": 10
    }
  ],
  "mergeBehavior": "merge"
}

This helps me and now works