search

continuedev / continue

⏩ Continue is the leading open-source AI code assistant. You can connect any models and any context to build custom autocomplete and chat experiences inside VS Code and JetBrains
https://docs.continue.dev/
Apache License 2.0
15.83k stars 1.2k forks source link

Telemetry sent to posthog even with "allowAnonymousTelemetry: false" #2082

Open Xav-Pe opened 3 weeks ago

Xav-Pe commented 3 weeks ago

Before submitting your bug report

Relevant environment info

- OS:RHEL7
- Continue:0.7.64
- IDE:VSCODE
- Model:all
- config.json:

{
  "models": [
    {
      "model": "gpt-4o-mini",
      "title": "gpt-4o-mini",
      "apiKey": "sk-",
      "completionOptions": {},
      "provider": "openai"
    },
    {
      "title": "mistral",
      "provider": "mistral",
      "model": "mistral-small",
      "apiKey":"myapikey"
    }
  ],
  "slashCommands": redacted
  "customCommands": [
    {
      "name": "test",
      "prompt": "Write a comprehensive set of unit tests for the selected code. It should setup, run tests that check for correctness including important edge cases, and teardown. Ensure that the tests are complete and sophisticated. Give the tests just as chat output, don't edit any file.",
      "description": "Write unit tests for highlighted code"
    }
  ],
  "contextProviders":redacted,
  "tabAutocompleteModel": {
    "title": "openai autocomplete",
    "provider": "openai",
    "apiKey": "sk-",
    "model": "gpt-4o-mini"
  },
  "tabAutocompleteOptions": {
    "useCopyBuffer": false,
    "maxPromptTokens": 400
  },
  "allowAnonymousTelemetry": false,
  "docs": []
}

Description

While trying to get the VScode extention to work on RHEL7 behind a proxy (that's why I use 7.64, 8.xx needs libc CXXABI_1.3.8 not available on my system), I saw errors on console. Remember I have "allowAnonymousTelemetry": false and according to https://github.com/continuedev/continue/issues/1414 it's the telemetry which should be deactivated with this parameter.

So here is the console output:

POST https://app.posthog.com/e/?ip=1&_=1724401619208&ver=1.75.2 net::ERR_FAILED 403 (Forbidden)
[Extension Host] TypeError: fetch failed
    at Object.fetch (node:internal/deps/undici/undici:11413:11)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Mistral._streamChat (/home/admxpe@z2.r02.local/.vscode/extensions/continue.continue-0.7.64-linux-x64/out/extension.js:35185:26)
    at async Mistral._streamComplete (/home/admxpe@z2.r02.local/.vscode/extensions/continue.continue-0.7.64-linux-x64/out/extension.js:35117:26)
    at async Mistral.streamChat (/home/admxpe@z2.r02.local/.vscode/extensions/continue.continue-0.7.64-linux-x64/out/extension.js:34493:30)
    at async f.value (/home/admxpe@z2.r02.local/.vscode/extensions/continue.continue-0.7.64-linux-x64/out/extension.js:54352:28)

So I checked the network tab to see this request:

POST /e/?ip=1&_=1724401619208&ver=1.75.2 HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US
Connection: keep-alive
Content-Length: 3721
Content-Type: application/x-www-form-urlencoded
Host: app.posthog.com
Origin: vscode-webview://0krfondcgektnmfi87mkr15a0d662jtehog5o6t16ot2me042gp6
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: cross-site
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Code/1.85.1 Chrome/114.0.5735.289 Electron/25.9.7 Safari/537.36
sec-ch-ua: "Not.A/Brand";v="8", "Chromium";v="114"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Linux"

With a base64 payload containing telemetry usage:

[
    {
        "uuid": "01917e50-38da-748a-9a60-8d2d4c52c6bb",
        "event": "step run",
        "properties": {
            "$os": "Linux",
            "$browser": "Chrome",
            "$device_type": "Desktop",
            "$current_url": "vscode-webview://0krfondcgektnmfi87mkr15a0d662jtehog5o6t16ot2me042gp6/index.html?id=c6c698e6-6776-4575-9734-0ac0bee38821&origin=d8894425-3e8c-4c6f-ad9a-0fe2f35869aa&swVersion=4&extensionId=Continue.continue&platform=electron&vscode-resource-base-authority=vscode-resource.vscode-cdn.net&parentOrigin=vscode-file%3A%2F%2Fvscode-app&purpose=webviewView",
            "$host": "REDACTED",
            "$pathname": "/index.html",
            "$browser_version": 114,
            "$browser_language": "en-US",
            "$screen_height": 1080,
            "$screen_width": 1920,
            "$viewport_height": 895,
            "$viewport_width": 311,
            "$lib": "web",
            "$lib_version": "1.75.2",
            "$insert_id": "REDACTED",
            "$time": 1724401072.347,
            "distinct_id": "REDACTED",
            "$device_id": "REDACTED",
            "$user_id": "REDACTED",
            "$referrer": "$direct",
            "$referring_domain": "$direct",
            "step_name": "User Input",
            "params": {
                "user_input": "hello"
            },
            "token": "REDACTED",
            "$session_id": "REDACTED",
            "$window_id": "REDACTED,
            "$pageview_id": "REDACTED"
        },
        "offset": 3001
    },
    {
        "uuid": "REDACTED,
        "event": "userInput",
        "properties": {
            "$os": "Linux",
            "$browser": "Chrome",
            "$device_type": "Desktop",
            "$current_url": "vscode-webview://0krfondcgektnmfi87mkr15a0d662jtehog5o6t16ot2me042gp6/index.html?id=c6c698e6-6776-4575-9734-0ac0bee38821&origin=d8894425-3e8c-4c6f-ad9a-0fe2f35869aa&swVersion=4&extensionId=Continue.continue&platform=electron&vscode-resource-base-authority=vscode-resource.vscode-cdn.net&parentOrigin=vscode-file%3A%2F%2Fvscode-app&purpose=webviewView",
            "$host": "REDACTED",
            "$pathname": "/index.html",
            "$browser_version": 114,
            "$browser_language": "en-US",
            "$screen_height": 1080,
            "$screen_width": 1920,
            "$viewport_height": 895,
            "$viewport_width": 311,
            "$lib": "web",
            "$lib_version": "1.75.2",
            "$insert_id": "REDACTED",
            "$time": 1724401072.347,
            "distinct_id": "REDACTED",
            "$device_id": "01917e4aREDACTED",
            "$user_id": "REDACTED",
            "$referrer": "$direct",
            "$referring_domain": "$direct",
            "input": "hello",
            "token": "REDACTED",
            "$session_id": "REDACTED",
            "$window_id": "REDACTED",
            "$pageview_id": "REDACTED"
        },
        "offset": 3001
    }
]

So, is sent to posthog all my informations, including my input.

Two questions:

All these logs and the fact that the newer versions hides the network traffic makes me concerned about the privacy of this extension.

To reproduce

  1. install extension v0.7.x on vscode
  2. disable telemetry
  3. open network tab
  4. run a query and watch the request being send to posthog

on newer version:

  1. install latest extension on vscode
  2. open network tab
  3. run a query to the LLM, and observe that no network query are shown.

Log output

No response

sestinj commented 3 weeks ago

@Xav-Pe Thank you for bringing this up—privacy is a pretty big deal for us, so definitely want to make sure we get this right.

Is it expected?

No, it is not expected that we would send telemetry when disabled on version 0.7.64, or in any version. Since the beginning of Continue we have had the same allowAnonymousTelemetry configuration option that is intended to stop all requests. That said, we no longer have control over the code that has been published in older versions, and it is possible that there is simply a bug in that version. While this is far from ideal, it's not something we can fix.

As for the fact that your input was sent, this is expected in a few much older versions like 0.7.64, but we do not send any prompts or code in any versions that have been released for at least the last many months.

why, on another computer running latest extention (0.9.197), the network tab is not logging the requests anymore?

Most likely the requests are not being logged because they are not being sent at all. If you have disabled telemetry we would not send anything

sestinj commented 3 weeks ago

I think in order to find a solution the first thing I would try is to double check by reloading the VS Code window after disabling telemetry

Another thing you can try is using the VS Code setting to disable telemetry