[Ollama API] Integrate with a Local LLM via API

[drupol]

Hello,

This extension is using ChatGPT (OpenAI) by default. Would it be a good idea to let users customize which service to use (must be compatible with OpenAI API) so that we could use a local LLM service? We should also be able to specify with LLM to use.

I already have my own Ollama server, and I'd like to use it.

Thanks.

[micz]

I'm planning to add the ability to call the ChatGPT API (#40) and also integrate a local LLM. I am considering Ollama as a solution for the local LLM.

This will be added to milestone 2.1 as a new feature.

Thank you.

[drupol]

Yes, I was thinking about Ollama. Looking forward for this cool feature ! :)

[micz]

I released a preview of this feature, you can find it here: https://github.com/micz/ThunderAI/releases/tag/v2.1.0pre1

@drupol may you test it and give feedback? Please backup your custom prompts before trying this pre-release. Thank you.

[drupol]

I tested it just now and I was not able to get the list of models.

[micz]

May you post the console log?

[drupol]

Argh, CORS issue!

[micz]

Ok, I forgot about it, sorry. You need to add this OLLAMA_ORIGINS=moz-extension://* to the ollama server environment variables.

[drupol]

It worked! I succeeded to get the list of models.

However, after selecting one, I got a crash, here's the report:

AdapterDeviceID: 0x9a49
AdapterDriverVendor: mesa/iris
AdapterDriverVersion: 24.1.5.0
AdapterVendorID: 0x8086
Add-ons: fr-dicollecte%40dictionaries.addons.mozilla.org:6.3.1webext,thunderai%40micz.it:2.1.0pre1,google%40search.mozilla.org:1.0,amazondotcom%40search.mozilla.org:1.1,bing%40search.mozilla.org:1.0,ddg%40search.mozilla.org:1.0,wikipedia%40search.mozilla.org:1.0,thunderbird-compact-dark%40mozilla.org:1.3
AvailablePageFile: 0
AvailablePhysicalMemory: 1176666112
AvailableSwapMemory: 6200197120
AvailableVirtualMemory: 4752605184
BackgroundTaskMode: 0
BuildID: 20240730200333
ContentSandboxCapabilities: 119
ContentSandboxCapable: 1
ContentSandboxLevel: 0
CrashTime: 1724325063
DOMFissionEnabled: 1
DOMIPCEnabled: 1
DesktopEnvironment: kde
EMCheckCompatibility: true
GpuSandboxLevel: 0
GraphicsCriticalError: |[0][GFX1-]: Wayland protocol error: error marshalling arguments for start_drag (signature ?oo?ou): null value passed for arg 1
 (t=4399.45) 
GraphicsNumActiveRenderers: 2
GraphicsNumRenderers: 1
HeadlessMode: 0
InstallTime: 1723559684
IsWayland: 1
LastStartupWasCrash: 0
MozCrashReason: error marshalling arguments for start_drag (signature ?oo?ou): null value passed for arg 1

Notes: NixOS 24.11 (Vicuna)FP(D00-L1000-W0000000-T01) WR? WR+ EGL? EGL- GL Context? GL Context+ 
ProductID: {3550f703-e582-4d05-9a08-453d09bdfdc6}
ProductName: Thunderbird
ReleaseChannel: default
SafeMode: 0
SecondsSinceLastCrash: 4904572
StartupCacheValid: 0
StartupCrash: 0
StartupTime: 1724320663
SubmittedFrom: Client
TelemetryClientId: c556ff9c-0f8c-4e7b-80fd-95718355ec3f
TelemetryEnvironment: {"build":{"applicationId":"{3550f703-e582-4d05-9a08-453d09bdfdc6}","applicationName":"Thunderbird","architecture":"x86-64","buildId":"20240730200333","version":"128.1.0","vendor":null,"displayVersion":"128.1.0esr","platformVersion":"128.1.0","xpcomAbi":"x86_64-gcc3","updaterAvailable":false},"partner":{"distributionId":null,"distributionVersion":null,"partnerId":null,"distributor":null,"distributorChannel":null,"partnerNames":[]},"system":{"memoryMB":15702,"virtualMaxMB":null,"cpu":{"count":8,"cores":4,"vendor":"GenuineIntel","name":"11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz","family":6,"model":140,"stepping":1,"l2cacheKB":1280,"l3cacheKB":12288,"speedMHz":4800,"extensions":["hasMMX","hasSSE","hasSSE2","hasSSE3","hasSSSE3","hasSSE4_1","hasSSE4_2","hasAVX","hasAVX2","hasAES"]},"os":{"name":"Linux","version":"6.10.4","locale":"en-US","distro":"NixOS","distroVersion":"24.11"},"hdd":{"profile":{"model":null,"revision":null,"type":null},"binary":{"model":null,"revision":null,"type":null},"system":{"model":null,"revision":null,"type":null}},"gfx":{"D2DEnabled":null,"DWriteEnabled":null,"ContentBackend":"Skia","Headless":false,"EmbeddedInFirefoxReality":null,"TargetFrameRate":60,"adapters":[{"description":"Mesa Intel(R) Xe Graphics (TGL GT2)","vendorID":"0x8086","deviceID":"0x9a49","subsysID":null,"RAM":0,"driver":null,"driverVendor":"mesa/iris","driverVersion":"24.1.5.0","driverDate":null,"GPUActive":true}],"monitors":[{"screenWidth":3440,"screenHeight":1440},{"screenWidth":1920,"screenHeight":1200}],"features":{"compositor":"webrender","hwCompositing":{"status":"available"},"gpuProcess":{"status":"unused"},"webrender":{"status":"available"},"wrCompositor":{"status":"blocked:FEATURE_FAILURE_DISABLE_RELEASE_OR_BETA"},"openglCompositing":{"status":"available"},"omtp":{"status":"unused"}}},"appleModelId":null,"hasWinPackageId":null},"settings":{"blocklistEnabled":true,"e10sEnabled":true,"e10sMultiProcesses":8,"fissionEnabled":true,"telemetryEnabled":false,"locale":"en-US","intl":{"requestedLocales":["en-US"],"availableLocales":["en-US"],"appLocales":["en-US"],"systemLocales":["en-US"],"regionalPrefsLocales":["fr-BE"],"acceptLanguages":["en-US","en"]},"update":{"channel":"default","enabled":false,"autoDownload":false,"background":false},"userPrefs":{"browser.cache.disk.capacity":1048576,"browser.search.region":"BE","general.config.filename":"<set>","network.trr.strict_native_fallback":false,"privacy.donottrackheader.enabled":true,"widget.content.gtk-high-contrast.enabled":true},"sandbox":{"effectiveContentProcessLevel":0,"contentWin32kLockdownState":3},"addonCompatibilityCheckEnabled":true,"isDefaultBrowser":null,"defaultSearchEngine":"google","defaultSearchEngineData":{"loadPath":"[app]google@search.mozilla.org","name":"Google","origin":"default","submissionURL":"https://www.google.com/search?q="}},"profile":{"creationDate":18785,"firstUseDate":18785},"addons":{"activeAddons":{"fr-dicollecte@dictionaries.addons.mozilla.org":{"version":"6.3.1webext","scope":1,"type":"dictionary","updateDay":19348,"isSystem":false,"isWebExtension":true,"multiprocessCompatible":true,"blocklisted":false,"description":"Spelling dictionary for the French language.","name":"French spelling dictionary","userDisabled":false,"appDisabled":false,"foreignInstall":false,"hasBinaryComponents":false,"installDay":19348,"quarantineIgnoredByApp":false,"quarantineIgnoredByUser":false},"thunderai@micz.it":{"version":"2.1.0pre1","scope":1,"type":"extension","updateDay":19957,"isSystem":false,"isWebExtension":true,"multiprocessCompatible":true,"blocklisted":false,"description":"Use ChatGPT to enhance you emails!","name":"ThunderAI","userDisabled":false,"appDisabled":false,"foreignInstall":false,"hasBinaryComponents":false,"installDay":19957,"signedState":0,"signedTypes":"[]","quarantineIgnoredByApp":false,"quarantineIgnoredByUser":false}},"theme":{"id":"thunderbird-compact-dark@mozilla.org","blocklisted":false,"description":"A theme with a dark color scheme.","name":"Dark","userDisabled":false,"appDisabled":false,"version":"1.3","scope":4,"foreignInstall":false,"hasBinaryComponents":false,"installDay":18871,"updateDay":18871},"activeGMPlugins":{"dummy-gmp":{"version":"0.1","userDisabled":false,"applyBackgroundUpdates":1}}}}
TelemetryServerURL: https://incoming-telemetry.thunderbird.net
TelemetrySessionId: ff420c40-2d0c-4325-ae56-3deb8399f9ec
Throttleable: 1
TotalPageFile: 24716304384
TotalPhysicalMemory: 16464793600
UptimeTS: 4399.55171445
Version: 128.1.0
useragent_locale: en-US
This report also contains technical information about the state of the application when it crashed.

[micz]

Did Thunderbird crashed when you choose a model in the list in the option page?

[drupol]

Looks like I got it working!

[drupol]

Did Thunderbird crashed when you choose a model in the list in the option page?

Yes. Then I restarted and apparently, the proper model was selected correctly.

[micz]

I suspect it could be something related to you OS and Thunderbird. It never occurred to me on Windows. I'll try to replicate the problem.

[micz]

Looks like I got it working!

Great!!

[micz]

Looks like I got it working!

In the console there is an error, may you send it to me? When did it occurred? which version of Thunderbird are you using?

[drupol]

I can't reproduce it. I think it was unrelated. I'm using version 128.1.0esr

[micz]

Did it happen that the chat window opened but nothing was written inside it?

[drupol]

I think it happened, I left the office, I can't really tell you more at the moment, I'll be back tonight or tomorrow.

[micz]

I think it happened, I left the office, I can't really tell you more at the moment, I'll be back tonight or tomorrow.

I got it [#117], I don't need more data. Thanks.

[micz]

I released 2.1.0pre2.

Please read the release notes, I fixed a few glitches and make some improvements. I think it could be ready for release. @drupol may you test it? Thanks.

[drupol]

Works very well, nice improvements! Looking forward for the final release :)

I notice that the prompts "Reply polite" is no more available, even after selecting some text.

[micz]

Works very well, nice improvements! Looking forward for the final release :)

Great, thanks!

I notice that the prompts "Reply polite" is no more available, even after selecting some text.

It should be available only when composing a message and it requires to select a text.

Isn't it not present in the compose window?

[drupol]

Indeed, in the compose window, it works, but not when replying. Maybe it should be also enabled there?

[micz]

Indeed, in the compose window, it works, but not when replying. Maybe it should be also enabled there?

The prompt asks for a rewrite, not a reply, I think it could not work in a reply.

You could add it in your Thunderbird using a custom prompts.

[micz]

Released in version 2.1.0.

[drupol]

excellent, thanks!