✨ feat(settings): add local llm support

[rshmhrj]

@ozdemir08 took a first stab at getting a local LLM working

• created AiClient interface
• made existing OpenAiClient implement interface -created new OllamaClient to call local LLM (defaulting to llama2)
• created settings for defaults
• created toggle button for enabling local llm usage
• rewired modal to use and pass AiClient
• rewired TranscriptSummarizer to use interface

TODO: still need to test 'request' and parsing from ollama -- right now computer is extremely slow and hanging when I try to call it locally

Local LLM not toggled:

Local LLM toggled on:

setup steps:

• install ollama here: https://ollama.com/download
• ollama pull llama2
• tested LLM works via ollama run llama2
• if local server not running, start with ollama serve
• curled test request:

  curl http://localhost:11434/api/generate -d '
  {
  "model": "llama2",
  "prompt": "Would you like to talk?",
  "stream": false
  }' | jq .response

• trying to use obsidian request to make local REST call, but not having luck atm
• okay, it finally came back after like 10-15 mins -- still need to pull only the response and format it properly:

  {
  "model": "llama2",
  "created_at": "2024-03-31T05:44:19.780342Z",
  "response": "In this video, we are going over how to configure Webpack for a single-spa application. A single-spa application is a web application that uses a single entry point for all of its features. We are going to cover the following topics:\n\n1. Disabling host checks: Webpack Dev Server tries to set up WebSocket or HTTP polling in order to get push notifications when saving and recompiling the code. However, this can fail some host checks, so we need to disable them for localhost.\n2. Solving warnings and errors: We are seeing yellow and red warnings and errors on a loop in the configuration. We need to set the `sockHost` and `sockPort` options in the `webpack.config.js` file.\n3. Optimizations: The optimizations block gives us a chance to split up the output bundles into multiple files. However, sometimes it assumes that we are injecting script tags into the HTML file with the `index.html` plugin, which can be tricky. By deleting the optimizations block, we are choosing how to do code splitting in a different way.\n4. Public path: The public path tells Webpack where to download all of our code splits from. We need to install the `systemjs-webpack-interop` library and create a file called `setPublicPath.js` in the documentation for `systemjs-webpack`.\n\nHere is the updated configuration after making these changes:\n```js\nmodule.exports = {\n  //... other options ...\n  optimization: {\n    splitChunks: {\n      chunks: 'all',\n      minSize: 10000,\n      minOf: 2\n    },\n    chunks: {\n      colors: {\n        test: /[^\\/]+\\.js$/,\n        include: []\n      }\n    }\n  },\n  publicPath: 'http://localhost:'\n};\n```\nIn this configuration, we have set `optimization` to `splitChunks` to enable code splitting. We have also set the `chunks` option to specify that the `colors` chunk should be included in the output. Finally, we have set the `publicPath` option to `http://localhost:`, which tells Webpack where to download all of our code splits from.",
  "done": true,
  "context": [518,25580,29962,3532,14816,29903,29958,5299,829,14816,29903,6778,13,13,21140,340,338,263,1301,924,310,263,612,15907,4863,29889,1815,366,2367,592,29871,29953,1820,3291,322,736,278,2582,408,385,443,21693,2791,3204,1051,29973,29871,13,11474,29871,13,262,445,4863,306,29915,29885,2675,304,748,975,920,22813,9306,763,1856,4870,322,9679,29899,786,526,1304,297,278,2323,805,29874,13622,6230,591,29915,276,2675,304,2244,437,591,505,304,671,263,22813,1358,763,1856,4870,920,1568,1856,4870,526,591,2675,304,505,304,5110,297,1797,304,437,2323,805,29874,1788,28725,322,278,13622,6230,591,29915,276,2675,304,7252,22813,9306,304,1788,28725,322,769,591,29915,276,2675,304,2436,263,1856,4870,2295,515,22728,304,1653,263,2323,805,29874,263,16500,577,937,1283,338,437,591,505,304,671,263,22813,1358,322,278,1234,338,694,366,437,451,505,304,671,263,22813,1358,565,366,526,6257,263,2060,515,22728,306,29915,29881,6907,393,366,671,263,22813,1358,763,1856,4870,3138,565,366,505,385,5923,775,2967,393,5505,3913,263,1422,2048,5780,763,330,15323,769,366,437,451,817,304,671,1856,4870,297,1797,304,679,2712,1985,411,2323,29178,278,13622,6230,306,29915,645,1510,9098,920,366,508,8206,393,1283,577,8369,1492,1286,591,526,5609,292,278,2323,805,29874,11747,687,29891,7799,515,445,2927,315,3289,934,565,366,505,263,2048,5780,763,330,15323,366,29915,276,451,2675,304,367,2221,304,5609,445,322,2149,372,304,679,964,1788,432,29914,29879,322,769,679,964,2323,805,29874,322,769,5766,393,338,451,2675,304,367,1407,4780,577,825,366,508,437,2012,338,366,508,2869,925,1653,263,5534,2286,363,596,2323,9758,2280,1156,366,1653,278,5534,2286,366,508,769,925,6036,393,5534,2286,408,263,2323,9758,2280,577,445,338,1554,306,6907,746,366,29915,276,2805,4687,411,385,5923,775,2967,393,1838,29915,29873,671,28285,338,925,671,263,5534,2286,3138,297,278,13638,310,4251,366,526,2675,304,367,773,263,22813,1358,763,1856,4870,322,1906,526,278,4251,393,591,29915,276,2675,304,5193,1048,297,278,1791,310,278,4863,577,278,2446,1139,338,310,3236,920,1568,1856,4870,626,306,2675,304,505,304,5110,28285,22920,508,6041,367,3265,11777,29891,322,372,508,679,2656,310,12092,322,577,372,29915,29879,263,6534,1139,304,2244,920,1568,626,306,2675,304,505,304,5110,278,1234,338,366,29915,276,451,2675,304,505,304,5110,263,15243,366,674,505,304,5110,263,2846,2712,565,366,29915,276,278,937,2022,4444,445,701,472,596,13013,3138,366,1074,7595,6401,24492,322,769,1653,2323,805,29874,599,674,1653,322,769,1584,7780,557,278,28285,2295,6284,363,366,304,505,372,599,731,701,304,748,263,2323,9758,577,411,6401,366,671,2323,9758,6401,607,674,6623,596,1856,4870,263,2217,2586,363,366,304,1207,372,6775,304,671,411,2323,9758,411,1776,366,671,1776,24492,7079,2323,805,29874,607,674,437,278,1021,372,674,6623,596,596,1856,4870,2295,363,366,322,411,7657,366,508,671,1653,2323,805,29874,607,674,1653,263,7657,2280,393,2307,756,263,1856,4870,2295,393,338,1985,408,263,2323,9758,2280,322,577,925,763,304,679,278,1407,9212,2675,366,437,451,505,304,5110,263,3353,14928,310,1856,4870,988,366,674,505,304,5110,777,1856,4870,338,17415,385,5923,1856,4870,2295,964,1554,393,508,367,502,519,491,2323,805,29874,322,393,4049,508,367,697,310,278,2898,342,5633,310,9725,1218,596,5923,775,22561,304,2323,805,29874,577,825,591,29915,276,2675,304,437,2678,373,297,445,4863,338,591,29915,276,2675,304,2436,263,1856,4870,2295,515,22728,322,297,393,591,29915,645,748,975,1269,760,310,372,322,2020,372,29915,29879,4100,363,263,2323,9758,565,372,338,4100,363,2323,805,29874,577,393,366,1073,825,26278,437,306,817,304,1207,304,590,5923,1856,4870,2295,577,393,372,674,664,408,263,2323,9758,4423,20759,697,1833,2655,304,748,975,1434,591,437,393,2466,591,817,304,5193,1048,6757,263,2217,2586,577,937,1283,727,526,263,7303,916,19707,306,505,2825,393,1048,297,4714,8286,10585,23797,2048,931,10585,322,1048,1788,435,269,577,1906,19707,1795,367,1781,304,1423,714,937,408,896,5649,777,310,278,1820,22001,1244,541,825,591,526,1811,304,12709,1244,338,591,526,1811,304,2507,445,2060,607,1492,1286,871,756,697,934,2927,315,3289,541,1033,505,1784,2066,408,278,2060,4947,16600,591,29915,276,1811,304,2507,445,4152,2060,964,697,8286,3883,697,297,4714,8286,3883,322,393,393,297,4714,8286,3883,674,769,367,16532,491,2184,435,269,577,278,4328,1546,1788,432,269,322,1856,4870,338,393,1788,435,269,338,599,1048,10585,297,278,4714,322,1856,4870,338,599,1048,10585,297,596,2943,10585,577,472,8878,931,2012,310,297,278,4714,577,10940,591,29915,276,9963,1048,599,310,278,10585,2675,373,297,278,4714,577,763,1788,8329,1053,1554,393,29915,29879,366,1073,1407,9436,1788,1444,1794,681,3138,746,591,29915,276,9963,1048,405,13427,15476,1749,343,2753,15476,263,716,3489,322,769,10594,393,2768,310,1749,11846,393,338,2675,304,367,1856,4870,20759,1235,29915,29879,270,573,964,920,437,591,1653,263,1856,4870,2295,363,385,297,29899,15965,3513,3883,393,338,502,519,491,1788,432,29914,29879,322,884,508,367,1304,491,2323,805,29874,408,263,2323,9758,577,408,366,508,1074,591,29915,345,2355,445,775,2307,3971,577,445,338,278,2323,805,29874,11910,310,278,775,322,445,934,2927,3209,338,2998,408,278,6251,934,470,278,1667,934,577,278,6251,934,338,988,278,22813,1358,8665,322,278,4100,2655,1048,278,6251,934,338,393,3099,393,366,5609,515,445,6251,934,674,367,1754,3625,304,1788,432,29914,29879,964,2323,9758,3099,366,5609,515,738,916,934,674,451,367,1754,3625,304,1788,432,29914,29879,297,2323,9758,577,2289,372,29915,29879,871,445,934,393,13750,322,746,591,29915,276,9963,1048,825,338,278,3450,363,1749,297,4714,3883,470,920,437,591,679,278,2834,25785,964,2323,805,29874,278,1234,338,366,505,304,1284,596,6251,934,322,769,1207,1854,393,366,29915,276,5609,292,278,2834,25785,5149,20759,1244,591,526,263,9654,934,697,2655,306,29915,29881,763,304,4443,408,591,1369,2805,964,445,28285,2295,338,393,5491,366,9455,29915,29873,2675,304,817,304,2436,263,1856,4870,2295,6446,515,22728,5491,366,1073,366,2307,505,697,1922,2845,1549,366,1073,1653,7657,623,470,6401,24492,470,1776,24492,322,896,29915,345,2307,1925,4208,263,1856,4870,2295,363,366,577,825,306,29915,29885,2675,304,1510,1244,338,451,825,366,29915,276,2675,304,505,304,437,746,366,29915,276,4969,263,2323,805,29874,2280,541,3265,591,29915,276,925,2675,304,748,975,372,515,22728,577,393,366,508,1074,1432,8424,310,372,596,4982,338,2869,925,2675,304,367,23815,825,366,2307,505,304,1106,901,763,445,322,393,393,29915,29879,2289,825,366,29915,276,2675,304,505,304,437,304,596,775,297,1797,304,1207,372,664,519,408,263,1788,432,29914,29879,3883,322,263,2323,9758,4423,577,278,937,2655,338,393,591,505,304,505,385,6251,934,322,445,508,2869,4049,367,697,310,278,8938,12239,5633,310,9725,1218,385,5923,825,4870,2295,304,671,2323,9758,577,306,29915,29881,6907,871,2534,697,6251,1298,322,1584,2534,393,697,6251,1298,367,443,17514,470,21560,1235,592,1510,366,825,306,2099,491,871,697,322,21560,577,366,508,884,505,385,1409,322,1856,4870,6511,445,577,591,1033,505,2752,11955,2752,322,3814,1691,319,3289,2752,3161,28725,322,472,445,1298,1286,727,526,2211,6251,2066,322,577,607,697,310,963,338,330,11586,367,278,697,393,1788,925,3430,363,278,29586,366,1073,18973,577,445,338,14712,964,263,2323,8286,3883,393,338,297,278,4714,577,565,591,505,2211,6251,2066,607,310,278,2211,6251,2066,674,367,278,697,393,12169,964,278,3883,372,29915,29879,1900,5491,925,304,505,697,451,9301,304,505,901,541,1900,5491,925,304,505,697,884,263,3619,2655,393,366,29915,645,1074,338,263,4257,697,577,591,29915,645,437,393,11955,338,393,322,769,2712,338,2752,2712,577,445,338,1554,393,338,884,3619,445,338,2788,304,278,1409,5174,1286,591,29915,345,2183,263,1024,304,1269,2655,393,29915,29879,297,278,1409,322,445,338,884,1554,393,306,29915,29881,6907,451,2599,565,366,508,29915,29873,679,3448,411,372,297,596,1856,4870,2295,1363,2748,1449,591,29915,276,1811,304,2507,263,2323,6251,934,964,263,2323,297,4714,8286,3883,20759,2446,701,591,29915,345,2355,278,1962,577,278,1962,278,1962,338,5934,366,1073,988,437,591,864,278,934,393,338,2825,491,28285,304,367,7180,322,372,884,2295,1973,825,2924,310,1962,934,437,591,864,577,306,5491,748,411,278,1788,3489,3646,577,3489,3646,3620,472,278,1962,3402,363,278,11846,322,491,2322,372,29915,29879,451,2675,304,664,411,1788,1315,541,565,366,1735,372,304,3489,3646,372,674,664,411,1788,304,2244,5136,6703,366,508,671,501,5773,470,319,5773,408,3489,22525,322,1906,674,664,408,1532,322,1584,763,366,1033,1708,2820,411,777,310,278,916,3489,22525,408,1532,541,306,29915,29881,6907,773,278,1788,432,294,3402,322,727,29915,29879,1790,2655,2000,3489,297,278,1962,988,366,508,2367,278,3489,263,1024,445,338,2869,366,1016,29915,29873,817,372,322,306,29915,29881,2869,6907,451,2599,372,577,591,1033,1024,1749,3489,11955,3138,322,445,6858,4805,5285,310,1788,28725,2599,445,322,577,306,1016,29915,29873,437,372,322,372,338,451,4312,1788,21230,2307,9906,278,1024,310,596,3883,1363,372,29915,29879,297,278,1053,2910,3431,2446,2655,306,763,2534,263,2752,2910,1244,445,338,451,4100,363,2323,9758,470,3099,338,925,1554,306,763,322,445,445,338,2869,591,29915,276,2289,2289,3802,1244,1235,29915,29879,2869,19957,1244,591,29915,276,2675,304,1074,565,445,1736,727,526,263,7303,2712,393,591,29915,276,2675,304,505,304,788,297,2678,541,1235,29915,29879,937,1074,278,1108,322,297,278,4714,322,769,591,29915,645,2329,372,491,6480,278,1856,4870,292,27684,20759,577,304,1065,1856,4870,306,29915,29885,330,11586,505,263,2471,297,590,3577,4663,697,310,963,338,2675,304,367,263,2048,2471,697,310,963,338,2675,304,367,263,1369,2471,6041,278,1369,2471,338,2000,9080,20759,306,4140,727,338,697,2655,591,29915,276,4567,515,1749,1856,3577,372,2355,777,2254,577,306,29915,645,437,5802,363,263,4464,769,975,1244,306,29915,645,437,4464,15743,5849,577,746,591,29915,276,2734,372,408,263,1923,591,864,372,304,367,13126,297,5849,4464,322,769,591,29915,276,1065,746,591,526,5214,372,363,5802,591,864,372,304,367,297,5802,4464,20759,577,1235,29915,29879,1018,263,343,2753,2048,21023,20759,306,1258,451,6456,278,1962,7970,1235,29915,29879,1018,372,1449,1244,372,338,343,1022,20759,577,278,1059,393,591,29915,276,2805,1244,338,393,5934,366,29915,345,2355,263,13543,5877,297,1244,445,3430,763,777,7649,29990,322,366,7359,29915,29873,5429,1856,4870,304,437,7649,29990,577,363,393,591,29915,276,2675,304,671,10555,304,4505,393,366,1073,23741,508,884,664,470,916,752,22058,6041,366,1016,29915,29873,1584,817,350,1107,470,1134,2471,541,297,445,1206,591,437,1363,591,437,505,777,7657,424,7649,29990,2675,373,577,278,3883,5751,4004,17645,920,437,366,4386,3058,2066,322,825,306,29915,29885,2675,304,1827,338,738,934,393,10614,411,278,8329,7649,29990,260,2673,306,864,289,370,569,304,1889,372,306,437,451,864,590,2943,10585,304,367,19356,491,350,1107,1363,896,2307,505,1063,491,278,2022,1058,7344,29879,1906,9279,322,769,591,29915,276,2675,304,1827,671,350,1107,23466,20759,306,29915,345,2307,13252,350,1107,304,2274,7649,29990,322,7657,577,1244,591,748,591,29915,345,5429,1856,4870,599,8286,2066,817,304,748,1549,350,1107,937,1235,29915,29879,1074,565,393,3796,20759,577,1922,445,934,393,591,29915,276,3063,472,338,278,934,393,471,2825,491,28285,372,756,7657,297,372,322,372,756,1749,2927,315,3289,934,297,372,366,3117,7359,29915,29873,5148,472,1784,1375,2164,2066,763,445,372,29915,29879,2656,310,22769,1492,306,437,864,304,1106,472,697,760,310,372,2466,925,278,1407,1407,937,760,8369,372,29915,29879,5432,1788,8329,6036,445,338,825,278,3489,3646,1788,1258,304,1749,11846,338,1286,372,338,5432,1788,451,6036,322,408,366,1795,367,2221,304,4140,1788,8329,6036,338,4100,363,372,304,367,11098,491,1788,21230,577,591,29915,276,373,278,1492,5702,1244,20759,1286,1235,29915,29879,1074,565,591,508,1369,701,263,4870,2906,1923,20759,322,372,2825,278,1667,330,4692,306,29915,29885,2869,2675,304,1735,278,445,304,367,2927,6433,4874,20759,20759,577,591,29915,276,5214,11955,338,1286,825,306,29915,29885,330,11586,437,1286,338,1235,29915,29879,1074,565,445,1736,411,1788,1315,1235,29915,29879,1018,372,714,577,727,29915,29879,263,4700,2000,2323,805,29874,1708,2057,7613,322,372,756,1788,435,269,2307,7500,701,297,278,4714,322,306,671,445,322,372,29915,29879,9024,304,297,278,5106,577,372,29915,29879,263,982,310,925,6724,714,508,306,2254,701,590,2655,411,1788,435,269,322,505,372,664,20759,577,599,393,591,29915,276,2675,304,437,1244,1827,1788,8329,1053,591,817,304,4377,714,278,3988,363,445,934,577,372,29915,29879,15683,29871,29947,29900,29947,29900,2927,315,3289,1235,29915,29879,1074,565,591,508,2254,372,975,1244,1244,372,338,577,591,817,591,29915,276,330,11586,1018,304,2254,445,934,964,445,4434,577,591,29915,276,330,11586,3509,445,565,565,366,29915,276,297,14418,366,508,1584,437,2246,29899,5563,7272,1492,1286,16735,756,263,6494,306,1348,297,278,4714,2991,411,2246,29899,5563,7272,306,29915,29885,1854,896,29915,645,2329,372,4720,322,577,491,278,931,366,6505,445,4863,5505,896,29915,645,505,372,4343,577,21915,1244,29915,29879,278,937,1108,393,591,29915,276,8790,591,29915,276,8790,591,508,29915,29873,2254,445,1363,372,29915,29879,2869,373,263,1422,5354,3236,306,29915,345,2355,297,7458,411,3236,577,1363,591,29915,276,871,2734,1856,4870,2906,1923,12430,591,29915,276,451,773,372,363,5802,727,9455,29915,29873,263,12176,5253,310,6993,21838,1048,8718,701,599,3236,7274,304,1749,15683,322,297,445,1206,577,825,591,508,437,338,788,263,2906,1923,2908,322,769,1827,591,817,304,788,777,4805,2933,9066,7331,2933,9066,515,1749,1856,3938,384,2906,1923,20759,577,1286,591,29915,276,2675,304,1827,2130,2761,2758,3978,338,278,697,393,3732,19435,664,591,29915,276,2675,304,1827,1235,738,5354,1207,738,7274,304,1749,1923,577,1235,29915,29879,11086,278,1813,322,1018,1449,372,3282,29915,29873,664,366,505,304,10715,596,1856,4870,2906,1923,746,366,1735,278,1856,4870,2295,1244,591,526,322,577,278,937,760,3796,1438,13328,18116,393,526,772,3262,701,526,12327,5414,322,591,29915,276,330,11586,4505,1906,2086,541,322,278,937,760,3796,577,591,29915,276,8790,445,16087,5766,322,443,16476,322,577,1286,393,591,29915,276,297,263,2058,988,445,674,2869,322,445,674,2869,763,2254,701,297,6757,4874,322,2323,805,29874,322,664,304,1510,393,304,366,577,2323,2319,1708,2057,756,263,2217,4682,2000,14426,6987,1235,29915,29879,1074,1235,29915,29879,1074,565,306,508,437,393,21023,21915,577,20759,825,29915,29879,10464,1244,306,4140,591,29915,276,1303,1434,373,7331,322,1286,591,29915,276,373,7331,29903,338,393,825,471,10464,306,29915,29885,451,263,6893,10151,1854,1235,29915,29879,1235,29915,29879,748,1250,304,988,591,892,577,565,366,29915,276,373,7331,607,306,29915,29885,451,1854,565,14418,674,1584,1235,592,21915,565,591,29915,276,373,7331,769,1286,445,338,445,1410,29891,2656,310,13543,1244,20759,591,29915,276,451,2675,304,437,278,14426,1243,372,947,664,306,505,1304,372,1784,3064,541,363,278,16563,310,445,4863,3133,292,373,591,29915,276,451,2675,304,671,372,1492,1286,297,2114,306,29915,29885,330,11586,505,304,437,697,2655,304,1207,1854,393,1438,4436,748,3448,20759,1235,29915,29879,748,1250,304,988,591,892,1244,591,526,577,21915,1286,591,29915,276,2805,1438,278,18116,322,1286,372,29915,29879,2654,1584,2805,445,8340,3495,3978,4839,577,1235,29915,29879,4505,445,1108,825,29915,29879,10464,1244,338,393,28285,2906,1923,14335,304,731,701,2563,11373,470,372,14335,304,731,701,7331,1248,1847,297,1797,304,679,263,5503,12519,10940,366,4078,263,934,322,337,12198,322,769,19763,278,1813,2748,393,5930,322,297,1797,304,437,393,372,756,304,1207,263,3287,310,7274,304,15683,322,591,29915,276,17581,777,3495,12747,577,366,11262,3495,1423,2748,1449,322,445,338,871,1641,1304,373,596,15683,372,29915,29879,451,1554,896,29915,645,367,1304,297,5802,577,591,29915,276,451,763,925,2805,8177,310,4100,6993,2712,1244,363,596,5802,2280,541,2748,591,11086,1286,591,748,591,508,1584,1074,393,591,29915,345,2355,1856,4870,2906,1923,5934,393,5735,337,13234,338,9615,20759,577,445,445,1856,4870,2295,1244,445,29871,29906,29955,3454,338,2289,599,393,366,817,297,1797,304,679,263,2323,9758,4423,1985,278,3489,3646,3646,338,2428,4100,278,6251,338,2428,4100,322,769,366,3117,674,817,777,2906,1923,6433,297,278,9886,5633,310,445,4863,306,29915,29885,2675,304,748,975,263,7303,901,2712,393,366,1795,1065,964,746,366,29915,276,23815,470,1856,393,2295,393,591,7359,29915,29873,3595,3447,577,697,310,963,338,393,591,892,8859,8790,896,763,13328,322,2654,18116,373,263,2425,297,1244,577,366,1795,817,304,731,263,577,384,3495,322,263,577,384,2011,297,1244,366,508,748,1106,1438,701,373,1856,3577,263,7444,304,664,304,2274,963,2253,541,445,322,445,18690,29879,1856,4870,2906,1923,988,437,306,1207,263,2009,304,14511,403,445,5503,12519,2563,11373,6433,6456,278,278,2563,11373,6433,338,871,577,393,278,1813,6336,337,18132,746,366,4078,278,3620,322,727,29915,29879,3078,901,304,372,1135,393,541,372,508,367,3265,12327,5414,565,366,29915,345,2355,263,13328,322,2654,4315,925,373,2425,772,3262,2712,701,322,372,508,367,7575,304,505,278,5735,337,13234,2675,577,366,29915,645,6041,817,445,1790,2655,338,322,4049,366,29915,645,1074,297,1856,4870,2295,322,5994,17063,760,322,445,5994,17063,760,508,2869,4473,2712,701,746,591,29915,276,1811,304,2254,596,11846,408,263,1788,432,29914,29879,3883,577,278,2769,2020,372,4473,267,372,701,278,5994,17063,11846,338,263,982,310,640,22155,920,1856,4870,8536,1169,599,596,2066,964,1962,22813,793,577,8369,746,306,1065,1856,4870,1244,372,14391,697,934,393,29915,29879,29871,29896,29941,29941,4679,18711,2167,278,5994,17063,541,2908,4076,366,263,8825,304,6219,701,445,2927,3209,964,2999,2066,278,1108,411,773,372,763,393,338,393,1286,607,697,310,1906,2066,338,278,6251,934,322,607,697,881,367,16532,491,2184,435,269,322,769,674,278,916,6743,884,367,16532,6041,372,15894,393,366,29915,276,11658,292,2471,8282,964,278,4544,934,411,2380,29889,1420,18665,29899,262,577,372,508,367,3265,28722,577,491,21228,5994,17063,366,29915,276,451,2869,3907,596,2048,3109,14413,366,29915,276,278,871,2655,393,366,29915,276,2599,338,366,29915,276,5934,306,29915,29885,2675,304,6755,920,304,437,775,24368,297,263,1422,982,577,565,591,748,304,278,1856,4870,5106,322,748,304,1009,775,24368,4004,896,5193,1048,1023,1422,5837,310,2599,775,24368,411,263,2323,805,29874,306,21005,697,3153,982,322,393,982,947,451,671,278,5994,17063,2908,297,2114,306,1348,372,723,367,7088,8000,304,1722,393,701,20759,577,366,508,505,2999,6251,3291,366,508,505,13883,6219,521,18801,6433,322,366,508,505,7343,24802,577,306,2289,723,6907,2599,7343,24802,746,366,29915,276,2599,263,2323,805,348,366,29915,276,1811,304,1653,263,2323,297,4714,3883,363,445,4152,2060,411,7343,1053,366,508,1603,679,2999,1962,22813,793,322,896,29915,645,367,366,1073,8072,27545,322,3447,372,674,1603,664,372,29915,645,1603,664,411,1788,432,29914,29879,6124,635,565,596,565,596,2060,4947,4802,3307,28285,6593,565,366,2601,3307,9562,470,366,2436,3307,775,28285,674,1369,2599,777,6433,491,2322,304,6219,372,701,372,29915,645,1653,763,9691,943,521,18801,322,6433,763,393,577,727,727,526,13501,304,2599,775,8536,1169,7343,24802,322,777,310,278,1856,4870,2322,6433,393,674,1207,596,11846,2189,424,322,366,526,451,26285,292,4180,746,366,5217,445,5994,17063,2908,366,3763,9455,29915,29873,20759,577,393,29915,29879,393,29915,29879,697,2655,393,366,29915,276,330,11586,505,304,1348,1048,1790,2655,322,445,697,29915,29879,5051,4100,322,322,306,1348,393,445,338,278,1833,697,306,29915,29885,330,11586,748,975,297,445,4863,338,278,970,2224,577,278,970,2224,411,1856,4870,338,1554,393,16058,871,304,1856,4870,577,565,366,29915,276,2599,9679,29899,786,306,4140,727,526,2788,22001,322,9679,29899,786,541,366,1016,29915,29873,505,304,1348,1048,445,13452,297,1856,4870,366,437,505,304,1348,1048,445,278,970,2224,372,10603,1856,4870,988,437,306,5142,599,310,590,775,8536,1169,515,322,577,1363,411,263,2323,9758,2280,591,1795,505,697,2280,373,697,1923,322,769,393,4544,934,373,263,1422,1923,470,1584,565,896,29915,276,373,278,1021,1923,541,896,29915,276,925,472,1422,10898,1856,4870,508,1369,2805,9613,1048,988,437,306,5142,278,590,775,8536,1169,577,278,19075,982,304,451,505,304,15982,1048,393,338,366,2601,263,3489,2000,1788,1315,28285,4124,459,769,366,1653,263,934,2000,731,970,2224,445,338,599,297,278,5106,363,1788,1315,1856,4870,322,5768,322,769,366,2367,372,278,1024,310,596,366,2367,372,278,1024,310,596,1788,432,29914,29879,3883,1244,577,445,338,278,1024,310,278,1788,432,29914,29879,3883,577,278,1024,338,5491,1554,763,366,1073,596,1638,322,769,278,2060,1024,372,508,367,3099,2289,372,29915,29879,6514,338,297,278,1053,2910,297,1749,1243,5177,975,1244,306,7359,29915,29873,1584,2289,3939,278,1053,2910,3447,1235,592,437,393,1492,1286,577,306,29915,29885,330,11586,1827,11955,338,472,445,3988,20759,1235,29915,29879,11086,322,591,508,437,1286,2012,310,1286,2012,310,10594,297,278,2989,3988,591,508,1827,322,11955,1244,1286,20759,577,1554,306,1258,338,3907,372,577,393,372,694,5520,664,322,372,29915,29879,5934,393,393,577,28337,3508,29915,29873,1985,15128,975,1244,9360,372,29915,29879,1363,306,9445,278,1923,1235,29915,29879,1369,372,1250,701,20759,1244,591,526,577,1922,366,925,505,304,437,445,970,2224,2655,748,1106,701,278,5106,363,6757,4874,541,4870,322,5768,393,674,1207,372,2253,322,445,674,11200,731,278,970,2224,393,297,278,1856,4870,5106,372,5717,372,4444,372,373,278,11340,20759,727,338,697,1407,1833,2655,306,29915,29885,2675,304,1510,1244,297,278,1856,4870,2295,322,393,338,1492,1286,306,626,373,7331,541,565,306,4607,975,304,7331,29903,769,2712,674,1369,1641,9391,1235,29915,29879,748,1244,21915,577,372,372,29915,29879,17581,304,5142,278,11955,2280,577,304,1207,372,472,664,366,508,437,7331,1565,322,769,10715,825,445,674,437,338,263,1856,4870,674,5706,385,17122,12289,304,671,363,445,2060,278,17122,12289,674,451,367,9311,287,491,596,4714,322,297,16735,366,508,2869,925,2649,10228,304,2337,9311,23199,1078,373,15683,727,29915,29879,263,518,29914,25580,29962,13,797,445,4863,29892,591,526,2675,975,920,304,10822,2563,4058,363,263,2323,29899,1028,29874,2280,29889,319,2323,29899,1028,29874,2280,338,263,1856,2280,393,3913,263,2323,6251,1298,363,599,310,967,5680,29889,1334,526,2675,304,4612,278,1494,23820,29901,13,13,29896,29889,3295,17961,3495,12747,29901,2563,4058,9481,5656,14335,304,731,701,2563,11373,470,7331,1248,1847,297,1797,304,679,5503,25913,746,14238,322,337,2388,6504,278,775,29889,2398,29892,445,508,4418,777,3495,12747,29892,577,591,817,304,11262,963,363,15683,29889,13,29906,29889,4956,1747,18116,322,4436,29901,1334,526,8790,13328,322,2654,18116,322,4436,373,263,2425,297,278,5285,29889,1334,817,304,731,278,421,21852,8514,29952,322,421,21852,2290,29952,3987,297,278,421,27327,29889,2917,29889,1315,29952,934,29889,13,29941,29889,20693,326,17063,29901,450,5994,17063,2908,4076,502,263,8825,304,6219,701,278,1962,22813,793,964,2999,2066,29889,2398,29892,6041,372,15894,393,591,526,11658,292,2471,8282,964,278,4544,934,411,278,421,2248,29889,1420,29952,7079,29892,607,508,367,28722,29889,2648,21228,278,5994,17063,2908,29892,591,526,23906,920,304,437,775,24368,297,263,1422,982,29889,13,29946,29889,5236,2224,29901,450,970,2224,10603,2563,4058,988,304,5142,599,310,1749,775,8536,1169,515,29889,1334,817,304,2601,278,421,5205,1315,29899,27327,29899,1639,459,29952,3489,322,1653,263,934,2000,421,842,19858,2605,29889,1315,29952,297,278,5106,363,421,5205,1315,29899,27327,1412,13,13,10605,338,278,4784,5285,1156,3907,1438,3620,29901,13,28956,1315,13,5453,29889,26500,353,426,13,29871,849,856,916,3987,2023,13,29871,13883,29901,426,13,1678,6219,1451,18801,29901,426,13,418,521,18801,29901,525,497,742,13,418,1375,3505,29901,29871,29896,29900,29900,29900,29900,29892,13,418,1375,2776,29901,29871,29906,13,1678,2981,13,1678,521,18801,29901,426,13,418,11955,29901,426,13,4706,1243,29901,847,29961,3823,29914,29962,3124,29889,1315,29938,19637,13,4706,3160,29901,5159,13,418,500,13,1678,500,13,29871,2981,13,29871,970,2605,29901,525,1124,597,7640,11283,13,3400,13,28956,13,797,445,5285,29892,591,505,731,421,20640,2133,29952,304,421,5451,1451,18801,29952,304,9025,775,24368,29889,1334,505,884,731,278,421,305,18801,29952,2984,304,6084,393,278,421,27703,29952,19875,881,367,5134,297,278,1962,29889,9788,29892,591,505,731,278,421,3597,2605,29952,2984,304,421,1124,597,7640,29901,1673,607,10603,2563,4058,988,304,5142,599,310,1749,775,8536,1169,515,29889],
  "total_duration": 747247376805,
  "load_duration": 1473860879,
  "prompt_eval_count": 1633,
  "prompt_eval_duration": 400145404000,
  "eval_count": 501,
  "eval_duration": 345640948000
  }

Here's the current result in Obsidian:

Here's the result with the Local LLM off using 3.5-turbo (much nicer, IMO):

[ozdemir08]

Hi Rishi, thanks for looking into this and already making progress. I have a few high-level feedback:

Running a local LLM

First and foremost, I believe we should leave the logic of running a local LLM to an external library like ollama-js as opposed to the way you suggest below.

ollama pull llama2
tested LLM works via ollama run llama2
if local server not running, start with ollama serve
curled test request:
... 
...

The ollama-js library must have solved many problems including running it locally, downloading the models, managing the lifecycle of it etc and it's better to focus on the actual problem instead of introducing a technical problem and solving it.

Settings page

I can see us adding Gemini or another LLM support soon. So, I think we should not have a toggle that says "Enable local LLM", but instead a dropdown that allows selecting between OpenAI - Chat GPT / Meta - Llama.

The settings page will have model-agnostic settings like summary size, date etc. and model-specific settings like OpenAI key, ChatGPT model etc. For the Llama case, I envision the settings like this: It will have a dropdown for models and then there will be a button next to it, that says "Download model". As models are large objects, users should download them intentionally.

Another thing is that I have an M1 Macbook Air, which is a relatively new laptop and when I use Lllama, it often freezes it. That's why, I am concerned that it may not be as effective on many laptops . Also, mobile phones may have similar issues. Even if they run smoothly, they may heat up the battery etc. That's why I would still keep this as an experimental feature and not enable it by default.

Implementation

Due to similar reasons, it's important to write the code in a way that the AIClient interface has multiple implementations(in this case only 2) and we do not rely on a boolean like enable local llm, but instead use an enum and initialize the interface with the right implementation in YoutubeVideoSummaryModal.

Testing

Initially, I had not implemented any tests as the plugin was small and I prioritized speed and shipping over technical excellence. As the plugin code grows, we will need to start adding tests. No need to do anything now, but this is something for us to consider later.

Your contributions

You are the first contributor of the plugin and here you are working on your second PR. I must say that this is amazing to see for me. If you would like, I can add you as a core contributor/owner of the plugin. If not, I can add your name in the readme and give you credit. Let me know, which one you would prefer :)