search

janhq / cortex

Drop-in, local AI alternative to the OpenAI stack. Multi-engine (llama.cpp, TensorRT-LLM, ONNX). Powers 👋 Jan
https://cortex.so
Apache License 2.0
1.78k stars 97 forks source link

feat: `cortex embeddings INPUT MODEL_ID` #647

Open 0xSage opened 1 month ago

0xSage commented 1 month ago

Does cortex.cpp have this already?

https://platform.openai.com/docs/api-reference/embeddings

This is step before cortex files ...

louis-jan commented 1 month ago

neat

louis-jan commented 1 month ago

Does cortex.cpp have this already?

https://platform.openai.com/docs/api-reference/embeddings

This is step before cortex files ...

Yes cortex.cpp have this already

0xSage commented 4 weeks ago

I'm testing this but the model id (and token counts) are dropped in the response.

e.g. 

❯ cortex embeddings "hi"
{
  data: [
    {
      embedding: [
          0.009946033358573914,  0.0035855502355843782, -0.0032624329905956984,
         -0.001015871181152761,   0.008039350621402264,  0.0002463190467096865,
        -0.0060457284562289715,   0.005288013257086277,   0.008071962743997574,
        -0.0003013485111296177,   -0.00937418732792139,  0.0028152556624263525,
          0.014788639731705189, -0.0009417738765478134, -0.0008780430653132498,
           0.00435042567551136,   0.004187021404504776,   0.008690613321959972,
         0.0006068534567020833,    0.00262630102224648,  0.0009669634746387601,
         -0.009229307062923908,  -0.004353167023509741,   0.004069868009537458,
         0.0019174072658643126,   -0.01061160210520029,   -0.01317606307566166,
          0.006087783258408308,  -0.004666577558964491,   -0.00437656557187438,
         0.0007791431853547692,   0.009918656200170517,   0.002660483820363879,
          0.008012434467673302,  -0.004314407706260681,   0.015089921653270721,
        -0.0011164051247760653,    -0.0092488257214427,    0.01922435872256756,
         -0.011242173612117767,    0.02200542949140072, -0.0037525948137044907,
          0.014841334894299507,  -0.001528711523860693,   0.003504831111058593,
          0.008743774145841599,   0.013840559870004654,  0.0010166524443775415,
           0.00516469543799758,  -0.003348605940118432,   0.017159370705485344,
          -0.01039204467087984,  0.0010975850746035576,   0.021666444838047028,
         0.0013618911616504192,   -0.01364470086991787,  -0.010000248439610004,
        -0.0019074968295171857,  -0.003674417966976762,   0.007858168333768845,
          0.008236742578446865,  -0.007554563693702221,   0.002303753513842821,
          0.011989221908152103,   0.001310529187321663,   0.019879180938005447,
         -0.005047136917710304,  -0.016054565086960793,    0.00918799452483654,
          0.008179975673556328,  -0.027703912928700447, -0.0006355145014822483,
            0.0147264888510108,  0.0026045155245810747,  -0.013167967088520527,
          0.012205726467072964,   0.002417328767478466,     0.0101541206240654,
          0.014287085272371769,   -0.01305917464196682,  -0.005873063579201698,
         0.0017505884170532227,  0.0016493778675794601,   0.010036461055278778,
        -0.0045578209683299065, -0.0066118319518864155,  -0.003198853926733136,
         -0.010488429106771946,  -0.018016289919614792,    0.00837547518312931,
          0.004892445635050535,   0.004631889518350363,   0.005953103303909302,
           -0.0035823336802423,  -0.014715082943439484,  -0.008343889378011227,
         -0.013891108334064484,   0.003839879995211959,  0.0035273656249046326,
         -0.006966897752135992,
        ... 3996 more items
      ],
      index: 0,
      object: 'embedding'
    }
  ],
  model: '_',
  object: 'list',
  usage: { prompt_tokens: 0, total_tokens: 0 }
}
louis-jan commented 5 days ago

cc @vansangpfiev

I'm testing this but the model id (and token counts) are dropped in the response.

e.g.

❯ cortex embeddings "hi"
{
  data: [
    {
      embedding: [
          0.009946033358573914,  0.0035855502355843782, -0.0032624329905956984,
         -0.001015871181152761,   0.008039350621402264,  0.0002463190467096865,
        -0.0060457284562289715,   0.005288013257086277,   0.008071962743997574,
        -0.0003013485111296177,   -0.00937418732792139,  0.0028152556624263525,
          0.014788639731705189, -0.0009417738765478134, -0.0008780430653132498,
           0.00435042567551136,   0.004187021404504776,   0.008690613321959972,
         0.0006068534567020833,    0.00262630102224648,  0.0009669634746387601,
         -0.009229307062923908,  -0.004353167023509741,   0.004069868009537458,
         0.0019174072658643126,   -0.01061160210520029,   -0.01317606307566166,
          0.006087783258408308,  -0.004666577558964491,   -0.00437656557187438,
         0.0007791431853547692,   0.009918656200170517,   0.002660483820363879,
          0.008012434467673302,  -0.004314407706260681,   0.015089921653270721,
        -0.0011164051247760653,    -0.0092488257214427,    0.01922435872256756,
         -0.011242173612117767,    0.02200542949140072, -0.0037525948137044907,
          0.014841334894299507,  -0.001528711523860693,   0.003504831111058593,
          0.008743774145841599,   0.013840559870004654,  0.0010166524443775415,
           0.00516469543799758,  -0.003348605940118432,   0.017159370705485344,
          -0.01039204467087984,  0.0010975850746035576,   0.021666444838047028,
         0.0013618911616504192,   -0.01364470086991787,  -0.010000248439610004,
        -0.0019074968295171857,  -0.003674417966976762,   0.007858168333768845,
          0.008236742578446865,  -0.007554563693702221,   0.002303753513842821,
          0.011989221908152103,   0.001310529187321663,   0.019879180938005447,
         -0.005047136917710304,  -0.016054565086960793,    0.00918799452483654,
          0.008179975673556328,  -0.027703912928700447, -0.0006355145014822483,
            0.0147264888510108,  0.0026045155245810747,  -0.013167967088520527,
          0.012205726467072964,   0.002417328767478466,     0.0101541206240654,
          0.014287085272371769,   -0.01305917464196682,  -0.005873063579201698,
         0.0017505884170532227,  0.0016493778675794601,   0.010036461055278778,
        -0.0045578209683299065, -0.0066118319518864155,  -0.003198853926733136,
         -0.010488429106771946,  -0.018016289919614792,    0.00837547518312931,
          0.004892445635050535,   0.004631889518350363,   0.005953103303909302,
           -0.0035823336802423,  -0.014715082943439484,  -0.008343889378011227,
         -0.013891108334064484,   0.003839879995211959,  0.0035273656249046326,
         -0.006966897752135992,
        ... 3996 more items
      ],
      index: 0,
      object: 'embedding'
    }
  ],
  model: '_',
  object: 'list',
  usage: { prompt_tokens: 0, total_tokens: 0 }
}