inja templating {{input}} not working

[saymrwulf]

local inference server running in LMStudio URLsource with http POST & 1) request body without templating { "messages": [{ "role": "user", "content": "hello"}]} --> works & returns expected response body 2) request body with improper templating like { "messages": [{ "role": "user", "content": "hello {input}"}]} --> works & returns response body with garbage content (as expected), which means there was no text replacement to begin with 3) request body with proper templating { "messages": [{ "role": "user", "content": "hello {{input}}"}]} --> {"error":"'messages' field is required"} which means the templating escaped and destroyed the json structure without leaving a json syntax error

Environments (same error in both environments): a) Apple Silicon, Sonoma, fresh OBS , LMStudio, URLsource b) Win11 x64, fresh OBS, LMStudio, URLsource

[royshil]

can i see the request builder screenshot? it might be formattign the JSON incorrectly resulting in an error from OpenAI specifically the problem is with newlines \n not injected properly

[saymrwulf]

several attempts: first a direct copy from the LMStudio LocalInferenceServer Model client example code but with inserted newlines by me, then a more minimal representation with correct indentations but without newline, then the most minimal version on one line, and lastly a version without templating just to show that the setup works in general.

[royshil]

@saymrwulf i think th problem is malformatted JSON body are you still stuck on this?

[saymrwulf]

still stuck on this. malformed json ? The 2nd pic is the one with the most simple request. still fails.

[saymrwulf]

could you post a json request that according to you should not fail ? One that contains the templating mechanism ?

[royshil]

@saymrwulf for example

{
    "model": "gpt-3.5-turbo",
    "messages": [
      {
        "role": "user",
        "content": "{{input}}"
      }
    ]
  }

[saymrwulf]

not working. As soon as I remove one pair of {} , the chat works, of course not as intended, because no templating/replacement has happened (as expected). The construct "{{input}}" however is flawed. Is there some extra installation/configuration required for inja to work ?

[royshil]

@saymrwulf i don't think there's dependency problem since inja is built from source inside the plugin. i suspect that {{input}} doesn't work simply because it doesn't have any input and the request never fires. in your latest examples you had "Agg to Empty" on what if you remove that?

[saymrwulf]

I removed "Agg to Empty". Same result. The file that is fed into URLencode does exist and it contains live data. The only thing I find a bit suspicious is that it contains a lot of empty lines. Other than that the setup looks normal.

[royshil]

@saymrwulf so if you hit "test" in the request builder - the body preview shows the content instead of {{input}}?

[saymrwulf]

no the body stays as it is with the line { "role": "user", "content": "{{input}}" } and all the rest as it was from the get go

[saymrwulf]

... and then the server does not like this whole string and thinks ..."{{ ... is some sort of escape mechanism that destroy the whole "messages" structure

[royshil]

@saymrwulf if the body doesn't replace {{input}} then it doesn't have input for the replacement can i see your latest request builder screen again? and the body text verbatim use "```" to paste the body text here so i can see exactky how its formatted

[saymrwulf]

[saymrwulf]

this is the body: " { "messages": [ { "role": "system", "content": "Always answer in rhymes." }, { "role": "user", "content": "{{input}}" } ], "temperature": 0.7, "max_tokens": -1, "stream": false } "

[royshil]

@saymrwulf i think you should at any rate enable "Skip empty" and "Skip same" the template seems fine so i see no reason for inja to not replace the {{input}} except when it's empty try the "skip"s and see, please

[saymrwulf]

Problems are caused by other faults - has nothing to do with inja indeed. Observe weird OBS behavior stubbornly refusing to function according to the config settings. Constantly deviating between what you see and what you get. And if you violently force OBS to accept config changes (by deleting the source one wants to reconfigure and re-install it), OBS is always crashing now. (MacBook M3 128GB). (BTW: The skip-empty trick is basically a way to force OBS to come up with the correct error message which is, that it finds an emptyness only...)

[royshil]

closing for now