search

alpaca-core / ac-local

Alpaca Core local inference SDK
MIT License
1 stars 0 forks source link

llama: why is `input_prefix_bos` user-provided in main example? #13

Closed iboB closed 1 month ago

iboB commented 3 months ago

Find out the significance of this. For now we're ignoring this. Maybe we will have to add it

pminev commented 2 months ago

input_prefix_bos was added in this PR . It seems like it was needed for llama-2-chat prompt format. If we're going to support it, I think we'll need it.

Basically we can deduct that if client will want chat we can pass the BOS before adding the input prefix, so we can hide it from the clients.

IvanFilipov commented 2 months ago

Maybe with the investigation completed, we can close this issue. With little to no context, as far as I understand we need a new working item - to add the support.

In general, may I suggest that we add a hyperlink to each abbreviation, which doesn't pop up as a first google search result. In my opinion, getting contributors up to speed on a topic, will be much easier that way.

iboB commented 2 months ago

Still not clear if there's something actionable to be done on our part

iboB commented 1 month ago

I wouldn't call this completed, but we'll reintroduce the option just in case