roy651
commented
1 month ago @obuzek per your comment on the other thread: I understand the mechanism you describe and I acknowledge that you're utilizing the "system" prompt as part of the SDG. However, I think that, this design choice is something that should be relatively easy to resolve with a suitable instruction to the SDG ("Do not generate this part"...). Assuming technically it's feasible and the discussion is about the need - One might argue that the set of Q&A (and context) samples should suffice for the model to learn the necessary "rules" and pattern of the specific skill. I do think that this need is valid and quite widespread. As the use of LLMs expands and it not necessarily plain conversation-like, and as the complexity of input and output patterns rises, we see frequent deviations of the model from the required patterns. I expect to see others come up with a similar request.
bjhargrave
(Issue was moved from here as it touches multiple areas) When defining a new skill in the pattern of question-context-answer it seems that ilab will generate all of those when it creates the augmented dataset. The implication is that they are all dynamically generated.
I would like to propose that ilab will support an additional entry in the skill's yaml allowing for a "system"-style which will preface the question when generating the answer. This part should be static and need not be generated for each new entry. Most likely, it should be static for the entire skill, hence it should reside in the top level of the yaml with the "task_description".
I think that this capability is something that may be required by additional skills as it follows, the now so common, system/user pattern and I assume others would require that too.