davetron5000 / declare-ai.org

Source for declare-ai.org
https://declare-ai.org
1 stars 2 forks source link

Prompt reproducibility #5

Open moritztim opened 6 hours ago

moritztim commented 6 hours ago

Perhaps the scope of this could be expanded to store the exact configuration used when generating. Allowing for reproducibility of outputs. This could be just another property in the structured declaration that accepts a value in json. The parameters could be standardized, but that would probably take a lot of effort. It shouldn't be too hard to create a unified object for llm parameters, as they're all quite similar, and there seems to be a standard for image generation: https://github.com/civitai/civitai/blob/9f426ba5f985fdb9f2739bb3fdfec2b9c390ccb9/src/components/ImageMeta/ImageMeta.tsx#L42C1-L51C26

davetron5000 commented 5 hours ago

I worry that being too prescriptive this early in the tech hype cycle would ossify something that might not make sense later. I wonder if, instead, there could be some understanding that values could contain free-form text or free-form structured data. Or, perhaps, a link to said information with an appropriate rel value (e.g. describedby or perhaps a new one? I think rel can technically be anything)

The main goal of this is transparency for people consuming content, and not e.g. the source of truth for how the content was produced (though I can see the value of also having that).

moritztim commented 4 hours ago

I worry that being too prescriptive this early in the tech hype cycle would ossify something that might not make sense later.

That's a good point. I'm really hoping for this to become the main standard for this type of thing, which is very possible, since there doesn't seem to be any competition and this one has a full release that includes all the necessities and is simple to implement. So keeping additional complexity hidden at first glance (perhaps only showing some properties in an overview and elaborating in the actual specification), should be considered, particularly in #4.

I wonder if, instead, there could be some understanding that values could contain free-form text or free-form structured data. Or, perhaps, a link to said information with an appropriate rel value (e.g. describedby or perhaps a new one? I think rel can technically be anything)

I think you're right, there might be systems that output structured data in their own format and since said tool would be able to reproduce it with that data, it's probably best to just accept free-form structured data, as you put it. However, particularly with image generation there's often non text input involved, which couldn't be included, so linking should be an option. describedby seems fitting.