Open SergeyMenshykh opened 7 months ago
@SergeyMenshykh I don't think it will be good. BinaryContent.Data is required but ImageContent.Data and AudioContent.Data are optional, so casting these two to BinaryContent would give inconsistent behavior. Following this way, we should create a DataKernelContent class with Uri and BinaryData parameters and inherit from it.
This feature should be worked on in conjunction with https://github.com/microsoft/semantic-kernel/issues/5263 (which should address @Krzysztof318's concerns around BinaryContent.Data being required). Because BinaryContent
could/should be hydrated with either a URL or a byte array, it should likely be renamed to FileContent
.
The main scenario that we'd like to support is the following...
BinaryContent
class is predominantly used for the Assistants APIImageContent
with it, the user should simply be able to cast it to the correct type.As a test of the new hierarchy, the following POC should be possible:
ChatMessageContent
(today they come in via the AnnotationContent
, so there needs to be a way to easily turn this into BinaryContent
/FileContent
; perhaps AnnotationContent
is merged with BinaryContent
/FileContent
and the data like Quote
can be stored in a metadata field)BinaryContent
or FileContent
)ImageContent
to perform a text-to-image taskAudioContent
to perform audio-to-text task
@matthewbolanos please provide details of the issue that would be fixed by the change.