Open mdemoret-nv opened 3 months ago
Ideally, it should be capable of outputting any type which is returned by the endpoint processing lambda.
Both HttpServerSourceStage
and HttpClientSourceStage
support a payload_to_df_fn
method, however the output is a DataFrame
.
If we wanted to do this I think there are two options:
1) Break API compatibility and change this to a request_to_msg_fn
method
2) Add a second optional method to convert both the payload and the DataFrame
to a message.
However currently for HttpServerSourceStage
specifying payload_to_df_fn
forces the Python impl of the stage (although the server itself is still implemented in C++).
I'm thinking instead doing something similar to what the DeserializeStage
does and do:
1) Add a message_type
& task_type
constructor arguments
2) Add a request_to_task_payload_fn
constructor argument. When None
a default C++ impl will be used. When not-none the Python source stage impl will be used.
Is this a new feature, an improvement, or a change to existing functionality?
Improvement
How would you describe the priority of this feature request
Medium
Please provide a clear description of problem this feature solves
Currently, the
HttpSourceStage
only supports outputting aMessageMeta
object. This is very limiting in situations where some additional metadata needs to be attached to the message that is only available with the HTTP request object.Describe your ideal solution
Support outputting different data types by the
HttpSourceStage
class. Ideally, it should be capable of outputting any type which is returned by the endpoint processing lambda. At a minimum,ControlMessage
s should be able to be generated by this stage.Additional context
No response
Code of Conduct