This example requires standing up the pulsar docker-compose stack + the pulsar plugin package. This will throw an error as the input to the from property is a string and so the plugin thinks it is a URI pointing to an internal file. (using | does the same thing)
The issue is opened here rather than on Pulsar as it's a more generic issue, e.g. the same in Kafka plugin:
- id: avro_to_kafka
type: io.kestra.plugin.kafka.Produce
from: {{ json('[1, 2, 3]') }} <- This gives an error and we have to do "{{ json('[1, 2, 3]') }}" which becomes a string
.....
atm, it doesn't matter what the output of the expression is, it always gets converted to a string because we have to escape the {{ with " or pipe |
Ideas worth exploring
We may consider using YAML tags e.g. from: !!map "{{ inputs.dict }}" to let users cast some dynamically rendered values to a specific data type expected by a plugin:
Feature description
This example requires standing up the pulsar docker-compose stack + the pulsar plugin package. This will throw an error as the input to the
from
property is a string and so the plugin thinks it is a URI pointing to an internal file. (using|
does the same thing)However, if you pass a map directly, it works ok:
The issue is opened here rather than on Pulsar as it's a more generic issue, e.g. the same in Kafka plugin:
passing JSON-type input leads to the same issue:
atm, it doesn't matter what the output of the expression is, it always gets converted to a string because we have to escape the {{ with " or pipe |
Ideas worth exploring
We may consider using YAML tags e.g.
from: !!map "{{ inputs.dict }}"
to let users cast some dynamically rendered values to a specific data type expected by a plugin:Alternatively, we need to reconsider how some plugins render dynamic properties to enforce coercing types
The https://github.com/kestra-io/kestra/issues/2962 might be one way to handle such use cases, but more as a workaround
Alternative - pipe operator to a given type