search

atlasmap-attic / atlasmap-runtime

AtlasMap runtime engine and design time services
http://atlasmap.io
Other
3 stars 8 forks source link

Support for InputStreams in input #207

Open zregvart opened 7 years ago

zregvart commented 7 years ago

Currently there is only support for String in the input of JSON and XML modules, we should support InputStream as input.

See XmlModule.java#L167-L171 and JsonModule.java#L145-L150.

When `InputStream` is specified in the input the following exception is thrown: ``` io.atlasmap.api.AtlasException: Unsupported input object type=io.atlasmap.json.v2.JsonComplexType at org.apache.camel.component.atlasmap.AtlasEndpoint.onExchange(AtlasEndpoint.java:209) at org.apache.camel.impl.ProcessorEndpoint$1.process(ProcessorEndpoint.java:71) at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61) at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148) at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548) at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) at org.apache.camel.processor.Pipeline.process(Pipeline.java:120) at org.apache.camel.processor.Pipeline.process(Pipeline.java:83) at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:110) at org.apache.camel.component.connector.ConnectorConsumerProcessor.process(ConnectorConsumerProcessor.java:66) at org.apache.camel.component.salesforce.SalesforceConsumer.processMessage(SalesforceConsumer.java:194) ... 35 common frames omitted ```
igarashitm commented 7 years ago

Easy short term fix would be to rely on camel converters, so in camel-atlasmap do getBody(String.class) to make sure always putting String into atlasmap. If we really need stream for huge content handling, then we'll need to rewrite the JsonModule and XmlModule using stream based parser, JsonParser and SAX, instead of ObjectMapper and DOM currently using.

igarashitm commented 7 years ago

Hmm that requires camel-atlasmap to look into source type and see if it's json/xml, which is dirty. So JsonModule/XmlModule should handle some type conversion anyway, which means we need to propagate the encoding of the stream as well.

igarashitm commented 7 years ago

Changed my mind, right now it looks better to deserialize into String in camel-atlasmap if source is JSON/XML.

igarashitm commented 7 years ago

Reserve this as a big enhancement to rewrite JsonModule and XmlModule to stream based. atlasmap/camel-atlasmap#26 will address a short term fix.