Hurence / logisland

Scalable stream processing platform for advanced realtime analytics on top of Kafka and Spark. LogIsland also supports MQTT and Kafka Streams (Flink being in the roadmap). The platform does complex event processing and is suitable for time series analysis. A large set of valuable ready to use processors, data sources and sinks are available.
https://logisland.github.io
Other
110 stars 28 forks source link

Feature/url cleanner #568

Closed MiniPlayer closed 3 years ago

MiniPlayer commented 3 years ago

Added URLCleaner, see issue #567 .

In addition to those changes, I cleaned up the URLDecoder by removing unused properties. And adding error in record when the parsing fails (log in an error as well in logs). I changed the log level from WARN to ERROR, maybe this could be an issue (log spamming) ? Tell me if this is the case.

I also added a URIDecoder processor that decode an URI using the java.net.URI. But the string being decoded must be a correctly formed URI (so not already decoded for exemple as it may result in illegal characters).