search

ThejanRupasinghe / ei-elastic-custom-publisher

A custom observer for WSO2 Enterprise Integrator Analytics which publishes data to Elasticsearch. https://docs.wso2.com/display/EI611/Customizing+Statistics+Publishing
2 stars 6 forks source link

Lack of published data fields in Elastic #4

Open amerireza opened 5 years ago

amerireza commented 5 years ago

I am using this component and it works well but, I can see so many data when I use ei-analytics but published data from ei-elastic publisher is just a few of them. These are fields I get when I use ei-analytics.

metaTenantId | messageFlowId | host | hashCode | componentName | componentType | componentIndex | componentId | startTime | endTime | duration | beforePayload | afterPayload | contextPropertyMap | transportPropertyMap | children | entryPoint | entryPointHashcode | faultCount | eventTimestamp

ThejanRupasinghe commented 5 years ago

Yes. This component was developed in relation to the requirement explained in this article. So it only extracts a few fields from the PublishingFlow which are needed to build the visualizations in Kibana.

It can be improved to extract everything you have mentioned, as it is done here. https://github.com/ThejanRupasinghe/ei-elastic-custom-publisher/blob/d141e40c4757900df9c8562b280ee1fb8419229b/src/main/java/org/wso2/custom/elastic/publisher/publish/ElasticStatisticsPublisher.java#L62-L107

amerireza commented 5 years ago

OK, I wanna add other event types but I don't know how can I see all types. What is the abstract or parent class of these all event types?

ThejanRupasinghe commented 5 years ago

PublishingFlow from wso2-synapse is intercepted in the above shown code. It has an array of PublishingEvents, which the message flow has gone through. You can iterate through it and extract the needed fields.

amerireza commented 5 years ago

Thank you for the information. But the componentIndex variable is private and eventTimestamp variables are not existing in the PublishingEvents.java class.

ThejanRupasinghe commented 5 years ago

Yes. componentIndex is private, but can access event related times from here. ElasticMediationFlowObserver is written using the MessageFlowObserver interface exposed by the EI Analytics profile, so this is the only visibility we have here.

amerireza commented 5 years ago

What do you say, if we add a getter for the componentIndex in the PublishingEvent.java to be able to retrieve the value, in the next versions?

ThejanRupasinghe commented 5 years ago

Yes. We with that change we should be able to access componentIndex here.

amerireza commented 5 years ago

Yes. We with that change we should be able to access componentIndex here.

I have committed my changes here and it is waiting for review.