search

ControlSystemStudio / cs-studio

Control System Studio is an Eclipse-based collections of tools to monitor and operate large scale control systems, such as the ones in the accelerator community.
https://controlsystemstudio.org/
Eclipse Public License 1.0
111 stars 96 forks source link

Contribute Elasticsearch Messagelog history #2579

Open MichaelRitzert opened 5 years ago

MichaelRitzert commented 5 years ago

I'd like to contribute the Elasticsearch based logging solution that we have been using for a while now. There are a few parts building on each other: – a very simple interface to Elasticsearch (via JSON queries and HTTP) – a model merging initial data retrieved from a database with live data coming in via JMS + an implementation using ES as the database. From the design of the code, it should also be possible to include PostgreSQL as another option. – the corresponding GUI forked from org.csstudio.alarm.beast.msghist. This also includes an OPI widget.

Currently, I'm using org.json and com.sun.jersey, so this targets Java 8 and the CSS 4.5.x branch. For Java 11+ / CSS master, switching the the respective standard solutions (javax.json and java.net.http) is prepared.

If there is interest, I'd prepare a pull request to have it in the main release. Any proposal where such code could go?

Additionally, there's the C++ library that we use to perform logging via JMS from the IOCs. Most likely this will just go on our github page as its own project.

shroffk commented 5 years ago

Hello, You can create a new module org.csstudio.logging.es under the core/utility module.

I am not sure about the C++ libraries location.

MichaelRitzert commented 5 years ago

OK. I will create a pull request for the actual code. We will then have to see about how to integrate it in the build, i.e. features and repo.

MichaelRitzert commented 5 years ago

I'm depending on too many things from outside core, so I don't think it can go there. There are at least dependencies on org.csstudio.apputil for StartEndDialog, and additionally org.csstudio.opibuilder for the OPI widget.

shroffk commented 5 years ago

ok, you are right...if it depends on non core things then it might be best to move it to applications

shroffk commented 5 years ago

I think diag or appunorganized would both work

MichaelRitzert commented 5 years ago

I think I have the build working with the ElasticSearch client in utilities and the rest of the code in appunorganized. The remaining question is about the features. The ES client is tiny, I just added this to core.utility.feature. The ES client is a bit larger. Shall I create a new feature for it?

berryma4 commented 5 years ago

Yes, please create a new feature.

MichaelRitzert commented 5 years ago

I updated the pull request accordingly. #2583

MichaelRitzert commented 4 years ago

So the pull request has been merged, but now the build again times out https://travis-ci.com/ControlSystemStudio/cs-studio/builds/123782993 so that the new features are not yet available in the p2 repo. How can we proceed?

berryma4 commented 4 years ago

I'll ask Travis ci for more time

On Fri, Aug 23, 2019, 06:13 Michael Ritzert notifications@github.com wrote:

So the pull request has been merged, but now the build again times out https://travis-ci.com/ControlSystemStudio/cs-studio/builds/123782993 so that the new features are not yet available in the p2 repo. How can we proceed?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ControlSystemStudio/cs-studio/issues/2579?email_source=notifications&email_token=AAGQQBJXVWBIA3OHRHZS3VTQF6Z4LA5CNFSM4HHWHGWKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD47Y56Y#issuecomment-524259067, or mute the thread https://github.com/notifications/unsubscribe-auth/AAGQQBO7RKWYDJDZH7YXNLLQF6Z4LANCNFSM4HHWHGWA .

berryma4 commented 4 years ago

more time has been added, and the build fails right at the end, out of memory. I put in a pull request to check the new memory addition.

[INFO] org.csstudio.desy.snl.feature ...................... SUCCESS [  0.210 s]
[INFO] snl-repository ..................................... FAILURE [01:54 min]
[INFO] applications-p2repo ................................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 59:26 min
[INFO] Finished at: 2019-08-23T17:11:23Z
[INFO] Final Memory: 1849M/1989M
[INFO] ------------------------------------------------------------------------
[ERROR] Java heap space -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/OutOfMemoryError
berryma4 commented 4 years ago

2611