knorrie
commented
2 years ago Hi, thanks for your question. It took a while to get back to you. :|
- Is there a "simple" method to generically extract the stats directly from the Mendix admin socket? (ie. without the Python?) This might be the "beterer" method, as I can then point my GoLang colleague in the direction of the Admin port's stats documentation (which I haven't yet found) and life would be great again [...]
There's no real documentation about the Runtime Admin API because it's an "internal" API. This means so much as "if you're doing anything with it, don't ask Mendix Support if the API changes etc". But, it's also a very simple and stable API, for many years already. If you run the m2ee cli program with -vvv (or dump the traffic to/from the port) you can already see the requests and responses flying by.
E.g. if the Admin API password would be 'hevisko' (and port 9k), then the following would give you part of the stats already: curl -X POST http://localhost:9000 -H "X-M2EE-Authentication:aGV2aXNrbw==" -H "Content-Type:application/json" --data '{"action": "runtime_statistics", "params": {}}' where the garbage is simply the base64 of the password. In src/m2ee/client.py you can see the function in Python that does the same, and in the rest of the client.py file, all actions that are implemented. This should be fairly trivial to reimplement in Go, it's really nothing more than a simple json POST request. (Hint: look at the develop branch, not master)
Failing that "simple"/documented/etc. API to the stats, I see the need for a plugin, but that plugin would then be either:
(a) a major hack to parse Munin output of
m2ee
Lol, no.
(b) I integrate it as part of the
m2eeand provide a PR for you to integrate inm2ee-tools
So, basically, all you need to do (when running this under the same system user permissions as the application is running) to get and pretty print all the stats is:
#!/usr/bin/python3
import m2ee
import pprint
m2 = m2ee.M2EE()
stats, java_version = m2ee.munin.get_stats('values', m2)
pprint.pprint(stats)So you can do import m2ee from any other Python program and get the stats, look at what's in there and then do whatever you need to do. (For the munin plugin in here, it's just a bunch of print statements that print the values.)
[Note: The difference between the result of this get_stats and calling the Admin API actions directly is that get_stats has post-processing code which deals with Mendix versions that are too old to have certain information, to hotfix stats from some old versions which have bugs putting values in the wrong place. But, I actually think all of that fixup code is already for Mendix versions which are older than what's officially supported now...]
Which route would Mendix etc. advise me to follow?
I'd say, either simply 'reverse engineer' the simple Admin API calls for runtime_statistics and server_statistics and work with the results directly.
[Note: the get_stats also uses the get_all_thread_stack_traces API call in the process. You want to avoid using this if you want to gather stats frequently (munin is just once every 5 minutes) because I suspect it causes a stop-the-world in the JVM.]
In the case of (b), any advice specifics to follow?
No, not really. I would really recommend starting with a single-purpose proof of concept for yourself.
I mean, if you know you're running on a standardized combination of JVM and Mendix versions for your own apps, then you only have to look at what kind of stats you get once, and tetris them somewhere in place so that your netdata system starts showing useful graphs.
Hans
hvisage
Good day, Years ago (as in 2014 when I first got to administer a Mendix stack), Munin was the thing, and though it's not bad per se, I've been busy deploying https://Netdata.cloud as the next big thing and one of the "funs" for me is the Mendix stacks I manage, so I started to consider to import the stats into Netdata as discussed here https://learn.netdata.cloud/docs/agent/collectors/python.d.plugin but then I started to think and my questions:
Failing that "simple"/documented/etc. API to the stats, I see the need for a plugin, but that plugin would then be either: (a) a major hack to parse Munin output of
m2ee(okay, that might be a general help for more munin people, but that feels just like a yet another trailer at the back that can fail type solution) or (b) I integrate it as part of them2eeand provide a PR for you to integrate inm2ee-toolsIn the case of (b), any advice specifics to follow?