Add library version information in GET /version

fgalan commented 4 years ago

It would be an interesting feature that Orion informs of the exact versions of libraries on which it depends in the GET /version operation. Something like this (draft):

{
"orion" : {
  "version" : "2.4.0_20200802203650",
  "uptime" : "0 d, 0 h, 10 m, 16 s",
  "git_hash" : "4bf835364feec277dbcd5e146f4077494151e3c9",
  "compile_time" : "Sun Aug 2 20:40:45 CEST 2020",
  "compiled_by" : "jenkins",
  "compiled_in" : "int2-iot-jenkins-fe-01",
  "release_date" : "Sun Aug 2 20:40:45 CEST 2020",
  "doc" : "https://fiware-orion.rtfd.io/",
  "libversions": {
     "boost": "1.53",
     "libcurl": "7.29.0",
     "libmicrohttpd": "0.9.70",
     "libuuid": "2.23.2",
     "mongodriver": "legacy-1.1.2",
     "openssl": "1.0.2k",
     "rapidjson": "1.1.0"
  }
}
}

Two things to take into account:

Backward compability. Although probably this is not actually a problem, as we are adding a new element to version ("libversions") but not modifying any existin one. A similar case occur when we added "doc" element time ago and if it was not a problem then, it is not a problem now.
How to get the versions information. In order to so an accurate information, the information should be get from the library itself, avoiding any "hardwire" of the version numbers in the Orion Context Broker code.

cesarjorgemartinez commented 4 years ago

It's a very interesting feature for DevOps. Will it be for version 2.5.0?

Regards, Cesar Jorge

flopezag commented 4 years ago

Are you taking into account security? It would provide sensitive security information about the system.

jason-fox commented 4 years ago

I agree with @flopezag on this, a REST endpoint isn't the right place to hold metadata information about the build where any potential attacker could use the information to exploit the security.

For DevOps I assume you are using Docker containers . I would the assume you would be better off just adding LABEL attributes to the image and running docker inspect

e.g:

docker inspect --format='{{json .Config.Labels}}' fiware/idm | python -m json.tool

{
    "maintainer": "FIWARE Identity Manager Team. DIT-UPM",
    "org.nodejs.version": "10",
    "org.opencontainers.image.authors": "",
    "org.opencontainers.image.description": "OAuth2-based authentication of users and devices, user profile management, Single Sign-On (SSO) and Identity Federation across multiple administration domains.",
    "org.opencontainers.image.documentation": "https://fiware-idm.readthedocs.io/",
    "org.opencontainers.image.licenses": "MIT",
    "org.opencontainers.image.source": "https://github.com/ging/fiware-idm",
    "org.opencontainers.image.title": "Identity Manager - Keyrock",
    "org.opencontainers.image.vendor": "Universidad Polit\u00e9cnica de Madrid."
}

The names of many of the attributes are standardized. If you need additional custom ones, you could add dynamic labels to the Docker build hook.

See this open PR as an example: https://github.com/telefonicaid/fiware-orion/pull/3679

fgalan commented 4 years ago

Are you taking into account security? It would provide sensitive security information about the system.

ContextBroker version itself and githash commit are already sensitive security information being shown ;)

It is a matter of allowing this operation only to authorized users (e.g. using a PEP).

fgalan commented 4 years ago

I agree with @flopezag on this, a REST endpoint isn't the right place to hold information about the build.

For DevOps I assume you are using Docker containers . I would the assume you would be better off just adding LABEL attributes to the image and running docker inspect

e.g:
docker inspect --format='{{json .Config.Labels}}' fiware/idm | python -m json.tool
{
    "maintainer": "FIWARE Identity Manager Team. DIT-UPM",
    "org.nodejs.version": "10",
    "org.opencontainers.image.authors": "",
    "org.opencontainers.image.description": "OAuth2-based authentication of users and devices, user profile management, Single Sign-On (SSO) and Identity Federation across multiple administration domains.",
    "org.opencontainers.image.documentation": "https://fiware-idm.readthedocs.io/",
    "org.opencontainers.image.licenses": "MIT",
    "org.opencontainers.image.source": "https://github.com/ging/fiware-idm",
    "org.opencontainers.image.title": "Identity Manager - Keyrock",
    "org.opencontainers.image.vendor": "Universidad Polit\u00e9cnica de Madrid."
}
The names of many of the attributes are standardized. If you need additional custom ones, you could add dynamic labels to the Docker build hook.

See this open PR as an example: #3679

The problem with this is:

As far as a understand it is based in a key-value system and the values for the keys are "hardwired". I mean, they are not taken from the library itself. Thus, the following is not accomplished: "How to get the versions information. In order to so an accurate information, the information should be get from the library itself, avoiding any "hardwire" of the version numbers in the Orion Context Broker code."
It doesn't work in the case of using CB without docker (i.e. RPM or built from sources)

jmcanterafonseca commented 4 years ago

Are you taking into account security? It would provide sensitive security information about the system.

ContextBroker version itself and githash commit are already sensitive security information being shown ;)

It is a matter of allowing this operation only to authorized users (e.g. using a PEP).

debug-wise the feature is interesting but not necessarily has to be offered as a REST interface. it could be offered only to a trusted user. for instance, as a command line interface utility available on the docker image.

kzangeli commented 4 years ago

It surprises me that you worry about security here. This is open source. If anybody is interested in the versions of the libs, all he has to do is to clone the repo and look at the makefile. Or, compile the broker and run 'ldd' on it. Easy as pie.

fgalan commented 4 years ago

Fully agree with @kzangeli

Moreover, it can be even simpler. It is just a matter of:

Do GET /versions
Get the value of git_hash. Let it be 4bf835364feec277dbcd5e146f4077494151e3c9
Go to the repository to https://github.com/telefonicaid/fiware-orion/commit/4bf835364feec277dbcd5e146f4077494151e3c9
Click "Browse files" and navigate to https://github.com/telefonicaid/fiware-orion/blob/4bf835364feec277dbcd5e146f4077494151e3c9/doc/manuals/admin/build_source.md
Voila! You have the full library version information just under "The Orion Context Broker uses the following libraries as build dependencies"

Thus, the only way of protecting the CB in this way is completely forbidding the GET /version (using a PEP, for instance). Including a "libversions" elements in this operation is not going to make security worse or better ;)

Anjali-NEC commented 4 years ago

Hi @fgalan @kzangeli @jmcanterafonseca @flopezag @jason-fox @cesarjorgemartinez sir,

I would like to work on this issue, please let me know if no one is working on this. As per my understanding, we need to add versions of libraries on which it depends in the GET /version operation.

I have partially fixed this issue By adding exact versions of libraries in https://github.com/telefonicaid/fiware-orion/blob/master/src/lib/common/defaultValues.h and modify the code for GET /version operation in https://github.com/telefonicaid/fiware-orion/blob/master/src/lib/serviceRoutines/versionTreat.cpp

Please confirm the above understanding and approch.

fgalan commented 4 years ago

Assigned. Thanks for your interest in this issue!

With regards to your approach, one comment. It would be better to take the version library from the library itself, instead of from our own code in Orion. For instance, libmicrohttpd seems to have a constant MHD_VERSION which provided the version number in hexadecimal (it would need to be converted to x.y.z format, of course). Other libraries maybe have similar constants.

Anjali-NEC commented 4 years ago

Hi @fgalan sir,

I have fixed this issue and below is the output of GET /version: getv

However, There is no way has been found to get the versions of libuuid and mongodriver in existing code. So, I have hard coded these versions.

Below is the screenshot for the same.
llll

Please confirm the above approch.

fgalan commented 4 years ago

Looks much better this way :)

Some extra comments:

It would be great if you could transform "0x00097000" to something more useful for users in the x.y.z from, such as"0.9.70". Not sure how it works but it seems there is a correspondence between the digits in MDH_VERSION and the x.y.z form.
I think that better than provide a hardwired value, uuid and mongo versions should be removed. However, let me have a closer look at these libraries in the case I can find something useful

Thanks for your work!

fgalan commented 4 years ago

I think that better than provide a hardwired value, uuid and mongo versions should be removed. However, let me have a closer look at these libraries in the case I can find something useful

In the case of mongo driver, please have a look at mongo/version.h. There is interesting stuff there, specially kVersionString constant.

With regards to uuid, I'm afraid I don't find anything interesting in uuid/uuid.h, so if at the end it is not possible to find its version number, just remove it from the GET /version output. It is a pretty stable library and with a low utilization in Orion, so it is not a problem omitting its version.

Anjali-NEC commented 4 years ago

Hi @fgalan Sir,

It would be great if you could transform "0x00097000" to something more useful for users in the x.y.z from, such as"0.9.70". Not sure how it works but it seems there is a correspondence between the digits in MDH_VERSION and the x.y.z form.

I have not found any way to transform MDH_VERSION in the x.y.z from.

In the case of mongo driver, please have a look at mongo/version.h. There is interesting stuff there, specially kVersionString constant.

I have tried multiple approaches to get version of mongo driver in Output by including mongo/version.h or defining kVersionString constant but I am unable to find any suitable approach. So as per my suggestion it should be removed.

fgalan commented 4 years ago

I have not found any way to transform MDH_VERSION in the x.y.z from.

Looking at how MDH_VERSION looks like:

0x00097000

if we ignore the two last digits, we have:

00 -> 0 -> 0. ... 09 -> 9 -> ... .9. ... 70 -> 70 -> ... .70

It's a bit weird as 70 in hexadecimal is not 70 in decimal, but it seems to work so. Mabye asking how it works to the libmicrohttpd devel list would be a good idea.

fgalan commented 4 years ago

I have tried multiple approaches to get version of mongo driver in Output by including mongo/version.h or defining kVersionString constant but I am unable to find any suitable approach. So as per my suggestion it should be removed.

I'd have a look and try to use in in a proof-of-concept. Apparently, it should be easy to get that information...

kzangeli commented 4 years ago

I implemented this for Orion-LD. Will post here once I have a minute.

On Tue, Oct 27, 2020, 13:50 Fermín Galán Márquez notifications@github.com wrote:

I have tried multiple approaches to get version of mongo driver in Output by including mongo/version.h or defining kVersionString constant but I am unable to find any suitable approach. So as per my suggestion it should be removed.

I'd have a look and try to use in in a proof-of-concept. Apparently, it should be easy to get that information...

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/telefonicaid/fiware-orion/issues/3681#issuecomment-717220662, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABIXKDC3EL5VH7GLLRJIRITSM26YVANCNFSM4PWK3QNA .

fgalan commented 4 years ago

Mabye asking how it works to the libmicrohttpd devel list would be a good idea.

I did it :) See https://lists.gnu.org/archive/html/libmicrohttpd/2020-10/msg00008.html

A promising function named MHD_get_version() is cited in that email. Could you have a look to it, please?

Anjali-NEC commented 4 years ago

Hi @fgalan sir,

A promising function named MHD_get_version() is cited in that email. Could you have a look to it, please?

I have updated in my PR #3709

fgalan commented 4 years ago

Once PR https://github.com/telefonicaid/fiware-orion/pull/3709 has been merged, some extra fixes need to be done. I'll take care of them.

fgalan commented 4 years ago

Finished in PR https://github.com/telefonicaid/fiware-orion/pull/3713

Will post here once I have a minute.

@kzangeli this is not necesary. I found the way at the end. Thanks anyway! :)

telefonicaid / fiware-orion

Add library version information in GET /version #3681