Restrict InfluxDB posts - Githubissues

darshank15 / GSoC_2020_FOSSOlogy

FOSSology generates a large set of data that is exported to the time-series influx database and visualized with the help of Grafana. I Wrote a fossdash_publisher script that collects useful data from FOSSology DB (Postgres) and exposes them to influx DB. Developed a visualization dashboard in Grafana by integrating influx as an input data source.

GNU General Public License v2.0

2 stars 0 forks source link

Restrict InfluxDB posts #12

Closed NicolasToussaint closed 4 years ago

NicolasToussaint commented 4 years ago

For InfluxDB:

disable Influx Admin account (already the case in docker-compose)
- [note] USER account seems to be necessary for Grafana to query the data (to be confirmed)
add authentication for to be allowed to post data (see InfluxDB Authentication)
- Ideally, have an authentication token for a given instance ID...

darshank15 commented 4 years ago

Done Authentication using the username and password. part1: Enable authentication in InfluxDB. ==> Commit : https://github.com/Orange-OpenSource/fossdash/pull/1/commits/beff4fcc6e1ef51afc03805b393db2a1d54e4c05 part2: In the publisher script, Added authentication head to authenticate the sending data request. ==> Commit : https://github.com/Orange-OpenSource/fossology/pull/3/commits/3b28e4a0486a9a730b890849c4c19fb689b1d64a

darshank15 commented 4 years ago

Currently, I have done authentication using a username and password for the database as discussed with @bartos

For Token-Based Authentication. I am not sure how we can generate a token for our specific database of InfluxDB. I am looking into it. https://docs.influxdata.com/influxdb/v1.8/administration/authentication_and_authorization#authenticate-using-jwt-tokens

darshank15 commented 4 years ago

Yes, We can do now token-based authentication as well,

But we Compulsory required expiration time ( UnixTimestamp formate ).

import jwt
key = 'fossology-fossdash-inlfuxdb-authentication-using-JWT'
payload_dict = { "username": "fossy", "exp": 1626772576}
encoded_token = jwt.encode(payload_dict, key, algorithm='HS256')
fossdash_url = "http://localhost:8086/write?db=fossology_db"
file_path = "temp_file"
file_data = open(file_path, 'rb').read()
Bearer_token = "Bearer "+encoded_token
header = {'Authorization': Bearer_token}
fossdash_response = requests.post(fossdash_url, headers=header,data=file_data)

darshank15 commented 4 years ago

Done the InfluxDB authentication using token-based. https://github.com/Orange-OpenSource/fossology/pull/3/commits/0f81a157edcab002c99f5e0206fd8d4496ed8784

darshank15 commented 4 years ago

Comes up with both types of authentication based on the dropdown selection.

Token-based authentication.
Username_password authentication.

As a discussion, we can't performs an administration related operation to influxDB.

As Fossology, We need to asks username and password assigned to us OR token assigned to us for pushing data to influxDB. We need to update these data though sysconfig UI page. Whenever the token gets expired.

NicolasToussaint commented 4 years ago

Works as expected

Wrong password leads to authentication error, as expected:

2020-08-20 10:19:02.085207
Fossology Instance UUID =  fossology-dev-2
All data metrics Generated
publishing FossDash URL = http://influxdb:8086/write?db=fossology_db
sending file to influxDB = /srv/fossology/repository/fossdash/fossdash-publish.20200820_101902
user_pass authentication 
status code = 401
{"error":"authorization failed"}

sending file to influxDB = /srv/fossology/repository/fossdash/fossdash-publish.20200820_101902
user_pass authentication 
status code = 401
{"error":"authorization failed"}

find_cmd =  find /srv/fossology/repository/fossdash -maxdepth 1 -iname "fossdash-publish*.reported" -ctime +4 > /srv/fossology/repository/fossdash/fossdash_clean_file_list

Concerning the use of token, it works well, but the process to create a token is not easy to find...