Add the possibility to track bidscoin usage

[marcelzwiers]

Is your feature request related to a problem? Please describe. Currently, it is unclear how much and how bidscoin is used

Describe the solution you'd like Using an opt-in/out user setting (#197), I want to get information on:

• When a BIDScoin session is started
• The (hashed) user-ID + host-ID
• The country (optional)
• The function that is called
• The error(s) that occurred
• The python version + operating system

Perhaps this information can be gathered and put into a dashboard using a combination of:

• A BIDScoin domain on the DCCN Apache server that logs the tracking URL requests made from BIDScoin. Such a request could look like: https://bidscoin.dccn.nl/tracking?call=bidsmapper&user=1234&host=5678&[max_total_length_url=2048_characters]
• Logstash
• The DCCN Grafana server

Describe alternatives you've considered I've considered etelemetry (used by e.g. heudiconv, nipype -> fmriprep, mriqc, etc)

[robertoostenveld]

here is a part of the apache log (ip addresses on the left cut off)

[hurngchunlee]

Another option using google analytics: https://practicaldatascience.co.uk/data-science/how-to-send-data-to-google-analytics-in-python-using-pygamp

btw, some thoughts on the Apache/Logstash/Grafana approach:

Grafana only displays data in time series format. Actually data should be transformed into one of the time series databases (e.g. prometheus, OpenTSDB) supported by Grafana; or if general purpose database (SQL or Elastic) is used behind Grafana, certain data structure/schema should be followed.

If filebeat or logstash is used to parse web server log, it is beter to store the data in Elastic and use Kibana for display. The so-called ELK stack.

[marcelzwiers]

Perhaps, but pygamp only had 6 commits altogether and seems no longer supporting the new API? https://github.com/flyandlure/pygamp/issues/2

[marcelzwiers]

Is Grafana Loki perhaps a solution? https://grafana.com/docs/loki/latest/get-started/overview/

[hurngchunlee]

Is Grafana Loki perhaps a solution? https://grafana.com/docs/loki/latest/get-started/overview/

Grafana Loki is very interesting. It may be also useful for collecting and analyzing server logs concerning IT security. Will have a look into it.

[marcelzwiers]

The Loki <> Grafana option seems to be the way forward. You can now see bidscoin tracking with:

$ bidscoin --tracking show
INFO | trackusage = 'yes'       # Upload anonymous usage data if 'yes' (maximally 1 upload every 1 hour)
Data upload example: -> https://telemetry.donders.ru.nl/bidscoin
{
   event:       bidscoin
   bidscoin:    4.1.1
   python:      3.11.5
   system:      Linux
   release:     5.14.21-150500.55.19-default
   userid:      2feb5e843ccb54a41416f963a6c88a5d
   hostid:      e138231a68ad82f054e3d756c6634ba1
}

NB: As you can see above, BIDScoin does NOT upload any identifying details about you nor
information about the data. Uploaded data is used to generate usage and error statistics,
and helps the developers to improve BIDScoin