search

cernopendata / opendata.cern.ch

Source code for the CERN Open Data portal
http://opendata.cern.ch/
GNU General Public License v2.0
661 stars 147 forks source link

nginx monitoring: Check categories #3476

Open psaiz opened 10 months ago

psaiz commented 10 months ago

We collect statistics based on the nginx access monitoring. The information is available here

The accesses can be split into categories, which are done based on invenio [here] (https://github.com/inveniosoftware/helm-invenio/blob/master/invenio/templates/configurations/logstash.yaml#L112)

We should verify that the categories make sense for opendata.

psaiz commented 10 months ago

I'm taking this snippet from an email from Tibor from the 2.10.2023. The email described work done to understand the CODP usage analysis, and it identifies different URL that were combined into categories

   Distinct IPs overall, firewall logs (/eos): 496
   Distinct IPs overall, firewall logs (/about): 3946
   Distinct IPs overall, firewall logs (/collection): 18773
   Distinct IPs overall, firewall logs (/visualise/events): 14054
   Distinct IPs overall, firewall logs (/visualise/histograms): 2967
   Distinct IPs overall, firewall logs (/record): 19033
   Distinct IPs overall, firewall logs (/search): 8670
   Distinct IPs overall, firewall logs (/): 72752
   Distinct IPs overall, firewall logs (*): 97679