Identify the no of logins in a day, in past one hour and in past one minute

[ashay-maheshwari]

We need to get count of no. of users logging in to https://vlabs.ac.in. This information can be obtained to get the following counts -

1. No of logins in a day (today)
2. No of logins in past one hour
3. No of logins in past one minute

To solve this problem following things are to be identified -

1. Where can we get this information from ? What log files are storing this information.
2. What log lines are to be parsed for near to accurate information
3. A script to regularly parse log files and evaluate results
4. Output should be stored in a JSON format in a json file whose structure is to be decided.

[ashay-maheshwari]

/edx/var/log/tracking/tracking.log contains log lines which clearly indicates the login event of a user. The sample log line is given below -

{"username": "AShayM", "event_type": "/dashboard", "ip": "14.139.82.6", "agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36", "host": "vlabs-test.vlabs.ac.in", "referer": "https://vlabs-test.vlabs.ac.in/login", "accept_language": "en-GB,en-US;q=0.8,en;q=0.6", "event": "{\"POST\": {}, \"GET\": {}}", "event_source": "server", "context": {"user_id": 9, "org_id": "", "course_id": "", "path": "/dashboard"}, "time": "2017-04-13T09:04:44.910429+00:00", "page": null}

From the above log line we can see the following -

"username": "AShayM" "event_type": "/dashboard" "referer": "https://vlabs-test.vlabs.ac.in/login" "time": "2017-04-13T09:04:44.910429+00:00"

Key obtained from log lines clearly indicated that user with username AShayM tried to refer https://vlabs-test.vlabs.ac.in/login and successful called /dashboard event

[ashay-maheshwari]

Command to count the number of logins on date 2017-04-13 - cat tracking.log | grep "https://vlabs-test.vlabs.ac.in/login" | grep "/dashboard" | grep "2017-04-13T" | wc -l

Command to count the number of logins in past one hour -

date_now=date +"%Y-%m-%dT%H";cat tracking.log | grep "https://vlabs-test.vlabs.ac.in/login" | grep "/dashboard" | grep $date_now | wc -l

Command to count the number of logins in past one minute -

date_now=date +"%Y-%m-%dT%H:%M";cat tracking.log | grep "https://vlabs-test.vlabs.ac.in/login" | grep "/dashboard" | grep $date_now | wc -l

Above commands can be properly framed in a shell script which must execute every half a minute to update the output file which can be a sample JSON file in the format as given below -

{
  "logins-in-a-day" : x1,
  "logins-in-an-hour" :  x2,
  "logins-in-a-minute" : x3
}

[ashay-maheshwari]

Duplicate of https://github.com/vlead/analytics-dashboard/issues/2