search

BradleyA / Linux-admin

Shell scripts to automate download of GitHub traffic statistics, cluster administration, and create an animated GIF.
MIT License
29 stars 6 forks source link

Design and develop a couple of shell scripts for GitHub repository traffic history more than 14 days in README.md #29

Closed BradleyA closed 4 years ago

BradleyA commented 5 years ago

Present GitHub repository traffic history more than 14 days in README

Unable to locate available solution that did NOT require logging into a web site with my GitHub account for a service or installing more tools or sharing my GitHub token with people I do not know or a service that limit number of repositories.

Design and develop a couple of shell scripts to do the following:

Solution: github-repository-traffic

BradleyA commented 5 years ago

https://developer.github.com/v3/ terminal: Basic authentication curl -u "BradleyA" https://api.github.com

In this example, the 'BradleyA' and 'Search-docker-registry-v2-script.1.0' values are provided for the :owner and :repo parameters in the path while :state is passed in the query string. curl -i "https://api.github.com/repos/BradleyA/Search-docker-registry-v2-script.1.0/issues?state=closed"

Conditional requests Most responses return an ETag header. Many responses also return a Last-Modified header. You can use the values of these headers to make subsequent requests to those resources using the If-None-Match and If-Modified-Since headers, respectively. If the resource has not changed, the server will return a 304 Not Modified.

Note: Making a conditional request and receiving a 304 response does not count against your Rate Limit, so we encourage you to use it whenever possible.

curl -i https://api.github.com/users/BradleyA

. . . { "login": "BradleyA", "id": 12129046, "node_id": "MDQ6VXNlcjEyMTI5MDQ2", "avatar_url": "https://avatars3.githubusercontent.com/u/12129046?v=4", "gravatar_id": "", "url": "https://api.github.com/users/BradleyA", "html_url": "https://github.com/BradleyA", "followers_url": "https://api.github.com/users/BradleyA/followers", "following_url": "https://api.github.com/users/BradleyA/following{/other_user}", "gists_url": "https://api.github.com/users/BradleyA/gists{/gist_id}", "starred_url": "https://api.github.com/users/BradleyA/starred{/owner}{/repo}", "subscriptions_url": "https://api.github.com/users/BradleyA/subscriptions", "organizations_url": "https://api.github.com/users/BradleyA/orgs", "repos_url": "https://api.github.com/users/BradleyA/repos", "events_url": "https://api.github.com/users/BradleyA/events{/privacy}", "received_events_url": "https://api.github.com/users/BradleyA/received_events", "type": "User", "site_admin": false, "name": "Bradley Allen", "company": "@bradleyaustintx", "blog": "", "location": "Cedar Park, Texas", "email": null, "hireable": true, "bio": "Bradley Allen has over 35 years of experience in global IT management, data center operations, software development, UNIX administration, project management ...", "public_repos": 28, "public_gists": 0, "followers": 3, "following": 0, "created_at": "2015-04-27T00:30:56Z", "updated_at": "2019-06-21T23:40:13Z" }

==>

curl -u "BradleyA" -i https://api.github.com/repos/BradleyA/Search-docker-registry-v2-script.1.0/traffic/clones Enter host password for user 'BradleyA': HTTP/1.1 200 OK Server: GitHub.com Date: Mon, 29 Jul 2019 01:40:32 GMT Content-Type: application/json; charset=utf-8 Content-Length: 330 Status: 200 OK X-RateLimit-Limit: 5000 X-RateLimit-Remaining: 4999 X-RateLimit-Reset: 1564368032 Cache-Control: private, max-age=60, s-maxage=60 Vary: Accept, Authorization, Cookie, X-GitHub-OTP ETag: "76f2af46340196485a6316f8aec60f51" X-GitHub-Media-Type: github.v3; format=json Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type Access-Control-Allow-Origin: * Strict-Transport-Security: max-age=31536000; includeSubdomains; preload X-Frame-Options: deny X-Content-Type-Options: nosniff X-XSS-Protection: 1; mode=block Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin Content-Security-Policy: default-src 'none' X-GitHub-Request-Id: D140:386C:215F816:3E29A00:5D3E4E90

{ "count": 5, "uniques": 4, "clones": [ { "timestamp": "2019-07-16T00:00:00Z", "count": 1, "uniques": 1 }, { "timestamp": "2019-07-19T00:00:00Z", "count": 1, "uniques": 1 }, { "timestamp": "2019-07-24T00:00:00Z", "count": 3, "uniques": 2 } ] }

===> curl -u "BradleyA" -i https://api.github.com/repos/BradleyA/Search-docker-registry-v2-script.1.0/traffic/views Enter host password for user 'BradleyA': HTTP/1.1 200 OK Date: Mon, 29 Jul 2019 01:46:19 GMT Content-Type: application/json; charset=utf-8 Content-Length: 1365 Server: GitHub.com Status: 200 OK X-RateLimit-Limit: 5000 X-RateLimit-Remaining: 4999 X-RateLimit-Reset: 1564368379 Cache-Control: private, max-age=60, s-maxage=60 Vary: Accept, Authorization, Cookie, X-GitHub-OTP ETag: "dc942f625f325768c2ed8cbda4a322e4" X-GitHub-Media-Type: github.v3; format=json Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type Access-Control-Allow-Origin: * Strict-Transport-Security: max-age=31536000; includeSubdomains; preload X-Frame-Options: deny X-Content-Type-Options: nosniff X-XSS-Protection: 1; mode=block Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin Content-Security-Policy: default-src 'none' Vary: Accept-Encoding X-GitHub-Request-Id: E812:1B0E:1F622A:3189F3:5D3E4FEA

{ "count": 157, "uniques": 74, "views": [ { "timestamp": "2019-07-15T00:00:00Z", "count": 7, "uniques": 4 }, { "timestamp": "2019-07-16T00:00:00Z", "count": 4, "uniques": 3 }, { "timestamp": "2019-07-17T00:00:00Z", "count": 15, "uniques": 8 }, { "timestamp": "2019-07-18T00:00:00Z", "count": 10, "uniques": 4 }, { "timestamp": "2019-07-19T00:00:00Z", "count": 12, "uniques": 6 }, { "timestamp": "2019-07-20T00:00:00Z", "count": 4, "uniques": 3 }, { "timestamp": "2019-07-21T00:00:00Z", "count": 3, "uniques": 2 }, { "timestamp": "2019-07-22T00:00:00Z", "count": 15, "uniques": 7 }, { "timestamp": "2019-07-23T00:00:00Z", "count": 13, "uniques": 5 }, { "timestamp": "2019-07-24T00:00:00Z", "count": 22, "uniques": 14 }, { "timestamp": "2019-07-25T00:00:00Z", "count": 22, "uniques": 8 }, { "timestamp": "2019-07-26T00:00:00Z", "count": 18, "uniques": 10 }, { "timestamp": "2019-07-27T00:00:00Z", "count": 3, "uniques": 1 }, { "timestamp": "2019-07-28T00:00:00Z", "count": 9, "uniques": 2 } ] }

==>

curl -u "BradleyA" -i https://api.github.com/repos/BradleyA/Search-docker-registry-v2-script.1.0/traffic/clones?per=week Enter host password for user 'BradleyA': HTTP/1.1 200 OK Server: GitHub.com Date: Mon, 29 Jul 2019 01:53:05 GMT Content-Type: application/json; charset=utf-8 Content-Length: 237 Status: 200 OK X-RateLimit-Limit: 5000 X-RateLimit-Remaining: 4995 X-RateLimit-Reset: 1564368785 Cache-Control: private, max-age=60, s-maxage=60 Vary: Accept, Authorization, Cookie, X-GitHub-OTP ETag: "fc945a70ebebd04d124270253b312d06" X-GitHub-Media-Type: github.v3; format=json Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type Access-Control-Allow-Origin: * Strict-Transport-Security: max-age=31536000; includeSubdomains; preload X-Frame-Options: deny X-Content-Type-Options: nosniff X-XSS-Protection: 1; mode=block Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin Content-Security-Policy: default-src 'none' X-GitHub-Request-Id: CD1C:3989:16B780A:2F81BED:5D3E5180

{ "count": 5, "uniques": 4, "clones": [ { "timestamp": "2019-07-15T00:00:00Z", "count": 2, "uniques": 2 }, { "timestamp": "2019-07-22T00:00:00Z", "count": 3, "uniques": 2 } ] }

==> curl -u "BradleyA" -i https://api.github.com/repos/BradleyA/Search-docker-registry-v2-script.1.0/traffic/popular/referrers curl -u "BradleyA" -i https://api.github.com/repos/BradleyA/Search-docker-registry-v2-script.1.0/traffic/popular/referrers Enter host password for user 'BradleyA': HTTP/1.1 200 OK Date: Mon, 29 Jul 2019 01:58:46 GMT Content-Type: application/json; charset=utf-8 Content-Length: 381 Server: GitHub.com Status: 200 OK X-RateLimit-Limit: 5000 X-RateLimit-Remaining: 4993 X-RateLimit-Reset: 1564369126 Cache-Control: private, max-age=60, s-maxage=60 Vary: Accept, Authorization, Cookie, X-GitHub-OTP ETag: "5ca10814b2153bfc54ac51dc4b1fc536" X-GitHub-Media-Type: github.v3; format=json Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type Access-Control-Allow-Origin: * Strict-Transport-Security: max-age=31536000; includeSubdomains; preload X-Frame-Options: deny X-Content-Type-Options: nosniff X-XSS-Protection: 1; mode=block Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin Content-Security-Policy: default-src 'none' Vary: Accept-Encoding X-GitHub-Request-Id: C8EE:098C:6C8F2:A398D:5D3E52D6

[ { "referrer": "forums.docker.com", "count": 79, "uniques": 37 }, { "referrer": "github.com", "count": 8, "uniques": 5 }, { "referrer": "stackoverflow.com", "count": 7, "uniques": 3 }, { "referrer": "Google", "count": 3, "uniques": 1 }, { "referrer": "docs.feedzai.com", "count": 1, "uniques": 1 } ]

==> curl -u "BradleyA" -i https://api.github.com/repos/BradleyA/Search-docker-registry-v2-script.1.0/traffic/popular/paths Enter host password for user 'BradleyA': HTTP/1.1 200 OK Server: GitHub.com Date: Mon, 29 Jul 2019 02:00:13 GMT Content-Type: application/json; charset=utf-8 Content-Length: 2069 Status: 200 OK X-RateLimit-Limit: 5000 X-RateLimit-Remaining: 4992 X-RateLimit-Reset: 1564369213 Cache-Control: private, max-age=60, s-maxage=60 Vary: Accept, Authorization, Cookie, X-GitHub-OTP ETag: "823aec79db3312e97bfea8873168b05e" X-GitHub-Media-Type: github.v3; format=json Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type Access-Control-Allow-Origin: * Strict-Transport-Security: max-age=31536000; includeSubdomains; preload X-Frame-Options: deny X-Content-Type-Options: nosniff X-XSS-Protection: 1; mode=block Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin Content-Security-Policy: default-src 'none' X-GitHub-Request-Id: D2A4:6AB8:17C33C1:316241C:5D3E532D

[ { "path": "/BradleyA/Search-docker-registry-v2-script.1.0", "title": "GitHub - BradleyA/Search-docker-registry-v2-script.1.0: view-private-registry...", "count": 81, "uniques": 63 }, { "path": "/BradleyA/Search-docker-registry-v2-script.1.0/blob/master/view-private-registry", "title": "Search-docker-registry-v2-script.1.0/view-private-registry at master · Bradle...", "count": 29, "uniques": 25 }, { "path": "/BradleyA/Search-docker-registry-v2-script.1.0/blob/master/view-private-registry-remote", "title": "Search-docker-registry-v2-script.1.0/view-private-registry-remote at master ·...", "count": 26, "uniques": 21 }, { "path": "/BradleyA/Search-docker-registry-v2-script.1.0/issues/8", "title": "docker registry setup and IP SANs · Issue BradleyA/Search-docker-registry-v2-script.1.0#8 · BradleyA/Search-docker-registr...", "count": 3, "uniques": 3 }, { "path": "/BradleyA/Search-docker-registry-v2-script.1.0/tree/master/images", "title": "Search-docker-registry-v2-script.1.0/images at master · BradleyA/Search-docke...", "count": 3, "uniques": 3 }, { "path": "/BradleyA/Search-docker-registry-v2-script.1.0/pulse", "title": "Pulse · BradleyA/Search-docker-registry-v2-script.1.0 · GitHub", "count": 3, "uniques": 2 }, { "path": "/BradleyA/Search-docker-registry-v2-script.1.0/graphs/traffic", "title": "Traffic · BradleyA/Search-docker-registry-v2-script.1.0", "count": 2, "uniques": 1 }, { "path": "/BradleyA/Search-docker-registry-v2-script.1.0/blob/master/README.md", "title": "Search-docker-registry-v2-script.1.0/README.md at master · BradleyA/Search-do...", "count": 1, "uniques": 1 }, { "path": "/BradleyA/Search-docker-registry-v2-script.1.0/issues", "title": "Issues · BradleyA/Search-docker-registry-v2-script.1.0", "count": 1, "uniques": 1 }, { "path": "/BradleyA/Search-docker-registry-v2-script.1.0/search", "title": "Search · GET /BradleyA/repos", "count": 1, "uniques": 1 } ]

browser: https://api.github.com/repos/BradleyA/Search-docker-registry-v2-script.1.0/releases

BradleyA commented 5 years ago

display Clones and Views with :+1: 

<img alt="Clones" src="https://img.shields.io/static/v1?label=CLones&message=26,24,1&color=blue"> <img alt="Views" src="https://img.shields.io/static/v1?label=Views&message=26,24,1&color=blue">
BradleyA commented 5 years ago 
#       column -t -s' ' filename
#       soffice --convert-to png ./clones
#       display ./clones.png
#
#       <img alt="Steam Views" src="https://img.shields.io/steam/views/100">
#
#       cd ~/github/BradleyA/automate/:repo
#       git pull
#       git commit -m '$DATE: automation the update of README table' README.md
#       git push README.md
#
BradleyA commented 5 years ago 
github-repository/parse.repository.data.sh  2.98.370  2019-08-08T23:47:37.538761-05:00 (CDT)  https://github.com/BradleyA/Linux-admin  uadmin  two-rpi3b.cptx86.com 2.97  
    github-repository/parse.repository.data.sh design complete, ready to create test cases
BradleyA commented 4 years ago

Solution: github-repository-traffic