search

super3 / yacdn.org

Yet Another CDN.
https://yacdn.org
GNU Affero General Public License v3.0
26 stars 4 forks source link

Logging and Performance Data #41

Closed montyanderson closed 5 years ago

montyanderson commented 5 years ago
montyanderson commented 5 years ago

@super3 I've implemented logging for serve routes.

serve#58 url: http://meowbaari.com/wp-content/uploads/2016/06/1464933654_cat_sleep.png
serve#58 url hash: 26d984569a517733fe19a5066bbf3f155a56064985608a7eddb66787fbfa0ffb
serve#58 file path: cache/26d984569a517733fe19a5066bbf3f155a56064985608a7eddb66787fbfa0ffb.png
serve#58 already in cache
serve#58 size: 0.02 MB
serve#58 done, took 12ms
serve#58 effective speed: 1.44 megabits/s
montyanderson commented 5 years ago

Time to first byte is difficult to work out accurately. I'll just do as soon as the server has sent data (even if not received).

super3 commented 5 years ago

That works

On Wed, Feb 27, 2019, 7:55 AM Monty Anderson notifications@github.com wrote:

Time to first byte is difficult to work out accurately. I'll just do as soon as the server has sent data (even if not received).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ovsoinc/yacdn.org/issues/41#issuecomment-467851389, or mute the thread https://github.com/notifications/unsubscribe-auth/AADuLydYnVRZQ2SZQM40b8AbEOynWZNTks5vRoAygaJpZM4bP38f .

super3 commented 5 years ago

@montyanderson How do I access your implementation of the logging? I just know how to access the PM2 logs right now.

montyanderson commented 5 years ago

@super3 It's just console.loging so it should show up in pm2 logs as normal.

However, it's only implemented for serve routes. Will get it implemented on proxy.

super3 commented 5 years ago

Ah ok that's why it was not showing up.

On Wed, Feb 27, 2019, 6:38 PM Monty Anderson notifications@github.com wrote:

@super3 https://github.com/super3 It's just console.loging so it should show up in pm2 logs as normal.

However, it's only implemented for serve routes. Will get it implemented on proxy.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ovsoinc/yacdn.org/issues/41#issuecomment-468074967, or mute the thread https://github.com/notifications/unsubscribe-auth/AADuL506vdBr49MlE3uCfVcX4X44yt27ks5vRxbggaJpZM4bP38f .

montyanderson commented 5 years ago

Will add logging of the Referer header.

super3 commented 5 years ago 
my-id-is-gangnam-beauty-e03/p480_b1403000_v1266000_a129000_0332.ts?
0|yacdn  | proxy#8333539 referer: undefined
0|yacdn  | serve#8333524 size: 0.98 MB
0|yacdn  | serve#8333524 size: 0.98 MB
0|yacdn  | serve#8333523 size: 2.05 MB
0|yacdn  | serve#8333523 size: 2.05 MB
0|yacdn  | serve#8333519 size: 2.06 MB
0|yacdn  | serve#8333519 size: 2.06 MB
0|yacdn  | serve#8333521 size: 4.71 MB
0|yacdn  | serve#8333521 size: 4.71 MB
0|yacdn  | serve#8333525 size: 1.50 MB
0|yacdn  | serve#8333525 size: 1.50 MB
0|yacdn  | serve#8333522 size: 0.66 MB
0|yacdn  | serve#8333522 size: 0.66 MB
0|yacdn  | proxy#8333540 url: https://odk-hls-seg.s3.amazonaws.com/v1/201808/drama/my-id-is-gangnam-beauty/my-id-is-gangnam-beauty-e05/p1080_b4763000_v4626000_a129000_0000.ts?
0|yacdn  | proxy#8333540 referer: undefined
0|yacdn  | serve#8333520 size: 2.40 MB
0|yacdn  | serve#8333520 size: 2.40 MB
0|yacdn  | serve#8333526 size: 0.37 MB
0|yacdn  | serve#8333526 size: 0.37 MB
0|yacdn  | serve#8333527 size: 0.80 MB
0|yacdn  | serve#8333527 size: 0.80 MB
0|yacdn  | proxy#8333541 url: https://odk-hls-seg.s3.amazonaws.com/v1/201809/drama/my-id-is-gangnam-beauty/my-id-is-gangnam-beauty-e12/p720_b2102000_v1965000_a129000_0010.ts?
0|yacdn  | proxy#8333541 referrer: undefined
0|yacdn  | proxy#8333542 url: https://odk-hls-seg.s3.amazonaws.com/v1/201808/drama/my-id-is-gangnam-beauty/my-id-is-gangnam-beauty-e03/p360_b716000_v579000_a129000_0373.ts?
0|yacdn  | proxy#8333542 referer: undefined
0|yacdn  | serve#8333528 size: 1.07 MB
0|yacdn  | serve#8333528 size: 1.07 MB
0|yacdn  | serve#8333531 size: 0.00 MB
0|yacdn  | serve#8333531 size: 0.00 MB

@montyanderson

With the flow of requests the logs are now unreadable. I think we need to figure out a different way to display them, or use some kind of stats panel.

montyanderson commented 5 years ago 
console.log(`proxy#${n} referer: ${ctx.request.headersreferer}`);

@super3 I'd left out a period.

Where is it spelt wrong? I can't find 'referrer' in the source code.

super3 commented 5 years ago

@montyanderson "The HTTP referer (originally a misspelling of referrer[1]) is an optional HTTP header field that identifies the address of the webpage (i.e. the URI or IRI) that linked to the resource being requested." https://en.wikipedia.org/wiki/HTTP_referer

Ha ha. So its technically correct.

super3 commented 5 years ago

@montyanderson Should there be an undefined referer?

0|yacdn  | Fri, 15 Mar 2019 18:15:54 GMT yacdn:cache lock true
0|yacdn  | Fri, 15 Mar 2019 18:15:54 GMT yacdn:cache cache/819ca90adae2aa499e40c71786ef498a84fb6c26f654608a35c5a0f97a6b1fc7.bin not found
0|yacdn  | Fri, 15 Mar 2019 18:15:55 GMT yacdn:server proxy#11882598 size: 0.00 MB
0|yacdn  | Fri, 15 Mar 2019 18:15:55 GMT yacdn:cache write stream ended
0|yacdn  | Fri, 15 Mar 2019 18:15:56 GMT yacdn:server proxy#11882599 url: https://m.zingmp3.vn/xhr/media/get-source?type=audio&key=LnJGTLGsBxJmghLtnTFHLGykpvZRxxxHl
0|yacdn  | Fri, 15 Mar 2019 18:15:56 GMT yacdn:server proxy#11882599 referer: undefined
0|yacdn  | Fri, 15 Mar 2019 18:15:56 GMT yacdn:cache https://m.zingmp3.vn/xhr/media/get-source?type=audio&key=LnJGTLGsBxJmghLtnTFHLGykpvZRxxxHl not in cache
0|yacdn  | Fri, 15 Mar 2019 18:15:56 GMT yacdn:cache lock true
0|yacdn  | Fri, 15 Mar 2019 18:15:56 GMT yacdn:cache cache/6e502722943ea9d4a1ef5a2d4a5109f9aadd6611879d0644a3f0d775b09a4ae4.bin not found
0|yacdn  | Fri, 15 Mar 2019 18:15:57 GMT yacdn:server proxy#11882599 size: 0.00 MB
0|yacdn  | Fri, 15 Mar 2019 18:15:57 GMT yacdn:cache write stream ended
0|yacdn  | Fri, 15 Mar 2019 18:15:57 GMT yacdn:server proxy#11882600 url: https://m.zingmp3.vn/xhr/media/get-source?type=audio&key=LmJntkHNWagFNbSymTFmkmyLpLZDEQsHE
0|yacdn  | Fri, 15 Mar 2019 18:15:57 GMT yacdn:server proxy#11882600 referer: undefined
0|yacdn  | Fri, 15 Mar 2019 18:15:57 GMT yacdn:cache https://m.zingmp3.vn/xhr/media/get-source?type=audio&key=LmJntkHNWagFNbSymTFmkmyLpLZDEQsHE not in cache
0|yacdn  | Fri, 15 Mar 2019 18:15:57 GMT yacdn:cache lock true
0|yacdn  | Fri, 15 Mar 2019 18:15:57 GMT yacdn:cache cache/a17a48bf4ce66070133d4746b4872b9cf50af6e88114a1752eb90e90ce9bf568.bin not found
0|yacdn  | Fri, 15 Mar 2019 18:15:58 GMT yacdn:server proxy#11882600 size: 0.00 MB
0|yacdn  | Fri, 15 Mar 2019 18:15:58 GMT yacdn:cache write stream ended
0|yacdn  | Fri, 15 Mar 2019 18:15:59 GMT yacdn:server proxy#11882601 url: https://m.zingmp3.vn/xhr/media/get-source?type=audio&key=knxmyzbHiimXtHtFHZHtkWkbmbslxx
0|yacdn  | Fri, 15 Mar 2019 18:15:59 GMT yacdn:server proxy#11882601 referer: undefined
0|yacdn  | Fri, 15 Mar 2019 18:15:59 GMT yacdn:cache https://m.zingmp3.vn/xhr/media/get-source?type=audio&key=knxmyzbHiimXtHtFHZHtkWkbmbslxx not in cache
0|yacdn  | Fri, 15 Mar 2019 18:15:59 GMT yacdn:cache lock true
0|yacdn  | Fri, 15 Mar 2019 18:15:59 GMT yacdn:cache cache/87e397f2963487d8be61e06ef1a6f2e36dab4b63baa64c987ce0c02e5c34bc21.bin not found
montyanderson commented 5 years ago

@super3 undefined referer means that either

super3 commented 5 years ago

@montyanderson Is there another way we can format this information? I've having very difficult time reading these logs now.