search

internetarchive / internet-archive-voice-apps

Voice Apps (Actions on Google, Alexa Skill) of Internet Archive. Just say: "Ok Google, Ask Internet Archive to Play Jazz" or "Alexa, Ask Internet Internet Archive to play Instrumental Music"
46 stars 42 forks source link

use AI HTTP endpoints instead of HTTPS because of better performance #405

Open hyzhak opened 5 years ago

hyzhak commented 5 years ago
amarjeet045 commented 5 years ago

Hello; I am Amarjeet Singh, I am an undergraduate student studying computer science in Jaypee University of Engineering & technology, I have worked on these languages Javascript, Python, HTML and CSS, I am very much interested in contributing for open source .so i wanted to contribute in this issue,

hyzhak commented 5 years ago

@amarjeet045 sure you can try

yagyanshbhatia commented 5 years ago

hey @hyzhak I'm starting out with this issue. can you breif me a bit about these endpoints in our project?? I understand https can be slow for short requests due to the initial handshake. But I've read theory only, can you point me in a direction to proceed?

yagyanshbhatia commented 5 years ago

hey @hyzhak quick update.. I'm using httpstat to measure the difference in the response time of endpoints

for example I took this endpoint - https://s3.amazonaws.com/gratefulerrorlogs/CrowdNoise.mp3 https gives

  DNS Lookup   TCP Connection   SSL Handshake   Server Processing   Content Transfer
[    470ms   |      611ms     |     743ms     |       487ms       |       413ms      ]
             |                |               |                   |                  |
    namelookup:470ms          |               |                   |                  |
                        connect:1081ms        |                   |                  |
                                    pretransfer:1824ms            |                  |
                                                      starttransfer:2311ms           |
                                                                                 total:2724ms

while http gives

 DNS Lookup   TCP Connection   Server Processing   Content Transfer
[    273ms   |      406ms     |       410ms       |       438ms      ]
             |                |                   |                  |
    namelookup:273ms          |                   |                  |
                        connect:679ms             |                  |
                                      starttransfer:1089ms           |
                                                                 total:1527ms

Is this considered a significant imrovement?? WHAT is considered as significant improvement??

hyzhak commented 5 years ago

@yagyanshbhatia thank you! Could you make comparisons for IA endpoints?

yagyanshbhatia commented 5 years ago

@hyzhak I'll make comparisons for a couple of endpoints here.. If you want comparison for all of them, let me know.

  1. endpoint - https://archive.org/download/moby_dick_librivox/Moby_Dick_1002.jpg with HTTPS 

    DNS Lookup   TCP Connection   SSL Handshake   Server Processing   Content Transfer
[     5ms    |      377ms     |    1473ms     |       384ms       |        0ms       ]
         |                |               |                   |                  |
namelookup:5ms            |               |                   |                  |
                    connect:382ms         |                   |                  |
                                pretransfer:1855ms            |                  |
                                                  starttransfer:2239ms           |
                                                                             total:2239ms

    with HTTP , 

    DNS Lookup   TCP Connection   Server Processing   Content Transfer
[     1ms    |      369ms     |       653ms       |        1ms       ]
         |                |                   |                  |
namelookup:1ms            |                   |                  |
                    connect:370ms             |                  |
                                  starttransfer:1023ms           |
                                                             total:1024ms

  2. endpoint - https://archive.org/download/jfjo2009-10-02.superlux.flac16/jfjo2009-10-02d1t02.mp3 with HTTPS

    DNS Lookup   TCP Connection   SSL Handshake   Server Processing   Content Transfer
[     0ms    |      394ms     |    1239ms     |       727ms       |        0ms       ]
         |                |               |                   |                  |
namelookup:0ms            |               |                   |                  |
                    connect:394ms         |                   |                  |
                                pretransfer:1633ms            |                  |
                                                  starttransfer:2360ms           |
                                                                             total:2360ms

    with HTTP

    DNS Lookup   TCP Connection   Server Processing   Content Transfer
[     0ms    |      462ms     |       723ms       |        0ms       ]
         |                |                   |                  |
namelookup:0ms            |                   |                  |
                    connect:462ms             |                  |
                                  starttransfer:1185ms           |
                                                             total:1185ms
hyzhak commented 5 years ago

@yagyanshbhatia Thank you! And sorry for being maybe not so clear in requirements. I'm interesting to test other IA endpoints, which returns json data, for example those: