search

shmoe / crunchycrawly

Python command line utility that crawls through Firefox bookmarks and Crunchyroll and returns a list of bookmarked series that have new episodes or seasons
GNU General Public License v3.0
0 stars 0 forks source link

No caching of unchanged web pages #1

Open shmoe opened 4 years ago

shmoe commented 4 years ago

Every execution the web page is pulled (twice!) in crunchyroll_lib.getShowPage(...) even if the page has not changed since last execution

shmoe commented 4 years ago

may not be possible with the headers currently returned by a HEAD request to Crunchyroll

shmoe commented 4 years ago

implemented as much as can be expect: getShowPage is only called once per url

more caching will require Crunchyroll to support Content-Length or Last-Modified headers