search

makingglitches / GooglePhotoDownload

Connects to Google Photos and downloads all content, keeping track of original data on disk and moving files only on the computer and already on server to respective directories, and tries to download the entire collection and store size info for quicker startup as well as original file sizes of those on disk, downloading the files still on disk first so they can be freed up. Supports multiple user accounts. Its just a better mousetrap. Google Takeout prepares whole archives of photos, this allows you to download them separately and keep track of some statistics as well on space usage.
6 stars 4 forks source link

Add code to speed up updatesize #35

Closed makingglitches closed 3 years ago

makingglitches commented 3 years ago

It takes forever to process all the HEAD requests to update the sizeonserver field of the database.

Likely the http request to grab the HEAD is slower than the GET to pull the actual media items url down.

This all being pretty amazing as this is google we're talking about and i'm not throttling these requests in this case.

Likely adding a secondary processing queue with a thread would be a nice direction to go to ensure not too many are running at once, which WILL result in google temp-throttling a free account.

makingglitches commented 3 years ago

Simply put this should not be happening on a connection that hits a 1mb/s speed on multiple files once they start downloading.

makingglitches commented 3 years ago

seriously save the lies about how its somehow beneficial to anything but these pieces of garbage to keep stealing photo sand the liek from this time period where it would be just as easy for you people to stop fucking up this time period.

makingglitches commented 3 years ago

anyway before i go back off to ML ville

it would be possible to mark each item in different levels of readiness for download and simply add another thread that processed items that were lacking to run alongside the one that starts the actual downloads since size updates are fucking slow as balls.

makingglitches commented 3 years ago

Added the same working fucking code Added another timer Stop giving my code to chomos or outright deleting it.