storeMediaItems: mediaItems is null

[bgoodr]

I have pulled in your fix to https://github.com/jonagh/gapi-querier/issues/1 into my fork. Ran it on my photos account, and ran into an error inside the storeMediaItems function. Turns out that the passed-in argument of mediaItems is non-null at several points in execution, but at some point, mediaItems becomes null.

I have added some debugging code into my fork to see this in the console. Last commit is:

https://github.com/bgoodr/gapi-querier/commit/826a5583a7f357e5e0309b0332c0ba77da7e2c7e

(don't pull that into your repo, this is just for demo purposes).

The output I see at the end (not expanding the array in the console as it has too much info): console

Could you add some error checking? I don't know if just having that function do nothing (ignore the null value), as I'm not sure about the intent there.

[jonagh]

Hmm. One could add if (!mediaItems) return; at the top of the storeMediaItems function, however I find it funny that it is undefined, as the API docs don't seem to suggest that it should be (https://developers.google.com/photos/library/reference/rest/v1/mediaItems/list).

Do you see any request errors in the network panel? or if you can find the network request which supposedly returns a result without mediaItems? that might clue us in on something.

Maybe this is expected behaviour from the API (and simply not documented?).. though I haven't encountered this issue myself (but I've only tested on two Google Photos accounts).

As mentioned at the top, you could add something to the effect of if(!mediaItems) return;, but I'm not comfortable adding that without understanding why mediaItems is undefined in the first place - it seems like it shouldn't be undefined.

PS. Thanks for putting up with my barely-tested code, and helping to work out these issues.

[bgoodr]

No problem. I'm away from keyboard at the moment, but when I get a chance, I will take a look at what the console network activity shows.

Thanks for your willingness to iterate on this for a bit. It may turn out that there's something boogered up in the input data in my photos account. Who knows yet. Luckily, this problem is consistently reproducible.

[bgoodr]

I looked at the network activity and did not find anything remarkable as an "error":

network console

"or if you can find the network request which supposedly returns a result without mediaItems? that might clue us in on something.": I don't know what do to in order to instrument the code to see that info.

I wonder: Looking at https://developers.google.com/photos/library/reference/rest/v1/mediaItems/list it indeed does not hint as to when the result value would be null. But it does not specifically state that it will never be null, does it? I mean, as I interpret your code, you are having to keep reading the results in "batches", and stopping when it is complete. So, maybe there is some bug in their system that allows "pageToken" to be non null but the result set is null, or something.

Could it be that the number of items per "batch" is evenly divisible into the total number of items in this particular google photo account, resulting in the null result on the very last one? I'm going to test that out by taking a photo with my phone (I have my Android phone automatically backup photos I take so it should be "an item" there), and rerun the page. If it doesn't give the null at the end, then that theory holds weight, else, it is a bogus theory.

[bgoodr]

That theory mentioned above is bogus. Just adding the new photo did not change the error.

[bgoodr]

Just for thrills, I'm going to add the if (!mediaItems) return; to see what the effect is.

[bgoodr]

Results of the experiment of the addition of if (!mediaItems) return; in a37157f: Along with that, I also added a new debug output statement so that we could see when it is null versus when it is non-null. I ran it and studied the console log while it was running and found that the null condition happens sporadically throughout the lifetime of the run. It did proceed though, and then failed in a similar manner in a different spot: See my annotated screenshot. The top part is the console log where the green arrows are for valid data, and the red arrows are for the null condition. In the lower part of the diagram we can see the place where it fails again in forgetMediaItems for similar reasons. I'll rerun it with the same pseudo-fix/hack done on forgetMediaItems and report back.

[bgoodr]

After that additional change in 86cbfa1, it ran all the way through:

annotated screenshots of console, result page, and first photo

Clicking on the first link listed in the page directs me to the photo I just took and "shared" into an album (see the bottom part of the above). So I'm not sure if things are working correctly. But the second link is not a part of an album.

Alright. That is a lot of experimentation info to absorb! :)

[jonagh]

K, I've done some additional testing and it does seem like mediaItems (and albums) can be undefined in results sometimes - like if you have an album without photos.

I've update the code to check for undefined result sets now. 8f91dd0d4fe474c560d6d61a68fe6d7706f41717

Although I don't understand why it would be returning empty results during the storeMediaItems process (https://imgur.com/c3FWB93), as it is just listing all your photos (not albums) - so I'm not sure what is going on there, perhaps that is just behaviour that Google Photos can exhibit - not sure?!

Also I suspect the reason that it outputs photos that ARE in shared-albums is simply because I didn't bother to check shared-albums (which is a different API call). For my own purposes I wanted to ignore shared albums.. but obviously it would be useful to have the choice. So I've added a second option to include shared-albums in the routine (you'll see it in the dropdown).

Hopefully things work now?

[bgoodr]

That worked like a charm! I used the Find out-of-album photos (including "shared" albums) and waited until completion, and the original link, that showed using the other option, is no longer there.

Thank you!! I'll have to give this some thought as to how I would use it in practice. But the fact that the links directly into the photos page gets me a long way to reorganizing my photos into new albums.

[jonagh]

Excellent. Glad it is of some use :) And thanks for working through the issues with me.

In my case I just went through each photo individually and manually added it to the appropriate album. I needed to do it manually anyways since I had to look at each photo to know where it should be put. Though I only had maybe around 50 photos that were not in an album already - if you have hundreds that could be a bother.

I did consider adding the feature to add out-of-album photos into a new album, though then I'd have to ask for write-permissions - and for my uses it wasn't necessary.