User does not have READ permission for FILE

[drboone]

Using 2791b3516860603ab093e3080ec16dda7dd134ec I'm getting the above error for Files and Photos in at least two groups to which I'm subscribed, and in which I can see both types of content in the web interface. Messages seem to download just fine. Log snippet for such an occurrence:

2019-10-27 16:18:17.984 EDT DEBUG urllib3.connectionpool Starting new HTTPS connection (1): groups.yahoo.com
2019-10-27 16:18:18.194 EDT DEBUG urllib3.connectionpool https://groups.yahoo.com:443 "GET /api/v2/groups/sparetimegizmos/files HTTP/1.1" 307 None
2019-10-27 16:18:18.197 EDT DEBUG YahooGroupsAPI Exception raised on uri: https://groups.yahoo.com/api/v2/groups/sparetimegizmos/files
2019-10-27 16:18:18.197 EDT DEBUG YahooGroupsAPI b'{"ygPerms":{"resourceCapabilityList":[{"resourceType":"GROUP","capabilities":[{"name":"READ"},{"name":"JOIN"}]},{"resourceType":"PHOTO","capabilities":[]},{"resourceType":"FILE","capabilities":[]},{"resourceType":"MEMBER","capabilities":[]},{"resourceType":"LINK","capabilities":[]},{"resourceType":"CALENDAR","capabilities":[]},{"resourceType":"DATABASE","capabilities":[]},{"resourceType":"POLL","capabilities":[]},{"resourceType":"MESSAGE","capabilities":[{"name":"READ"}]},{"resourceType":"PENDING_MESSAGE","capabilities":[]},{"resourceType":"ATTACHMENTS","capabilities":[]},{"resourceType":"PHOTOMATIC_ALBUMS","capabilities":[]},{"resourceType":"MEMBERSHIP_TYPE","capabilities":[]},{"resourceType":"POST","capabilities":[{"name":"READ"}]},{"resourceType":"PIN","capabilities":[]}],"groupUrl":"groups.yahoo.com","intlCode":"us"},"ygError":{"hostname":"gapi4.grp.bf1.yahoo.com","httpStatus":307,"errorMessage":"User does not have READ permission for FILE. Messa...","errorCode":1101,"sid":"SID:YHOO:groups.yahoo.com:9cde1ab15245cd0b2529a8749a2a3591:0"}}'
2019-10-27 16:18:18.197 EDT ERROR archive_files Couldn't access Files functionality for this group
2019-10-27 16:18:20.665 EDT DEBUG urllib3.connectionpool Starting new HTTPS connection (1): groups.yahoo.com
2019-10-27 16:18:20.734 EDT DEBUG urllib3.connectionpool https://groups.yahoo.com:443 "GET /api/v2/groups/sparetimegizmos/albums?count=5 HTTP/1.1" 307 None
2019-10-27 16:18:20.736 EDT DEBUG YahooGroupsAPI Exception raised on uri: https://groups.yahoo.com/api/v2/groups/sparetimegizmos/albums?count=5
2019-10-27 16:18:20.736 EDT DEBUG YahooGroupsAPI b'{"ygError":{"hostname":"gapi3.grp.bf1.yahoo.com","httpStatus":307,"errorMessage":"User does not have READ permission for GROUP. Mess...","errorCode":1101,"sid":"SID:YHOO:groups.yahoo.com:289f1bdf0c06b67ed6852ae95e4c7a55:0"}}'
2019-10-27 16:18:20.736 EDT ERROR archive_photos Couldn't access Photos functionality for this group

The browser session where I'm viewing the files and images is the one from which I pulled the cookies that the script is using.

[drboone]

There's one group (Spotmatic) where I can't even get messages via the script, but can read them in the web interface.

[IgnoredAmbience]

Interesting, I don't think I've seen a 307 response before. Seems related to #2, which was caused by a lack of auth cookies. Permissions json in the above is also showing no read permissions for files or photo access, and offering the join permission to the group.

Are you sure the cookies are correctly supplied, and the given account is a member of the group?

[d235j]

See https://github.com/IgnoredAmbience/yahoo-group-archiver/pull/79.

Should we record 307 responses?

[drboone]

I literally have the same browser session from which I pulled the cookies, still running, and it still has access to the files and images and such. I thought the json looked like it didn't match the web experience. I've checked the copypasta of the cookies a couple of times. Lemme try logging out and back in and refreshing the cookies.

[d235j]

@drboone can you paste the 'about' json of this group (Spotmatic) somewhere?

[drboone]

Logging out and back in hasn't changed the behavior.

about.txt

[drboone]

Oh, re Spotmatic, there's a traceback I should have shared:

Traceback (most recent call last):
  File "/usr/local/src/gittree/yahoo-group-archiver/yahoo.py", line 126, in archive_email
    yga.messages()
  File "/usr/local/src/gittree/yahoo-group-archiver/yahoogroupsapi.py", line 110, in get_json
    raise e
  File "/usr/local/src/gittree/yahoo-group-archiver/yahoogroupsapi.py", line 105, in get_json
    raise requests.exceptions.HTTPError(response=r)
requests.exceptions.HTTPError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/src/gittree/yahoo-group-archiver/yahoo.py", line 693, in <module>
    archive_email(yga)
  File "/usr/local/src/gittree/yahoo-group-archiver/yahoo.py", line 128, in archive_email
    logger.error("User doesn't have permission to access Messages in this group", err.message)
AttributeError: 'HTTPError' object has no attribute 'message'

[d235j]

@drboone are you using python2 or 3?

I just joined the group and I'm running the script (1ee4d1f2b088da06e71d6048923a139266c5bfe1) with the following command line: python3 yahoo.py -ct 'z=...' -cy 'v=...' -w 'sparetimegizmos' and everything is archiving properly: messages and files, waiting for the rest.

Do your cookies look like this? T: z=someLongText&a=someLongText&sk=someLongText&ks=someLongText&kt=someLongText&ku=someLongText&d=someLongText&af=someLongText Y: v=1&n=someLongText&l=someShortText=someShortText&r=hp&intl=us

[drboone]

Python 3.6.9.

My T cookies is just a long b64? string.

[d235j]

@drboone Valid cookies would be in the format I specified above. The someLongText and someShortText may be b64, but there should be specific sections like that.

How are you exporting your cookies?

[drboone]

Aha, something went wrong with the copypasta then. Trying again.

[drboone]

Theeeere we go. Now pulling files. Thanks!

[d235j]

Resolved — cookie copy/paste issue.