search

csaftoiu / yahoo-groups-backup

A python script to backup the contents of private Yahoo! groups.
The Unlicense
37 stars 18 forks source link

scrape_files falsely reports success #50

Open logological opened 4 years ago

logological commented 4 years ago

I'm running yahoo-groups-backup on an openSUSE Tumbleweed system. The default Firefox driver doesn't work for me (see Issue #49) so I am trying to use Chromium with the chrome driver. When running yahoo-groups-backup.py scrape_files this way, a Chromium window pops up and closes a few seconds later. Then the Python script quits. In the terminal I see the following messages:

Detecting the log-in page...
Detecting the log-in page...
Done processing all files!

There is no way that the script could have possibly downloaded all the files from the group in those few seconds, so there must be something wrong here. This problem happens whether or not I pass my Yahoo! credentials on the command-line using --login and --password.

hrenfroe commented 4 years ago

Fellow Tumbleweed user here. Do you have chromedriver installed separately or are you using the package manager version? I was tripped up by an old binary I had hanging around. The packaged version is at /usr/lib64/chromium/chromedriver, which is not in my path, so I symlinked it to a location that was in my path. At this time, with my fork, I have it backing up a private group.

logological commented 4 years ago

I installed chromedriver and chromedriver-bin using pip in a virtualenv. I forced installation of the old version of chromedriver-bin that matched the version of Chromium that I am using (0.77, IIRC). (If you try to use the latest version of chromedriver-bin with Chromium < 0.78, it will complain that the browser is out of date.)

By the way, I was wrong when I said I was using Tumbleweed—I'm actually using Leap 15.1. (The official Tumbleweed package for MongoDB is not building at the moment and the old version in the repository can't be installed due to a missing dependency.)

hrenfroe commented 4 years ago

Sounds about right. I'm also on Chromedriver/Chromium 77 so your setup is probably fine there.

I wonder if that's the login page not being detected properly. It's been a year since I fixed it, so I don't recall how it manifested. Have you tried my fork at https://github.com/hrenfroe/yahoo-groups-backup ? I've put in the login page fix, and changed the README, but haven't modified anything else.