colonelpanic8 / okcupyd

A Library that enables programmatic interaction with okcupid.com, using okcupid.com's private okcupid JSON API and html scraping when necessary.
MIT License
110 stars 18 forks source link

various profile.py fixes for new okcupid.com ui #81

Closed kfred closed 8 years ago

kfred commented 8 years ago

fixes for profile.id, profile.location, profile.age, profile.contacted, profile.liked, profile.match_percentage, profile.enemy_percentage.

Findings: logged-in user's profile still uses the old UI layout; therefore existing profile.location, profile.age will work when calling those functions on your own profile (as well as when search retrieves the logged-in user's location) non logged-in users' profiles uses the new UI layout, which requires updated html references

Implementation: *for profile.age and profile.location, first try the new UI selectors, and upon exception, attempt the old UI selectors, under the assumption that the logged-in user's profile is being called

Update (1/30/16):

kfred commented 8 years ago

Re: the failed checks, the tests appear to be running checks against old html-layout data stored in the vcr_cassettes yaml files.

I am not sure how to update those yaml cassette files with new data, but am happy to if you wouldn't mind offering some guidance.

If these tests are updated to retrieve data on active users, with current website data, those IndexError's and AssertionError's on the profile tests should go away.

kfred commented 8 years ago

Made two more updates--one to update parse_date_updated helper function to properly handle okcupid html dates prior to current year, and another to let profile.contacted return neatly formatted date structures using parse_date_updated helper function.

colonelpanic8 commented 8 years ago

Thanks! This looks great. There are invoke tasks to update the cassettes recorded for tests in tasks.py .

There is code that handles scrubbing login information already in place, but you will need to fill out a credentials file to make it work so that it can actually log in to okc

kfred commented 8 years ago

Alrighty, so I rerecorded the cassettes for those tests. As you can imagine, many of them now fail due to the UI changes. I am going to review further the results of tests associated with my recent commits tonight, make any needed changes, and push those cassettes to my fork.

I'm not sure the scrubbing process worked completely. I grep'ed the directory and found matches for my test username in pycache/test_credentials.cpython-27-PYTEST.pyc.

Are there any other places I should check that my test credentials have been scrubbed before pushing?

colonelpanic8 commented 8 years ago

I'm not sure the scrubbing process worked completely. I grep'ed the directory and found matches for my test username in pycache/test_credentials.cpython-27-PYTEST.pyc.

the pyc is a compiled python file. The scrubbing is just for the test cassette. I don't think the credentials or any pyc files will get checked in so everything should be all good. you should do a git grep, or ignore gitignored files

kfred commented 8 years ago

Ah, gotcha--that makes sense. Cool, I'll check out those arguments.

kfred commented 8 years ago

Just pushed two new commits, including stylistic improvements, better xpb precision and more elegant implementations of some of the functions I edited in my previous commits.

I was trying to keep profile.py and helpers.py updates in separate commits, but accidentally let an old helpers.py edit into https://github.com/kfred/okcupyd/commit/da49f95d1fdeddd57f96d82ced2050cd32c29bcb

Also, still reviewing the new cassettes' test results. I force-exited the first rerecord I did, and think I might re-rerecord and review before adding a commit for them.

kingyakshaver commented 8 years ago

@kfred hey sorry I never got a chance to look at this. Whats the status of everything here?

colonelpanic8 commented 8 years ago

I've merged this into master

kfred commented 8 years ago

@IvanMalison @kingyakshaver hey, I just saw this. I haven't used this in a while because okcupid suspended my account (oddly during a period of complete inactivity) :( and I haven't yet gotten back into the swing of things. But everything was working last I checked.

Good to see movement again on this project. Would love to see the essays working again; I previously started looking into them but I'll revisit them soon.