Let's get this project started!

elderamevans commented 9 years ago

OK, I realize this may not be the best channel for all, but it's currently the best one for me...

I've got the repo. I've got a good number of GRAMPS devs I can talk to. I've (somewhat) got the coding skills. I've got a bit of time. I've got a fully-functional FamilySearch-Python SDK. It sounds like we're going to get the get-go approval, even if it's only read-only access at first.

What I lack are a solid gameplan and a grasp of PyGObject (or whatever GRAMPS uses for its GUI).

I can throw some ideas as edited screenshots in here.

I've currently slapped the BSD license on the repo, but I'm more than willing to change it to GPL.

(As a side note: The reason I chose BSD for the SDK is to allow it to be compatible with the GPL, yet allow other developers to, if there's some agreement between FamilySearch/boss/school/whatever that requires the app that integrates with FamilySearch.org to be closed-source, it's available. A bit of a compromise, I know, but I felt it necessary.)

I don't know if I'll be able to contribute code any time soon, but expect some screenshots soon!

elderamevans commented 9 years ago

I forgot who originally suggested it, but I'm willing to set this project up so that FamilySearch.org isn't the only Web site that allows integration.

MyHeritage has a free API available.

Ancestry.com has an API, but on quick discovery, not a public one.

I can't say about any other "tree" sites...

elderamevans commented 9 years ago

Sample sign-in page (we don't necessarily need a "FamilySearch Central" page...

signin

"Comparison" tabs (in two of the places it's most likely going to be [people and relationships], along with potentially others)

comparetabs

elderamevans commented 9 years ago

(cc: @dsblank @sam-m888)

dsblank commented 9 years ago

Thanks for the note! BTW, Gramps 5.0 (github gramps-project master, as of a few minutes ago) will have a pluggable database backend. I'm not sure how that could work with FS, but it is now an option.

elderamevans commented 9 years ago

Sounds good! I'll see what we can do for development!

The good news is, FamilySearch now has an automated Sandbox registration process that gives us a sandbox key, username, and password (that we won't need to share).

I noticed talk about the DB switch. It seems the only issue with BSDDB3 on Windows is I can't find a quick, straightforward way to add it to Windows. I may be missing something, but that's another discussion for another time...

To be honest, though, I think the "Compare Online" approach might be the best, so that we're able to sync several databases simultaneously.

As for the integration with the new DB? The SDK returns (I think very) human-readable JSON. All we need to do is make the plugin have the two talk to each other, and then work on the GUI to control the backend.

I suppose this is now a good time to see how good the documentation is....

elderamevans commented 9 years ago

The good news is, it seems the JSON structure has been pretty much the same since the release of even nFS's JSON.

dsblank commented 9 years ago

@elderamevans Does each user of this code need their own key/username/password? Or is there some way of using one set for all development?

Regarding BSDDB on windows... you can do one of a few things:

Use one of the All In One (AIO) gramps installers, to just get the dependencies.
Install the windows BSDDB part individually
Wait a day or so, and you'll be able to use Gramps master without BSDDB.

Now, how does one go about exploring/testing what you have?

dsblank commented 9 years ago

BTW, Gramps data also has a method to go to and from a JSON-like format. (It is called "struct" and every object has a to_struct() and from_struct method)

elderamevans commented 9 years ago

@dsblank Not particularly, IIRC. I'll ask the FamilySearch Partner team about shared keys/UN/PW.

I've seen several repos with their sandbox keys public (as the requirement to obfuscate the key is obsolete), but without a developer sanbox UN/PW, it's useless to Average Joe.

As for poking around the SDK? For the most part, I think http://elderamevans.github.io/familysearch-python-sdk-opensource/ should explain it well enough. (I do apologize if it's incomplete; feel free to create a PR/issue if there's any confusion.)

My main intention was to couple the SDK documentation with the official FamilySearch documentation (no need to duplicate/manually sync official, external documentation!)

Nonetheless, I think the best thing to do to learn the layout of the responses is to poke the response sections with type(foo) and foo.keys(), and to read the applicable FamilySearch documenation.

Usually, though, the format is something along the lines of:

{"headers":
    {...},
"response":
    {
      "persons" : [{...}],
      "sourceDescriptions" : [{...}],
      "places" : [{...}],
    }
}

dsblank commented 9 years ago

Any idea what could cause this error?

Here is the code:

from familysearch import FamilySearch
from getpass import getpass
fs_address = "https://sandbox.familysearch.org"
user_agent = "MyGramps/1.0"
app_key = "MY_APP_KEY_WHICH_I_CAN_PROBABLY_SHARE"
fs = FamilySearch(user_agent, app_key, base=fs_address)
fs.login(input("Username: "), getpass())

and the error:

Traceback (most recent call last):
  File "fslogin.py", line 10, in <module>
    fs.login(input("Username: "), getpass())
  File "/usr/local/lib/python3.4/dist-packages/familysearch/authentication.py", line 46, in login
    response = self.post(url, credentials, headers)
  File "/usr/local/lib/python3.4/dist-packages/familysearch/__init__.py", line 235, in post
    url, data, headers, "POST", nojson), nojson)
  File "/usr/local/lib/python3.4/dist-packages/familysearch/__init__.py", line 177, in _request
    return self.opener.open(request)
  File "/usr/lib/python3.4/urllib/request.py", line 461, in open
    response = meth(req, response)
  File "/usr/lib/python3.4/urllib/request.py", line 571, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.4/urllib/request.py", line 499, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.4/urllib/request.py", line 433, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.4/urllib/request.py", line 579, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 400: Bad Request

elderamevans commented 9 years ago

Oh, yeah... I forgot to mention that you'll need to send in a request (probably to devsupport at familysearch dot org) to enable desktop authentication for the key.

dsblank commented 9 years ago

Ok, thanks. I sent them a note.

elderamevans commented 9 years ago

Sounds good!

Guess this is a note to me to develop a way to see the error for 400s (as an optional argument)...

sam-m888 commented 9 years ago

@dsblank Thanks for the heads-up, I've joined. Should I also "Generate an App Key to start connecting with our API in seconds." for Gramps or is it for personal use, and then email devsupport to enable desktop auth for the key?

@elderamevans Excellent work, I look forward to exploring this.

dsblank commented 9 years ago

@sam-m888 I don't think it would hurt to go through the process to get a key/id/password. I suspect that in the end, we'll just need one key for "the project" and users can enter their own id/password when they login through Gramps.

elderamevans commented 9 years ago

Thanks, both of you!

Still waiting for confirmation on sharing a key. It sounds like the UN/PW are not tied to the key (especially with being able to sign in through the webclient through sandbox.familysearch), but I'm still checking on that, too.

dsblank commented 9 years ago

I finally heard from someone from support@devsupport.desk-mail.com ... not working yet though...

elderamevans commented 9 years ago

OK. Sounds like keys can be shared. Let me create a gist...

elderamevans commented 9 years ago

@dsblank Familysearch-Python SDK desktop login diagnoser

elderamevans commented 9 years ago

@dsblank Turns out, keys can be shared; we just need one key per app.

Now, to get the issues with a sandbox account sorted out....

dsblank commented 9 years ago

I can't get anything to work via the "desktop" mode. Support says that they applied that mode to my account, but I can't login.
If we can share our keys, then I just need to pair your key with your app name, and use my sandbox id/password, right? Can you share your key?

dsblank commented 9 years ago

The error I get from the response is:

{u'error_description': u'Flow is undefined', u'error': u'invalid_request'}

dsblank commented 9 years ago

Found a reference to the error: "You have to request permission to use the password flow." Now, I wait some more...

elderamevans commented 9 years ago

@dsblank From what I understand, keys are not (completely) tied to the UN/PW, meaning you can sign in with Joe's key and your UN/PW.

That way, each won't have to get their own key sent in to enable desktop authentication. And, we're keeping it to one key, one app.

dsblank commented 9 years ago

@elderamevans So as long as you have a paired key/appname one can do whatever the app is allowed to do? I still can't do anything, as I have the "Flow" error. Can you share your key? How will this work?

elderamevans commented 9 years ago

@dsblank Still working on getting a key myself.

I did talk to the partner team, and it they said it's usually a quick turn-around to enable desktop login for the key. Perhaps sending them another email?

dsblank commented 9 years ago

@elderamevans How are you testing if you don't have a key?

I have sent them a few emails :smile:

elderamevans commented 9 years ago

I have my ways...

Well, I was borrowing a key from somebody, which I have long-since lost, and I've found a few... alternative routes... of getting authenticated.... which some, if not most, have been disabled...

The main thing is, the guys have been diagnosing a bug where select users weren't getting the password recovery emails (which I was one of them) for the Developer Site accounts, and they're also diagnosing why some email addresses are getting false flags of already having a developer site account.

dsblank commented 9 years ago

Ah, ok. That explains it. You have more patience than I have! Until one has a key that allows desktop login, there isn't much to do.

dangar46 commented 9 years ago

Was wondering the same? I have an API user name and PW already. Can I just link it to this project? From: Doug Blank notifications@github.com To: elderamevans/familysearch-gramps familysearch-gramps@noreply.github.com Sent: Thursday, June 4, 2015 3:31 PM Subject: Re: [familysearch-gramps] Let's get this project started! (#1)

@elderamevans So as long as you have a paired key/appname one can do whatever the app is allowed to do? I still can't do anything, as I have the "Flow" error. Can you share your key? How will this work?— Reply to this email directly or view it on GitHub.

elderamevans commented 9 years ago

@dangar46 It sounds like you should be able to sign in with your developer UN and PW on a different key than what you have.

The big thing is, you can't sign into sandbox with a production account, and vice versa.

dsblank commented 9 years ago

Ok, I encountered a bug in the familysearch-python-sdk-opensource... made a PR there. Not sure how you were able to login with that as it was.

Now, I guess I'll try to find some examples of working with the FS site.

elderamevans commented 9 years ago

@dsblank Not sure, either. Ah well, must've made a code change that caused it to work once, that broke it...

Still, sounds like you should be able to sign in now...

elderamevans / familysearch-gramps

Let's get this project started! #1