Ongoing Discussion

[psigourney]

This tutorial seemed pretty applicable to our project:

https://realpython.com/blog/python/python-web-applications/

[psigourney]

I think the first question would be whether any of us have previous experience with web development (I have none).

Is there a preferred framework to use? The AppEngine tutorial used WebApp2, but I've heard Django is popular also?

[ericaddison]

No web experience here :)

I don't know that there is a preferred option, but I think WebApp2 is straight forward enough that I would vote to keep using that one, for now at least. I did see that AppEngine is compatible with Django, which I've also heard is popular, and I'd like to learn about it...

What would you think about using WebApp2 for the mini-project, and considering Django for the final project? That way we could charge right ahead with the mini-project, but get some familiarity with Django as well, which I understand is maybe a more powerful framework?

[ericaddison]

I was having a hard time finding a way to get started writing a service for AppEngine... From various sources I was able to piece together a working example. Check it out here. I think from this basic example we'll be able to build up to the services we need to implement.

The next appengine-ey thing to do will be to get ourselves straight on the cloud storage I think ... both the standard cloud storage for images and the Datastore and/or Cloud SQL for database.

[psigourney]

I'm trying to get the Google sign-in to work. I seem to have it working, but I'm not sure how to pass/access the user information.

The default code stored in the .html page has 'profile' and 'id_token', but I'm not sure how to access these values.

<script>
  function onSignIn(googleUser) {
    // Useful data for your client-side scripts:
    var profile = googleUser.getBasicProfile();
    console.log("ID: " + profile.getId()); // Don't send this directly

to your server! console.log('Full Name: ' + profile.getName()); console.log('Given Name: ' + profile.getGivenName()); console.log('Family Name: ' + profile.getFamilyName()); console.log("Image URL: " + profile.getImageUrl()); console.log("Email: " + profile.getEmail());

    // The ID token you need to pass to your backend:
    var id_token = googleUser.getAuthResponse().id_token;
    console.log("ID Token: " + id_token)
  };
</script>

When I add a code snippet to the HTML to see if a user has signed in (even when the Google sign in button says they are), I get nothing:

{% if profile %}
    {% print profile.getName() %}
{% endif %}

On Sat, Sep 23, 2017 at 10:57 PM, Eric Addison notifications@github.com wrote:

I was having a hard time finding a way to get started writing a service for AppEngine... From various sources I was able to piece together a working example. Check it out here https://github.com/ericaddison/appengine_samples/tree/master/services. I think from this basic example we'll be able to build up to the services we need to implement.

The next appengine-ey thing to do will be to get ourselves straight on the cloud storage I think ... both the standard cloud storage for images and the Datastore and/or Cloud SQL for database.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ericaddison/APT_miniProject/issues/4#issuecomment-331685532, or mute the thread https://github.com/notifications/unsubscribe-auth/ACI-NfFB_RPFTyOfiJPp9afk3brilNdzks5sldNEgaJpZM4PZ4RZ .

[ericaddison]

I'm wondering about how we accomplish "passing the ID token to the backend"... I think that's how we'll get python access to user info. In your html snippet I think that is trying to access the python environment, right? And the profile variable is on the client side. ..

[psigourney]

aha! You use an HTTP POST to send the value.

https://developers.google.com/identity/sign-in/web/backend-auth

[ericaddison]

Sweet!

[ericaddison]

OK, I've been playing with some login stuff... Looks like there are two ways to login in using a google account, either with the users API like we did in the previous tutorial, or with this token business. I could not get the back-end token validation to work with the Python google-auth library, but I was able to make it work by calling the token validation endpoint from that link @psigourney posted...

I also started playing with Facebook login, but didn't get very far :/

Here is the silly page: https://ea-ut-apt-login.appspot.com/

And here is the github with the code: https://github.com/ericaddison/appengine_samples/tree/master/login

[psigourney]

I was thinking about it last night, and since we don't need to make it secure, we could do a hybrid of Google and Users.... Use the Google auth on the front page, then just POST the name & email address back to the server (rather than the whole token). In the get(self) function, have the email checked against the Users database/library/whatever, and if the user doesn't exist, create them. We'll need the internal Users anyway, for stream (photo album!) ownership. But this way we won't have to deal with passwords.

Let google do the user auth, then use the email addy from the google token as if it were passed by POST from an input box.

On Mon, Sep 25, 2017 at 12:13 AM, Eric Addison notifications@github.com wrote:

OK, I've been playing with some login stuff... Looks like there are two ways to login in using a google account, either with the users API like we did in the previous tutorial, or with this token business. I could not get the back-end token validation to work with the Python google-auth library, but I was able to make it work by calling the token validation endpoint from that like @psigourney https://github.com/psigourney posted...

I also started playing with Facebook login, but didn't get very far :/

Here is the silly page: https://ea-ut-apt-login.appspot.com/

And here is the github with the code: https://github.com/ericaddison/appengine_samples/tree/master/login

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ericaddison/APT_miniProject/issues/4#issuecomment-331778867, or mute the thread https://github.com/notifications/unsubscribe-auth/ACI-NdJHly6XKJLaY6YySoCQSF8WdYwAks5slzZcgaJpZM4PZ4RZ .

[psigourney]

follow up.... It would be horribly insecure in that anyone could manually POST any email address and the app would log them in as that user without any checks, but for a class project, I don't think that'd be a problem.

On Mon, Sep 25, 2017 at 12:13 AM, Eric Addison notifications@github.com wrote:

OK, I've been playing with some login stuff... Looks like there are two ways to login in using a google account, either with the users API like we did in the previous tutorial, or with this token business. I could not get the back-end token validation to work with the Python google-auth library, but I was able to make it work by calling the token validation endpoint from that like @psigourney https://github.com/psigourney posted...

I also started playing with Facebook login, but didn't get very far :/

Here is the silly page: https://ea-ut-apt-login.appspot.com/

And here is the github with the code: https://github.com/ericaddison/appengine_samples/tree/master/login

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ericaddison/APT_miniProject/issues/4#issuecomment-331778867, or mute the thread https://github.com/notifications/unsubscribe-auth/ACI-NdJHly6XKJLaY6YySoCQSF8WdYwAks5slzZcgaJpZM4PZ4RZ .

[ericaddison]

I think the token part works well enough ... the question I have now is how best to stash the user info? That is, we can send the token to the backend, do the validation to get all info like name and email, but then how do we stash that so we can keep using it? Or do we just write that data to our own user database and then all we need to know is the login state (which Google tracks for us)?

I agree let's just go with Google (and FB?) login, and not worry about passwords.

[ericaddison]

OK, I learned how to stash session information in webapp2 (including a userID and whether a session is logged in or not). See here: https://github.com/ericaddison/appengine_samples/tree/master/login2

I think this will be important so we can keep information about a user as they navigate through the site.

[psigourney]

You know, it's a good idea to fully read the documentation before spending a week pulling out your hair on user logins.

https://cloud.google.com/appengine/docs/standard/python/users/

Especially the part about "Google accounts and the development server". We can use the plain-old Users API and it'll provide dummy functionality in development, but once the app is pushed to production, in the app engine, we can specify for Users to use Google account authentication. REALLY wish I had read this a week ago. But it was at the bottom of the page and halfway down the page, something shiny caught my eye and I stopped reading.

[ericaddison]

Yeah, we certainly could use Users... One thing that weirds me out about it is that when you use the "logout_url" you can get from Users, it completely logs you out of all Google pages (like your open Gmail or GAE console...), not just the current app. I wonder if there is a way around that...

FWIW, using token-based login also works on a development server ... if you have an internet connection.

[ericaddison]

OR! (And this is something I was playing with), we can maintain our own login state variable, and just have a link that sets that variable to False when the user "logs out", avoiding the issue with logging out of all google stuff. Maybe?

[psigourney]

Ok... just kinda had a baby mini-breakthrough.

I'm able to do the following: Present the signed-out main index page. Have a user sign in and present the user's Manage page with a list of the user's streams. Have a user click the Create link and go to a Create page. Enter the new stream data in the Create page and have a stream created.

Just created a 'Patrick' branch and uploaded the code.

[ericaddison]

Learned how to use datastore. I pulled my hair out trying to use the google.cloud.datastore library to access the Client class to make queries ... but that was not even necessary! You just do ndbObject.query() for whatever the ndb object is.

Here is my practice: https://github.com/ericaddison/appengine_samples/tree/master/ndbplay

Now I think I have had enough practice, and tomorrow I will start actually thinking more on the actual project :)

[ericaddison]

@psigourney Looking at your Patrick branch ... nice! That looks great! I'll start working on my services tomorrow evening.

[ericaddison]

Have you guys looked at Bootstrap at all? Totally a great resource for starting with some pretty nice css/js templates...

[psigourney]

I remember hearing about it during my undergrad web programming course, but I never actually used it on any of my projects.

On Thu, Sep 28, 2017 at 6:35 PM, Eric Addison notifications@github.com wrote:

Have you guys looked at Bootstrap at all? Totally a great resource for starting with some pretty nice css/js templates...

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ericaddison/APT_miniProject/issues/4#issuecomment-332991886, or mute the thread https://github.com/notifications/unsubscribe-auth/ACI-NZ0l9tbpN9XDzF8hPyZgTSBVjYqeks5snC1WgaJpZM4PZ4RZ .

[ericaddison]

I'm trying it out now. If you check out my eric branch and go into the admin mode (if you log in as admin), you will see it in action. Pretty slick!

[psigourney]

Very nice. Going back to the data model... are you able to display the User information for a stream on the Manage page? When using KeyReference types in the model, when I'd list the Streams, for the user info I'd only get the object key, not the actual user email/name/etc.

On Thu, Sep 28, 2017 at 10:44 PM, Eric Addison notifications@github.com wrote:

I'm trying it out now. If you check out my eric branch and go into the admin mode (if you log in as admin), you will see it in action. Pretty slick!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ericaddison/APT_miniProject/issues/4#issuecomment-333023882, or mute the thread https://github.com/notifications/unsubscribe-auth/ACI-NSkFu6KyimW7MGBFU_YODq4yA1PMks5snGergaJpZM4PZ4RZ .

[ericaddison]

Right... Well, I think a couple of things could happen... Since we are dealing only with the Google users API for now, we can always access a logged in user variable. No problem: user = users.get_current_user(). That only provides us limited information though. If we want to retrieve our custom StreamUser entity associated with the user, since my branch stores a StreamUser using the google user ID as its key, you could just do:

stream_user = ndb.Key('StreamUser', user.user_id()).get()

This is a slight simplification of how my code in Main.py currently looks. Then we have access to all of the user information that we have stored for that user. We'd have to do another database call to get that info from the KeyObject, but I think essentially that is just like a traditional table join ... right?

If we wanted to extend to non-google logins, we would just have to create a different way of generating a user_id value (just an integer), and use our StreamUser entity. This would also mean we need a little more work to maintain login status, but that is not a big deal using webapp sessions.

Is that kind of what you were asking?

Also, should we plan on a phone call tonight or over the weekend?

[psigourney]

We can do the StreamUser lookups in the py script, but if we're sending the list of owned/subscribed streams to the html, how would we get the user information associated with each Stream object?

On Fri, Sep 29, 2017 at 8:35 AM, Eric Addison notifications@github.com wrote:

Right... Well, I think a couple of things could happen... Since we are dealing only with the Google users API for now, we can always access a logged in user variable. No problem: user = users.get_current_user(). That only provides us limited information though. If we want to retrieve our custom StreamUser entity associated with the user, since my branch stores a StreamUser using the google user ID as its key, you could just do:

stream_user = ndb.Key('StreamUser', user.user_id()).get()

This is a slight simplification of how my code in Main.py currently looks. Then we have access to all of the user information that we have stored for that user. We'd have to do another database call to get that info from the KeyObject, but I think essentially that is just like a traditional table join ... right?

Is that kind of what you were asking?

Also, should we plan on a phone call tonight or over the weekend?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ericaddison/APT_miniProject/issues/4#issuecomment-333128026, or mute the thread https://github.com/notifications/unsubscribe-auth/ACI-NVmiu-cD_dyc1bJ7Y1plhSli6Uh2ks5snPI-gaJpZM4PZ4RZ .

[ericaddison]

If we have a list of subscribed streams, something like

subs = StreamSubscriber.query(StreamSubscriber.user == stream_user.key).fetch()

then I think we can get the owners for each one by (untested):

sub_owners = [ stream.owner.get() for stream in subs]

But that is still in Python. Are you thinking of trying to retrieve the list of stream owners from javascript? I would think that we do this owners lookup in the py script and use it to construct the html...

[psigourney]

I think we need to decide on a data model now. That needs to be finalized before anything else can happen.

On Fri, Sep 29, 2017 at 11:46 AM, Eric Addison notifications@github.com wrote:

If we have a list of subscribed streams, something like

subs = StreamSubscriber.query(StreamSubscriber.user == stream_user.key).fetch()

then I think we can get the owners for each one by (untested):

sub_owners - [ stream.owner.get() for stream in subs]

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ericaddison/APT_miniProject/issues/4#issuecomment-333178144, or mute the thread https://github.com/notifications/unsubscribe-auth/ACI-NWcn9UX77OXGYXP02O5t4PTcTcekks5snR78gaJpZM4PZ4RZ .

[ericaddison]

Sounds good to me! Are you available to talk on the phone? Might be easier...

[psigourney]

This morning I had an epiphany. Instead of sending over a list of Stream objects in the template parameters (where some values are keys), instead do all the parsing/queries/etc in the Python, create a list of dicts and send that over instead.

Basically just send over the data we actually need to display:

    user_streams = Stream.query(Stream.owner == stream_user.key).fetch()
    owned_streams = []

    for parentStream in user_streams:
        streamItems = StreamItem.query(StreamItem.stream == parentStream.key).fetch()
        counter =  0
        newestItemDate = datetime.date(1900,1, 1)
        streamDict = {}
        for item in streamItems:
            counter += 1
            if item.dateAdded > newestItemDate:
                newestItemDate = item.dateAdded
        streamDict = {'streamName': parentStream.name, 'counter': counter, 'newestDate': newestItemDate}
        owned_streams.append(streamDict)

Then send over the owned_streams list to the HTML.

Basically, look at the wireframes, see what data is needed for each page, then send over just that specific data.

I also modified the data model. All this is in a new branch 'patrick_new'

class Stream(ndb.Model): owner = ndb.KeyProperty(indexed=True, kind='StreamUser') name = ndb.StringProperty(indexed=False) coverImageURL = ndb.StringProperty(indexed=False) numViews = ndb.IntegerProperty(indexed=False) tags = ndb.StringProperty(repeated=True) subscribers = ndb.KeyProperty(kind='StreamUser', repeated=True)

class StreamItem(ndb.Model): stream = ndb.KeyProperty(indexed=True, kind='Stream') owner = ndb.KeyProperty(indexed=True, kind='StreamUser') name = ndb.StringProperty(indexed=False) blobKey = ndb.BlobKeyProperty(indexed=False) URL = ndb.StringProperty(indexed=False) dateAdded = ndb.DateTimeProperty(indexed=False, auto_now_add=True)

class StreamUser(ndb.Model): email = ndb.StringProperty(indexed=True) firstName = ndb.StringProperty(indexed=False) lastName = ndb.StringProperty(indexed=False) nickName = ndb.StringProperty(indexed=False)

[ericaddison]

@psigourney Hey sorry I didn't read this fully until just now. I think this new data model would need to include something like a repeated subscriptions field in StreamUser, or else it will be terribly inneficient to find all of the streams a given user is subscribed to. Also it will be hard to find all of the streams that have the same tag.

I'm still going to push back on this idea, though, and advocate for your original scheme that treats Tags, Stream-Tag pairs, and Stream-Subscriber pairs as distinct entities. I would like to include the slight modification of having a stream store a list of items it owns, as I have done in #14 .

[ericaddison]

@psigourney And also I agree with your first comments. Do as much pre-processing in python as you can and then send results into Jinja.

[ericaddison]

@psigourney @mspagon Hey guys, I just merged Patricks PR, but there was a small issue with file names in Main.py. I fixed them and included that in the merge, so you should do a new pull from master.

Looking good I think!

[mspagon]

Just wanted to remind/confirm the 7:30 PM call we had planned for tonight.

Thanks guys!

[ericaddison]

Oh right! Thanks for the reminder!!!

[ericaddison]

@psigourney @mspagon Do you think we could do 8:30 instead?

[mspagon]

works for me @ericaddison

[psigourney]

Sure. 8:30 works.

[ericaddison]

Thanks, guys!

[mspagon]

@psigourney @ericaddison Were we going to meet on Google hangouts tonight? If so can we do 8:30? I'll be busy until then.

Thanks

[psigourney]

8:30 would be fine for me.

On Thu, Oct 5, 2017 at 5:24 PM, mspagon notifications@github.com wrote:

@psigourney https://github.com/psigourney @ericaddison https://github.com/ericaddison Were we going to meet on Google hangouts tonight? If so can we do 8:30? I'll be busy until then.

Thanks

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ericaddison/APT_miniProject/issues/4#issuecomment-334608825, or mute the thread https://github.com/notifications/unsubscribe-auth/ACI-NSBeJMniRRVOFaMvW5WWEdk1qDxZks5spVcHgaJpZM4PZ4RZ .

[ericaddison]

Fine with me! I might even be a touch late to that... :/

[ericaddison]

Sorry guys, I probably won't make it until 9

[ericaddison]

Feel free to start without me

[mspagon]

Want to wait until 9 or start at 8:30? @psigourney

[mspagon]

Just FYI, it looks like the "social" tab is for the extra credit - logging into facebook and sharing posts on facebook.

[psigourney]

Whoa! The image carousel on view single stream looks great!

[ericaddison]

@mspagon yeah, I just wanted something there ... I like what you put

@psigourney cool! PR #54 has that in it

[mspagon]

I just added a Google document to our team drive for "Phase 1" submission. @ericaddison @psigourney

I just need your input on the things we learned, mistakes we made, and things that magically worked section.

I will post screenshots of our implementation with descriptions or tooltips when I get off work around 6:00PM. I can also do the actual submission and CC you guys with a confirmation e-mail/screenshot.

Thanks guys.

[mspagon]

Feel pretty trashy about this time around. I didn't contribute anything and I realize that. I've spent a good chunk of time attempting to. I really just don't understand what's going on under the hood and I don't feel confident changing the major features.

I don't mind negative reviews, I just don't want the team to have negative synergy moving forward. So just know that I'm doing my best guys!!! Thanks.

Still looking at the file uploader and the entire project but will start looking at the Android stuff and hope that I can get a fresh start and contribute there.

Choosing a more popular, cohesive, and well-documented technology stack for the final project may be easier for me. I don't know.

[psigourney]

No worries. I'm starting to realize I have a Distributed Systems project to get working by Friday, along with reading two papers, writing analyses of them, and studying for the quiz. The past month has been insane.