search

willzeng / cograph

A way to build and explore webs of ideas.
http://www.cograph.co/
GNU General Public License v2.0
15 stars 3 forks source link

Pulls tweets #587

Closed davidfurlong closed 9 years ago

davidfurlong commented 9 years ago

TODO's

willzeng commented 9 years ago

Let's write up to exactly specify what we want this feature to do. Some suggestions.

willzeng commented 9 years ago

Given this, perhaps we should bypass the step of storing the tweets in the MongoDB and just throw them directly in Neo4j?

We'll want to build a particular controller that has access to the Neo4j db that is required into passport.js

willzeng commented 9 years ago

So I have coded up a little of this, but am having trouble debugging. Can you take me through the set up for testing twitter signup locally?

Right not I get the error:

Error: Failed to find request token in session at Strategy.OAuthStrategy.authenticate (C:\Users\W. J. Zeng\Documents\GraphDocs\code\clean\graphdocs\node_modules\passport-twitter\node_modules\passport-oauth1\lib\strategy.js:142:54) at Strategy.authenticate (C:\Users\W. J. Zeng\Documents\GraphDocs\code\clean\graphdocs\node_modules\passport-twitter\lib\strategy.js:85:40) at attempt (C:\Users\W. J. Zeng\Documents\GraphDocs\code\clean\graphdocs\node_modules\passport\lib\middleware\authenticate.js:341:16) at authenticate (C:\Users\W. J. Zeng\Documents\GraphDocs\code\clean\graphdocs\node_modules\passport\lib\middleware\authenticate.js:342:7) at callbacks (C:\Users\W. J. Zeng\Documents\GraphDocs\code\clean\graphdocs\node_modules\express.io\node_modules\express\lib\router\index.js:164:37) at param (C:\Users\W. J. Zeng\Documents\GraphDocs\code\clean\graphdocs\node_modules\express.io\node_modules\express\lib\router\index.js:138:11) at pass (C:\Users\W. J. Zeng\Documents\GraphDocs\code\clean\graphdocs\node_modules\express.io\node_modules\express\lib\router\index.js:145:5) at Router._dispatch (C:\Users\W. J. Zeng\Documents\GraphDocs\code\clean\graphdocs\node_modules\express.io\node_modules\express\lib\router\index.js:173:5) at Object.router (C:\Users\W. J. Zeng\Documents\GraphDocs\code\clean\graphdocs\node_modules\express.io\node_modules\express\lib\router\index.js:33:10) at next (C:\Users\W. J. Zeng\Documents\GraphDocs\code\clean\graphdocs\node_modules\express.io\node_modules\express\node_modules\connect\lib\proto.js:193:15)

Though I have switched the token code to the commented out local code in two places in passport.js

davidfurlong commented 9 years ago

Try using 127.0.0.1 instead of localhost On 12 Nov 2014 23:20, "Will Zeng" notifications@github.com wrote:

So I have coded up a little of this, but am having trouble debugging. Can you take me through the set up for testing twitter signup locally?

Right not I get the error: ''' Error: Failed to find request token in session at Strategy.OAuthStrategy.authenticate (C:\Users\W. J. Zeng\Documents\GraphDocs\code\clean\graphdocs\node_modules\passport-twitter\node_modules\passport-oauth1\lib\strategy.js:142:54) at Strategy.authenticate (C:\Users\W. J. Zeng\Documents\GraphDocs\code\clean\graphdocs\node_modules\passport-twitter\lib\strategy.js:85:40) at attempt (C:\Users\W. J. Zeng\Documents\GraphDocs\code\clean\graphdocs\node_modules\passport\lib\middleware\authenticate.js:341:16) at authenticate (C:\Users\W. J. Zeng\Documents\GraphDocs\code\clean\graphdocs\node_modules\passport\lib\middleware\authenticate.js:342:7) at callbacks (C:\Users\W. J. Zeng\Documents\GraphDocs\code\clean\graphdocs\node_modules\express.io\node_modules\express\lib\router\index.js:164:37) at param (C:\Users\W. J. Zeng\Documents\GraphDocs\code\clean\graphdocs\node_modules\express.io\node_modules\express\lib\router\index.js:138:11) at pass (C:\Users\W. J. Zeng\Documents\GraphDocs\code\clean\gra phdocs\node_modules\express.io\node_modules\express\lib\router\index.js:145:5) at Router._dispatch (C:\Users\W. J. Zeng\Documents\GraphDocs\code\clean\graphdocs\node_modules\express.io\node_modules\express\lib\router\index.js:173:5) at Object.router (C:\Users\W. J. Zeng\Documents\GraphDocs\code\clean\graphdocs\node_modules\express.io\node_modules\express\lib\router\index.js:33:10) at next (C:\Users\W. J. Zeng\Documents\GraphDocs\code\clean\graphdocs\node_modules\express.io \node_modules\express\node_modules\connect\lib\proto.js:193:15) '''

Though I have switched the token code to the commented out local code in two places in passport.js

— Reply to this email directly or view it on GitHub https://github.com/willzeng/graphdocs/pull/587#issuecomment-62813986.

willzeng commented 9 years ago

yep that worked! sweet.

willzeng commented 9 years ago

hmm, am not getting the tweets to pull in properly. I don't see any in the compose mongoDB for the heroku that we're using either. How did you see that the tweets were actually pulled?

willzeng commented 9 years ago

@davidfurlong Ok so have figured out the workflow to get signing in working and testing, but the call to get the user timeline is returning an undefined HTTP error. This is weird as I have tested it in the twitter API console and it seems to work fine.

Were you able to see the timeline data get pulled in at any point?

davidfurlong commented 9 years ago

@willzeng this should work now

willzeng commented 9 years ago

Yep that works for me. Now we need to take that string and turn it into a cograph that updates each time the use logs in?

willzeng commented 9 years ago

@davidfurlong Rad. So I've got the first check box ticked off for this PR. When you sign up with twitter for the first time it takes your most recent 200 tweets and drops them into an automatically created cograph. The next step could have two paths:

We could make it so that everytime you log in with twitter, your recent tweets are merged into that special "tweet" cograph.
And/OR We could add an 'import tweets' button to every cograph that lets your grab your most recent 'n' tweets and adds them to that cograph.

What do you think?

davidfurlong commented 9 years ago

Flexibility is probably best. One graph that has your tweets / people you follows? Then ability to add twitter "landscapes" based on a filter as an import button on a graph

On Mon, 9 Feb 2015 15:58 Will Zeng notifications@github.com wrote:

@davidfurlong https://github.com/davidfurlong Rad. So I've got the first check box ticked off for this PR. When you sign up with twitter for the first time it takes your most recent 200 tweets and drops them into an automatically created cograph. The next step could have two paths:

We could make it so that everytime you log in with twitter, your recent tweets are merged into that special "tweet" cograph.

And/OR We could add an 'import tweets' button to every cograph that lets your grab your most recent 'n' tweets and adds them to that cograph.

What do you think?

— Reply to this email directly or view it on GitHub https://github.com/willzeng/cograph/pull/587#issuecomment-73533246.

willzeng commented 9 years ago

Ok so basically both. I think the functionality is quite similar so doing one and then the other should be relatively straightforward.

willzeng commented 9 years ago

@davidfurlong See todo's at top

willzeng commented 9 years ago

Got the update on login functionality working @davidfurlong Check it out.

willzeng commented 9 years ago

Maybe the thing to do for import is to just let people type in twitter search queries and we just give them whatever results come in?

davidfurlong commented 9 years ago

Maybe the thing to do for import is to just let people type in twitter search queries and we just give them whatever results come in?

yeah thats cool + easy to do

willzeng commented 9 years ago

Lets plan to get that and some automatic linking of them done before pushing twitter integration live on Wednesday.

On 21:09, Mon, Feb 9, 2015 David Furlong notifications@github.com wrote:

Maybe the thing to do for import is to just let people type in twitter search queries and we just give them whatever results come in?

yeah thats cool + easy to do

— Reply to this email directly or view it on GitHub https://github.com/willzeng/cograph/pull/587#issuecomment-73590575.

willzeng commented 9 years ago

Yep, I don't have this bug. We'll try and replicate when you get here.

davidfurlong commented 9 years ago

the last todo is on branch nsearch, just needs functionality

willzeng commented 9 years ago

@davidfurlong The update is now fixed so that there is an auto updating functional cograph of a users tweets. Give this a check to make sure it is also working with your account.

If so, we can change the number so that initial signup pulls 200 tweets and each login checks the most recent 200? (it automatically avoids making duplicates) This should then be merge-able.

I created a new pull request for the query adder box: https://github.com/willzeng/cograph/pull/630

willzeng commented 9 years ago
willzeng commented 9 years ago

This is now working and stable, though when looking at the larger tweet cographs it makes me think that we should really be doing some optimizations.

davidfurlong commented 9 years ago

can you post an example? what kind of optimizations

davidfurlong commented 9 years ago

are we still storing it in the mongo (probably not a good idea)

willzeng commented 9 years ago

Ah yes we are, and needlessly so. Go ahead and remove that.

I mean that graphview gets a little slow with 200 nodes and lots of images. Set TweetNumber to 100 and give it a try.

We should probably do some processing of the images to be smaller res when in the graph. This issue is seperate from this PR.

On 15:21, Sat, Feb 21, 2015 David Furlong notifications@github.com wrote:

are we still storing it in the mongo (probably not a good idea)

— Reply to this email directly or view it on GitHub https://github.com/willzeng/cograph/pull/587#issuecomment-75376161.

davidfurlong commented 9 years ago

Are we good to go?

willzeng commented 9 years ago

If it worked for your account then yeah. We can then do another update that includes the functional custom search box import feature.

On Tue, Feb 24, 2015 at 6:45 PM, David Furlong notifications@github.com wrote:

Are we good to go?

— Reply to this email directly or view it on GitHub https://github.com/willzeng/cograph/pull/587#issuecomment-75818889.