PLANNING ISSUE : Reply via Twitter .

[sagarpreet-chadha]

Please describe the problem (or idea)

This is a planning issue for reply by twitter : feature .

Let's first discuss the best approach and then we shall break this into smaller independent issues .

More Details 🔜 !

• One way we can implement this is explained here : https://publiclab.org/notes/sagarpreet/02-16-2018/social-media-integration-and-part-of-interactive-project-maps

=> Email Integration #2672 (@namangupta01 )

[jywarren]

Cool! Link to Naman's new issue too!

On Sat, Apr 28, 2018, 3:25 AM Sagarpreet Chadha notifications@github.com wrote:

Please describe the problem (or idea)

This is a planning issue for reply by twitter : feature .

Let's first discuss the best approach and then we shall break this into smaller independent issues .

More Details 🔜 !

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/publiclab/plots2/issues/2671, or mute the thread https://github.com/notifications/unsubscribe-auth/AABfJ6EpFnMq_H5zR8vLtf7kXWZJbELxks5ttBlcgaJpZM4TrSYe .

[sagarpreet-chadha]

@namangupta01 , @jywarren ...lets start the discussion on this ? When are you both free ?

[namangupta01]

Yeah! On Jun 14, 2018 12:55, "Sagarpreet Chadha" notifications@github.com wrote:

@namangupta01 https://github.com/namangupta01 , @jywarren https://github.com/jywarren ...lets start the discussion on this ? When are you both free ?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/publiclab/plots2/issues/2671#issuecomment-397198245, or mute the thread https://github.com/notifications/unsubscribe-auth/ATwCYebYN7qM2c0Emtdw2jhwYIILp5r1ks5t8g_fgaJpZM4TrSYe .

[namangupta01]

I have some implementation in my mind...will post in a while.

[jywarren]

Joe (who just started work with the Public Lab staff) had a really good question -- would this be a channel for spam posting? Do we have to worry about Twitter bots spamming, or maybe think about a moderation strategy?

What are ways we could watch out for this? Open brainstorm!

Like, maybe response tweets have to be "liked" by... someone?

On Thu, Jun 14, 2018 at 4:47 AM Naman Gupta notifications@github.com wrote:

I have some implementation in my mind...will post in a while.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/publiclab/plots2/issues/2671#issuecomment-397237305, or mute the thread https://github.com/notifications/unsubscribe-auth/AABfJ_g3eeUD8-7bvLjXmQLYIz0LYM1qks5t8jEFgaJpZM4TrSYe .

[namangupta01]

Spamming is the major issue. I was reading documentation from past few days. I was also wondering if we implement as you said "response tweets have to be liked" i found a major issue. The issue is that whenever a user retweets on tweet (which is a node) we can know about the retweet through apis but when the retweet is liked we can't know about it. The only way to know about it is to search for every retweet status after certain interval of time which is very bad because lets suppose we have 10k retweets then we have to search for 10k retweets status for being liked in certain interval of time. And also this 10k figure will continue to rise over time.

[namangupta01]

And also one main point which could leads to no spamming is that we can only add the tweets of those users whose twitter handle is associated to any publiclab account just like we were doing in reply-by-email where email of email sender is first checked for associativity with the publiclab account. This could leads to no spamming.

[sagarpreet-chadha]

@namangupta01 ...great points !

• Spamming is the issue here , i guess the only moderation technique i can also think of is that a moderator has to approve the comment on a tweet by liking it (Similar to how things work in publiclab.org ) . => moderator can be people with PublicLab's twitter account OR a PL community person who has liked from his own account .

• I guess the point is to bring a member who is new to the community , so bringing comments of only existing people will solve half the problem .

What do you think ?

[namangupta01]

Great thinking! But the only point i am concern with is that we have to check explicitly for the tweet's being liked in every request for all the tweets on the publiclab tweets which can become very large request as the tweets increases.

[namangupta01]

And most of the time we will be making request for the tweets for nothing...that was my concern. What do you think?

[namangupta01]

However moderation technique on the publiclab website interface would work great. Like approving comments ?

[namangupta01]

@jywarren As email feature is almost complete we could discuss over it?

[jywarren]

I'll look to block out some time on this in a day or two... thank you! Trying to get all the big features fully up and running before moving on. Thanks!

On Mon, Jun 25, 2018 at 7:18 PM Naman Gupta notifications@github.com wrote:

@jywarren https://github.com/jywarren As email feature is almost complete we could discuss over it?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/publiclab/plots2/issues/2671#issuecomment-400125415, or mute the thread https://github.com/notifications/unsubscribe-auth/AABfJ-UOBtZ0G4haKmQRlSr0WJJ3Q30pks5uAW-ugaJpZM4TrSYe .

[namangupta01]

@jywarren Can we put node_id in tweets? So that it can be easily identified?

[namangupta01]

@jywarren Is it possible to put node_id in tweets?

[jywarren]

I think so, yes! But we turned off the autotweeting for a bit. Could we recognize links like publiclab.org/n/____ via Twitter API?

On Sun, Jul 15, 2018, 9:26 AM Naman Gupta notifications@github.com wrote:

@jywarren https://github.com/jywarren Is it possible that to put node_id in tweets?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/publiclab/plots2/issues/2671#issuecomment-405091999, or mute the thread https://github.com/notifications/unsubscribe-auth/AABfJ575zPOLYU6bpFdMMkZA0vl-Xg_Yks5uG0LpgaJpZM4TrSYe .

[namangupta01]

@jywarren So publiclab.org/n/____ will present directly in tweet or there are some other links which redirect to this link as before? ex:

[namangupta01]

I was researching and found that all the links that comes in tweet through twitter api is a twitter link which gets redirected to actual page. I was wondering if we use link to find the node and tweet contains more than one public.org/n/__(one for referencing post on public lab and other may be used in any of node content on public lab post itself) then in that case comment may get added to the wrong node because we donot know what the actual node we should add this comment? What do you think?

[namangupta01]

@jywarren So, by which way should i implement it ? using node_id or the other way. I am finalising the way of implementation.

[jywarren]

This is a bit more complex when we don't have the auto-tweeting happening, because we don't get to format the original. I can bring it up on our staff call or open call today (https://publiclab.org/opencall) and we can try to figure out a good way forward!

On Mon, Jul 16, 2018 at 6:20 PM Naman Gupta notifications@github.com wrote:

I was researching and found that all the links that comes in tweet through twitter api is a twitter link which gets redirected to actual page. I was wondering if we use link to find the node and tweet contains more than one public.org/n/__(one for referencing post on public lab and other may be used in any of node content on public lab post itself) then in that case comment may get added to the wrong node because we donot know what the actual node we should add this comment? What do you think? [image: screen shot 2018-07-17 at 3 40 13 am] https://user-images.githubusercontent.com/20709985/42786554-59b06d14-8974-11e8-96bd-e2c804da47c9.png [image: screen shot 2018-07-17 at 3 40 23 am] https://user-images.githubusercontent.com/20709985/42786555-59f820f0-8974-11e8-9aa8-ef78abc46dfe.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/publiclab/plots2/issues/2671#issuecomment-405399249, or mute the thread https://github.com/notifications/unsubscribe-auth/AABfJwu8hr_XV85dxj3NlaTRgsBCaZ8jks5uHRF3gaJpZM4TrSYe .

[namangupta01]

I am going to use twitter gem using which we can also do auto tweet --just let you know that we can do autotweeting without using any service. I will look for the way to format.

[jywarren]

OK, so what we're thinking of is:

1. start by implementing only replies by "recognized" twitter users, using the OAuth login system
2. attempt to find tweets by isolating publiclab.org/n/_____
3. could we display all comments in a list only to moderators and allow them to be "approved" somehow? Can you think through how that might work?
4. if we do start allowing all tweeted replies, let's consider having a way to "turn it off" if we start to get a LOT of spam, that way we can try things out.
5. are there ways to display an embedded tweet with its replies, and then, if there are tweeted replies, notify the original poster, but not actually copy them in?

[namangupta01]

I just had a call with @SidharthBansal. I asked about this. So we are not saving twitter handle right now which is require to recognise the users. I think we should save it somewhere so we can recognise it. @jywarren ?

[SidharthBansal]

@namangupta01 you can migrate the user tag model and add handler field. If you need any further help feel free to contact me. Thanks

[SidharthBansal]

https://github.com/publiclab/plots2/blob/master/doc/Omniauth.md https://github.com/publiclab/plots2#ssl-in-development publiclab/plots2#2893 these will surely help you

[namangupta01]

Thanks Sidharth! :)

On Thu, Jul 19, 2018 at 9:55 PM Sidharth Bansal notifications@github.com wrote:

https://github.com/publiclab/plots2/blob/master/doc/Omniauth.md https://github.com/publiclab/plots2#ssl-in-development publiclab/plots2#2893 https://github.com/publiclab/plots2/issues/2893 these will surely help you

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/publiclab/plots2/issues/2671#issuecomment-406336137, or mute the thread https://github.com/notifications/unsubscribe-auth/ATwCYUJLLS9Lc722_YJWLBdLhWed_orrks5uILMDgaJpZM4TrSYe .

[SidharthBansal]

I think request.env['omniauth.auth']["info"]["urls"]will work.

IMPORTANT: All the providers give different fields in their hash with different names. So, please take care of filling all other providers info -- url field as NULL if they are not twitter

See the hashes given here and compare:

https://github.com/arunagw/omniauth-twitter https://github.com/mkdynamic/omniauth-facebook https://github.com/omniauth/omniauth-github https://github.com/zquestz/omniauth-google-oauth2

See that there hashes are different. Also, please take reviews from me in your prs. I will be happy to help.

[SidharthBansal]

Google_oauth2 also has same name --> info -->urls But facebook has handler in extra_info. So, please fetch [extra][raw_info][link] and store in user_tag as handler. Github --> It is not written in the documentation. May be it does not provide. Search if github provides than assign handler of github to this field All those which can't be set in handler like Github you may put them as NULL.

[namangupta01]

@jywarren should i make separate column for every oauth?

[jywarren]

user tag model and add handler field

I wonder if we could just create a set of additional custom oa- fields for this info in a flat key:value system, rather than migrating the database... Sidharth, what do you think?

On Thu, Jul 19, 2018 at 2:18 PM Naman Gupta notifications@github.com wrote:

@jywarren https://github.com/jywarren should i make separate column for every oauth?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/publiclab/plots2/issues/2671#issuecomment-406368740, or mute the thread https://github.com/notifications/unsubscribe-auth/AABfJ_XCeor_CeJB2aXO3eJHr0FJhX2rks5uIM1agaJpZM4TrSYe .

[SidharthBansal]

We need not to complicate user_tag much. It should be easily understandable by the new comers. We need to add fields a lot of times in the future. So, consider the situation that we did according to this approach, then providers will have value as uid:provider:handler. Github doesn't provides any handler. Now sitaution comes we need image to be displayed at profile page. So this will create problem. So then we need to migrate the db. So, my opinion is to migrate the db from now.

[SidharthBansal]

We need to see the situation where in future we need to implement posts in and out of fb or twitter after logging in then also, we will need other fields. We can't make the value longer

[SidharthBansal]

Flat system is also a good option. But I am concerned about the better readability of the user_tags for the future.

[jywarren]

Let's think this through -- what modification exactly to the user_tag model are you looking for? Could we add just one text column, and store JSON in it? I'm trying to think of the most maintainable and flexible way, so we don't have a lot of very specific features on the user_tag model that won't be re-usable for different purposes. Make sense?

Thanks, all!

[SidharthBansal]

@jywarren yeah I got your point. Let's append the user_tag's val field.

[jywarren]

I do think using a longer text field could be useful for lots of things, so if you want to just have a collection of other things to store along with the key:value, we could do that...

On Thu, Jul 19, 2018 at 6:00 PM Sidharth Bansal notifications@github.com wrote:

@jywarren https://github.com/jywarren yeah I got your point. Let's append the user_tag's val field.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/publiclab/plots2/issues/2671#issuecomment-406427465, or mute the thread https://github.com/notifications/unsubscribe-auth/AABfJ4Hlow0UZgw-e-I0WBp9Lpt0MiC0ks5uIQFKgaJpZM4TrSYe .

[namangupta01]

@SidharthBansal Everytime i do login via twitter on local it redirect to public lab website. Any idea?

[namangupta01]

I guess i Got it why it is occuring

[namangupta01]

@SidharthBansal i am getting this error kindly help:

[namangupta01]

[SidharthBansal]

Add advance settings to allow https for unsecure websites

[SidharthBansal]

127.0.0.1:3001 is unsecure

[namangupta01]

Resolved thanks @SidharthBansal 👍

[SidharthBansal]

@jywarren I was today searching for how the flat system can be achieved. The following file is the main file in my Soc Project. https://github.com/publiclab/plots2/blob/f2fe98da351a562152a85bd068acb1a9e6e7f7c0/app/models/user_tag.rb#L24-L30 If we do append the string then the whole Soc Project will need update at multiple positions. Also all tests will fail. We need to write all the tests again. @namangupta01 what do you suggest? If we create a new field in the db then Soc project can be modified without any conflicts. All the tests and code for Soc Project will remain as it is.

[namangupta01]

I think @jywarren said here that can we add new text field where we can store JSON object that we are getting in response.

Let's think this through -- what modification exactly to the user_tag model are you looking for? Could > we add just one text column, and store JSON in it? I'm trying to think of the most maintainable and > > flexible way, so we don't have a lot of very specific features on the user_tag model that won't be re-usable for different purposes. Make sense?

Am i right @jywarren ?

[SidharthBansal]

Lets wait for jeffs input.

[namangupta01]

Yes......

[namangupta01]

My another concern is that when we will poll for new replies on twitter in every 5 seconds or in every 10 seconds we are going to find if particular handle exist or not in database. Suppose in every response we got some replies and then we have to find the handles for that we will be parsing the text field for extracting the handle and to check if that handle exist or not. Would it increase the time complexity of the system? Because for every handle checking we will be doing O(n) work? And also no indexing would work in this case to make the db transaction faster. Sorry @jywarren this is taking some time because i want to be fully confirmed about the implementation before opening the pr since it is going to be complex system.

[jywarren]

Hi! Yes, i was saying that if there is a lot of content you want to store together in a less-flat way, we could make a data text column in the user_tag table, and store JSON in it. But can you offer a clear example of the kind of data you want to be storing so we know if it's the appropriate way to store it? Thanks!

For the complexity, i agree, this could create significant load. Can you write pseudocode for the series of queries and I can help optimize? I'm thinking, we could craft a more efficient query, but help me figure out the desired steps first. Thanks!

On Fri, Jul 20, 2018 at 4:00 PM Naman Gupta notifications@github.com wrote:

My another concern is that when we will poll for new replies on twitter in every 5 seconds or in every 10 seconds we are going to find if particular handle exist or not in database. Suppose in every response we got some replies and then we have to find the handles for that we will be parsing the text field for extracting the handle and to check if that handle exist or not. Would it increase the time complexity of the system? Because for every handle checking we will be doing O(n) work? And also no indexing would work in this case to make the db transaction faster. Sorry @jywarren https://github.com/jywarren this is taking some time because i want to be fully confirmed about the implementation before opening the pr since it is going to be complex system.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/publiclab/plots2/issues/2671#issuecomment-406711222, or mute the thread https://github.com/notifications/unsubscribe-auth/AABfJ7Z9x-9Cwedez8T-gsMnYR4VQ75zks5uIjbCgaJpZM4TrSYe .

[namangupta01]

Pseudocode:

1. Fetch latest replies from Twitter :
2. For every reply in replies : check if it belongs to any node or post using publiclab.org/n/_____ and if that node exists then find if the user exist using tweet replier. If user exists add the comment.

   tweets = client.search("to:publiclab", since_id: 23423423).collect do |tweet|
   node = Node.where(some_condition) #To find if node exists
   if node.any?
   node = node.first
   tweet_user = tweet.user
   UserTag.all.each do |user_tag|
     if(some_condtion_to_check if user handle exists)
       add_comment
     end
   end
   end
   end