Facebook: How users want to pull in reports from Facebook

[Erioldoesdesign]

Which user group(s) are the primary audience for this feature/enhancement.

https://bristolstreetharassmentproject.ushahidi.io https://www.bristolwomensvoice.org.uk/

These folks do work around street harassment and stopping hate crime in Bristol in the UK.

Specify key use cases: They are a team of program/data workers and advocacy/storytelling. They also work closely with local authority and council to affect policy change and physical changes around the city.

e.g. They received reporst around harassment in a particular green space and they used platform reports to help get more police presence and street lighting put in that specific area.

Is your feature request related to a problem? Please describe. They currently collect posts via survey responses but are interested in both Twitter hashtag and email data sources. Through these conversations they talked about their activity on facebook: https://www.facebook.com/BristolWomensVoice/ https://www.facebook.com/BristolZeroTolerance/ https://www.facebook.com/events/487004545403788/

They mentioned that facebook is very valuable as an engagement channel for them and they receive comments on posts as well as facebook messages of people reporting street harassment toward sthem

Describe the solution you'd like This kind of deployment user would greatly benefit from further work on facebook messenger bots that report directly to the deployments and/or a way of identifying a specific facebook post's comments as potential reports.

Describe alternatives you've considered A conversation was had with the deployment owner around 'manual entry of posts/data' I advise that they could add these into the deployment themselves but the key signifier was levels of effort and time spent manually transferring reports to the deployment.

Automation is key here.

Additional context TBC I'll try to access a potential facebook report via their channels.

Aha! Link: https://ushahiditeam.aha.io/features/PROD-180

[rowasc]

Thanks Eriol, this is great info to have. @Erioldoesdesign @Angamanga wondering if the FB Bot prototype could work for them at least to get them started (if they have a server to set it up ) ? I know it had a lot of issues but it'd be better than manually copying, I think? I'd like to find a way to fund this using the learnings from Anna's FB Bot and any more user research we can get to improve upon it since it really sounds like a feature we should have :/

[Angamanga]

@rowasc @Erioldoesdesign If they have a server to set it up + then it needs to be approved by Facebook. Usually not that complicated, the Uchaguzi-one got approved within a couple of days. But I would suggest we look into the version of the fb-api first, the one we use is pretty old and that might complicate the approval-process...

[rowasc]

nice catch. I remember the version was potentially a problem at COMRADES and I don't think we fixed it back then because it wasn't affecting them 😬 Ok let me add it to the list of projects that I want to find out where they can be scheduled for our discussions . Thanks <3

[rohit645]

@Angamanga @tuxpiper @rowasc Currently, posts are being collected through forms/survey responses. I was wondering if FB could be used to serve as one of the data sources for posts-collection using its FB graph API.I was researching on this topic for quite a while. If this is possible can it be added to the third project i.e. Content disclaimer + work in the platform in GSoC ideas list? I would be glad to work on this. I am already working on it and will be sharing my draft proposal for the same shortly.

[Brad-Anthony]

I will be watching this progress carefully. This would be an immensely helpful development which we've been curious about for years. Especially if it can - through the graph if it's still accessible - pull in some of the old meta tags that would be pertinent to intelligence gathering.

[Brad-Anthony]

Before the Cambridge Analytica scandal rocked, we looked at the FB API as a report data source and it was a wealth of (non-personal) info available for pulling into posts that would really speed up data collection for important issues and analysis. If it's at all possible; I'm happy to help logistically in some way. We have an instance we can test on.

[tuxpiper]

Definitely interesting, though not sure about what flexibility is allowed in terms of GSoC projects at this point.. 🤔

@rohit645 Would you have a few more words specifically about how does what you have in mind relate to the OG by @Erioldoesdesign ?

[Erioldoesdesign]

@Brad-Anthony Do you have a deployment you currently or previously managed? As a user that would like this feature, your input and thoughts are super valuable to understand if this feature request does what is needs to!

@rohit645 and @tuxpiper if I can help flesh this out too let me know and I'll do what I can, but I expect your right with 1 week to go on GSoC it's best to have it as a stretch goal! 💯 💐

[Brad-Anthony]

@Erioldoesdesign Yes we have a fairly current instance with dev, staging and production environments based on the open source repo. I'm very happy to collab here to get this working, I've been super keen for this kind of feature for years so let's make it happen. Now I'm in lockdown I have a lot of time to contribute. I'll ping my SysAdmin to get things up to speed as well.

[Brad-Anthony]

Seeing the note above about the FB chat bot; if there's a possibility of a reporting workflow that runs through a conversational UI this would be amazing for non-tech savvy users who really can only handle using FB on a good day; but we could still get reports and data through them. I've been thinking about this - planning this - for a while but we haven't gotten around to development.

[Erioldoesdesign]

@Brad-Anthony @Angamanga is the best person to talk to about the chat bot prototype. There's some good work done there as part of a project we called 'COMRADES' so if you see that referenced anywhere, part of that project was a conversational facebook 'chat bot' for reporting via a deployment users group/page on facebook.

[Brad-Anthony]

I don't want to hijack the thread here; is there a better forum to discuss details and options?

[Erioldoesdesign]

@Brad-Anthony most likely this issue here: https://github.com/ushahidi/platform/issues/2081 It's the open ticket that describes improving the facebook chatbot but the content of the ticket is (to my understanding) a bit out of date/lacking in context.

But! definitely deferring to @rowasc and @Angamanga to direct more appropriately as they may want to create a new ticket rather than use this legacy one. They're all quite busy with Covid-19 response right now but as another open-source contributor, I'd happily work together to try and fill this issue out more!

[Angamanga]

@rohit645 We cannot add more projects to GSoC/Outreachy right now and we don't have the resources for another project I am afraid. But if you (and @Brad-Anthony) are interested, we made a facebook chatbot a few years back, and tested it during the Kenyan elections. The repository can be found here: https://github.com/ushahidi/platform-facebook-bot. There is also a paper describing our findings during the elections: http://oro.open.ac.uk/55325/.

[rohit645]

@tuxpiper @Angamanga got it. I would definitely love to work on this feature whenever it is started. @Brad-Anthony, we can have a separate thread for this feature on the Ushahidi 's Gitter channel. Thanks, @Erioldoesdesign :).

[rowasc]

thanks folks ! regardless of the GSOC status, let's keep the convo here, I think? that way everyone can participate

[Erioldoesdesign]

Let's keep the chat here in this issue @rohit645 and @Brad-Anthony so that the information is not lost in platforms like Gitter - GitHub issues is a 'source of truth' so while it might not be quick and chat-like it's a good place to keep records :)

[rohit645]

@Angamanga. While at Lockdown, I thought of giving a try to integrating this feature. I was wondering how this bot would possibly work. What would be it's expected behavior? What would be the user flow for using this bot?

As far as I can understand :

1. Users will be able to submit reports/form surveys with this chatbot.
2. There will be the conversation of the user with the bot. And the minimal bot will collect meaningful information (e.g. Title, description, etc.).
3. Users shall be able to submit the survey using this bot.
4. This data will then serve as the source information.

I have another question how this bot will be accessed by the user?

1. This could happen through the Ushahidi FB page.
2. Messenger link in the Ushahidi's client itself.

Any other technical guidance on this feature will be helpful.

[Brad-Anthony]

@rohit645 here's some insight from my background work for our project to answer your points.

Assumptions:

1. Correct
2. Correct. The bot would use the conservational UI to map user responses to specific data fields with data validation filters as an intermediate step that's not visible to users; title, map location, media attachments, categories. etc.
3. Correct
4. That submitted data will serve as the original source which is noted inside the Ush. report. At least in our use case we want to see the FB user account meta data to mitigate against malicious users and sock puppet accounts for moderation and security purposes.

Questions

1. I would think, ideally, these bots operate independently of the Ushahidi FB profiles. They should operate in their own organizational silos from the group running them; and connect to the users through their groups and chat UI. I imagine for various reasons we likely don't want to bottleneck on the Ushahidi FB page. @tuxpiper thoughts?
2. Do you mean putting a Facebook bot link inside Ushahidi platform? Maybe there's a compelling reason for this I don't see. I'm operating from the assumption that users will already be on Facebook and wanting to submit a report from there; but if they're already inside a Ushahidi platform client there's possibly little incentive to leave and report from FB...

Reference https://developers.facebook.com/docs/workplace/integrations/custom-integrations/bots/

[tuxpiper]

Yep just like each deployment has its URL , I think each deployment should have its chatbot under its own facebook group or embedded from there into their project's website. Unfortunately, this means an additional setup process for each deployer that would want to use this feature, but I don't really see a way around it.

For some ideas about basic chatbot style interaction for filling a Ushahidi Platform survey, I suggest taking a look at an example based on USSD: https://medium.com/studevs/my-google-summer-of-code-experience-gsoc-with-ushahidi-d314b634c6e1

[Erioldoesdesign]

@tuxpiper Could the set-up process be similar to how deployment owners set up Twitter data source which then prompts them to create a Twitter application?

The only hiccup I can see as a global concern is places where Facebook is banned or limited, but I don't know enough about the global situation of facebook blockage. I'm sure there's a similar problem using Twitter that some folks have probably gotten around with detailed techy-routes.

How does the Ushahidi team feel about this being scoped/explored as a community issue? I'm sort of forming a plan of action in my brain of the moving pieces needed for this but it'd be good to understand/not overstep the core team here.

[Erioldoesdesign]

There's a test deployment set-up with the chatbot running if I remember from my time internally 🤔 It might be nice to have someone techy pick that apart (in a nice way!) to understand what was done already?

[Brad-Anthony]

If the chat-bot assistant runs integrated with FB chat/messenger, then users would need access to FB to report on that channel. Are there any cases where a VPN doesn't resolve access issues? I used a VPN while living in China to access FB, so presumably there are only a handfull of countries with stronger national firewalls preventing access. As a fallback; users can always default to reporting on the platform client directly.

[Erioldoesdesign]

@Brad-Anthony this is for sure a reasonable and useful way of doing this work under firewall but I also know some countries monitor VPN usage (Uganda I believe during their social media tax time?) and also some users of Ushahidi platform are at the very beginning levels of knowing how to use the internet and services like Ushahidi.

For those in these restricted countries, I would want to try to see if we can detect that through language choice or maybe by user choice and then give them some support ins getting up a VPN too. Ushahidi's platform tries to include the most under-served of users after all :)

[Brad-Anthony]

@Erioldoesdesign I think we have options for reliable PHP based geo-location via user IP address. A backend dev could confirm this but I suspect that a simple logic loop could work, for example "If a user is visiting from a Uganda IP address then they are provided with URL redirect to page based instructions for assistance; or to whatever output is valuable.

[tuxpiper]

@tuxpiper Could the set-up process be similar to how deployment owners set up Twitter data source which then prompts them to create a Twitter application?

The only hiccup I can see as a global concern is places where Facebook is banned or limited, but I don't know enough about the global situation of facebook blockage. I'm sure there's a similar problem using Twitter that some folks have probably gotten around with detailed techy-routes.

How does the Ushahidi team feel about this being scoped/explored as a community issue? I'm sort of forming a plan of action in my brain of the moving pieces needed for this but it'd be good to understand/not overstep the core team here.

Yes I think that would be a very similar process. @Angamanga knows more about the particulars.

I feel pretty good myself about community contributions in general :)

[tuxpiper]

re: blocked facebook.

I'm not 100% sure it's the case, but knowing the ethos of facebook .. I'm kind of supposing.. wouldn't someone need to be signed in as a facebook user in order to even be able to open their messenger?

In that case, the end user would probably only contribute through the bot if they are already a facebook user, and in that case I'm supposing already familiar to circumventing country / institutional firewalls applying to them.

[Brad-Anthony]

@tuxpiper Yes, good point. The reporting via FB channel is likely only valuable for existing FB users UNLESS messenger integration is possible outside of FB. I don't know about the latest mechanics of this; but I know there have been some updates to the architecture over the last year.

All reporting channels will not be available equally to all users; I don't see this as an issue considering Ushahidi has been able to accept reports through various mediums for years.

[Erioldoesdesign]

good points both! re. facebook user being needed. I suppose I'm thinking of all the folks that have 'sock puppet' accounts (like me!) or people that create an account and then delete shortly after as well as those having activity tracked. Thoughts for further down the road!

I sadly don't have access to the demo facebook bot anymore (nor the test deployment that is was part of) though I remember a fair bit about it.

I'd love for us to start thinking about what requirements this feature would need so that I could perhaps work on some mock-ups and plans (unless someone else wants to run with that?)

Reading back over the initial issue, the user I interviewed for this also wanted a way to detect potential posts via comments sections in facebook. That sounds pretty difficult (or perhaps impossible?)

[Brad-Anthony]

@Erioldoesdesign we're quickly getting into the realm of credibility. Disposable sock puppet accounts come with some inherent issues that may warrant more in-depth conversations, specifically about verification, vetting and security.

Shall we map out the nuanced user cases to better anticipate some problems, and implementation directions?

@digitalhuman can you add your thoughts to this? If user anonymity is important should they be encouraged to use FB reporting options at all?

@Erioldoesdesign I don't know if it's every been fully fleshed out but wasn't there an option somewhere in the platform client to pick up hashtags on feeds? I would guess that anything more complicated would require some sort of NLP watching for location intelligence cues: coordinates,etc and seems to me like a much bigger scope.

[digitalhuman]

@Brad-Anthony well that is an easy question! :) NO. Because as soon as you create a relation between FB and you anonymity is gone. They track and correlate any bit they get their hands on.

[Brad-Anthony]

@digitalhuman thank you!

I'm not sure how far we want to go down this rabbit hole. Users worried about security, anonymity, should not be using FB at all anyway regardless. Oppressive governments have back doors into FB to track users through various means if they really want to; sock puppets or not.

[Erioldoesdesign]

@Brad-Anthony @digitalhuman Always good to have the public conversation (here in issues) as it at the very least, proves that we are thinking about it and considering it even if it doesn't become part of an implementation :) and you never know, we could stumble across a good idea in the discussion!

Re. Hashtags - pretty sure you could do this in your twitter data source set up. Though it can be more of a problem than a benefit if a user chooses a generic hashtag they end up with thousands of junk posts in pending approval...

Okay, so from my pov there's a few things:

1. The UX/product design process of adding facebook chat via a users specific group/page as a data source and the page/process journey of setting that up in Ushahidi platform as well as trying to support users setting these things up (where to find Facebook API or other details that Ush platform needs to function)

2. How a chatbot ingests messages e.g. Does it connect to a survey? how? which survey? does it cycles through the Q's in the survey in the messenger (similar to USSD). What do errors/answers that don't map look like? Can the admin that owns the facebook page 'overwrite' the survey chat process by sending a human written message from the same messenger 'window' does that 'cut off' that data source or does Ush platform ingest that as 'unstructured data' to then be structured? What happens if the full survey is not completed (does it ingest half a survey?) What about mandatory fields? How does it ingest location (most people don't use lat/lon in FB chat they'll say at best a full address and at worst a fuzzy location 'The church down the street') what happens if the location does not have an address, can we help users get lat/lon to add into the FB chat?

3. How does it display in Ush platform? strong personal preference for as an 'under review' post. Can we suggest a process of verification? We'll need to add facebook chat as a data source filter in the mode context window (the bit with surveys and profile pic). What do admins see when they go to edit the data source?

4. Exporting and importing data - how does FB chat affect this? importing is probably largely impossible/too difficult to be useful to admins but export could/would be useful but we need to be careful with PII data (if it's even ingested to begin with)

5. How would 'conversation with author' play into this? if at all.

6. What are the mandatory security concerns we should legally adhere to? What does GDPR say about this kind of work? Let's not get folks in legal trouble!

7. Documentation overall and specifically, common problems with setup, how to test it's working, and some advice about privacy, verification & security in the documentation hub and Ushahidi support pages

haha those are my 'off the top of my head' thoughts about this :^)

[Brad-Anthony]

@Erioldoesdesign I'll omit responses to numbers I don't have a comment on.

Re: #hashtags I have tried this using various tools in the past and I still think it's akin to casting a massive drift net that catches too much (fish)data to be useful. Unless there's a campaign that creates and uses a highly specific hashtag only that org uses for reports. My assumption is that a user should possess a certain intention and agency to submit a report and we shouldn't be encouraging passive collection methods for various reasons.

2. I imagine the chatbot stores the question responses as variables and passes them to a report/survey data field in the DB. The associated report/survey would be chosen during setup but allow for changes. Errors might look like a virtual assistant error response "I'm sorry I don't understand that response. Also, text tooltips should be included to precondition the user to enter the right format and we can facilitate this part of early validation by using progressive disclosure style questions with pre-loaded answers. For example, "What country are you in" and the bot already knows it's going to be one of 194+ possible entries so when they start entering the answer the choices narrow to the right selection. Maybe some auto-detection libraries could be used to pre-populate the relevant questions/associated data fields. An administrator should have an ability to manual override and engage with the user in the chat, to provide real-time assistance in person. Half a survey would likely act as an abandoned cart, held temporarily in the cache, with a potential reminder sent to the initiating party to come back in fill in the remainder. Location data might be most user friendly if there's a library we can use to help them "drop a pin" like in Google maps, Grab, Uber, etc where the geolocation heavy lifting happens in the background with no need for lat/long to confuse users. With a supplementary text entry for proper addresses as a fallback. Also, if a location doesn't have an address the Google API for instance has fallback geolocation references like the plus codes (ex. X4HM+3C, Cairo, Egypt)

3. For consistency, it should probably display as suggested "under review' or "moderation" step for human verification. If applicable, WordPress type options for comment moderation may work well here such as "Bypass moderation if user is previously verified". Should the user be able to toggle a security setting between low security non-anonymous report or high-security report which pseudonymizes or anonymizes their PII submissions. If the former, nothing needs to be scrambled and the PII contact data appears on the platform report; if the latter, an extra setting activates to only display the PII style data to the Ushahidi platform admins. The admins should be able to scramble/edit the PII data from the backend also just in case changes need to be made after the fact. I've often though of this working in a security setting as a pseudonym where needed, the report author name is scrambled to a non-PII key string and their authentic PII is only visible to the backend admins.

4. I can't think of a compelling use case for importing data to the chatbot. Export from the platform is relevant; but should take into consideration the security footing set by reporters during submission and/or also have their own data protection policy relevant to their own circumstances. Does it stand to reason to have an export setting that optionally prevents the exporting of PII data to limit the potential number of points of failure? @digitalhuman how to the high security footing apps/orgs handle this? In our case, I want that initial PII info to vet the reporter and build an instance of trust for followup reporting but after that early handshake maybe the PII can be disintegrated or at least have that option available.

5. This is a question for @digitalhuman - Vic which security policies, compliance levels, etc, would you say are mandatory + recommended? Is there an optimal target to balance privacy with security and protection?