openrightsgroup / cmp-issues

Centralised issue-tracking for the Blocked backend
2 stars 0 forks source link

Ensure data about user's affiliation is recorded from user-selected checkboxes #208

Closed alexhaydock closed 5 years ago

alexhaydock commented 5 years ago

The checkboxes when submitting a report read:

We discussed this data before in an earlier ticket but I can't locate that comment now and I think this issue deserves its own ticket.

For the purposes of further breaking down the 76% of user reports for which the user did not identify themselves in the body text of their report, it would be very useful to locate this data somehow and make it visible.


EDIT: It seems that this data has unfortunately been left un-recorded for the current entries in the database. This ticket is now more useful to track fixing this issue so that the data is recorded for future reports.

edjw commented 5 years ago

FWIW, this was mentioned in this issue here.

@alexhaydock – Where would be the best place to make it visible?

@dantheta – Can you have another look for that data and expose it when you get the chance?

alexhaydock commented 5 years ago

If it's at all possible, it would be useful to have entries for which the user has selected "I am a user of XXXXXX" or "I am someone who cares about censorship" cause those reports to be tagged in the database with the report origin of "Site User" and "Cares About Censorship" respectively.

I am satisfied that all of the reports from site owners are correctly tagged, so a report for which someone has selected "I am the owner of XXXXXX" should not cause any change (even if the reporter has selected one of the other categories too).

Similarly, ideally the "Cares About Censorship" tag should only be reflected in the database with the "Cares About Censorship" label if the user has not selected either of the other two options, and "I am someone who cares about censorship" is the only thing they selected.

I appreciate that doing this in an automated fashion might not be as easy as I'm hoping but I have my fingers crossed that it can be done. It will save me from needing to re-categorise 1,300 reports manually.

dantheta commented 5 years ago

The bad news is, it looks like these three fields have been going unrecorded for quite some time.

They used to be recorded by the email template that was generated by the system, but between the template changes and the initial reporting screen becoming semi-optional [1], the field values are not being recorded now.

[1] When a user is making a second or subsequent unblock request, the first of the unblock screens (username & email) is skipped and the saved values are used.

Please accept my profound apologies.

We can fix this of course; I don't think there's anywhere that we can get the previous submissions back from.

alexhaydock commented 5 years ago

Ah I see. That is a shame but at least the data we do have can lead to some meaningful conclusions. It's a good thing we've flagged it for now and can record the data going forward in the future.

For the 76% of reports in which the user doesn't openly specify their affiliation in the body text I can't really make any guesses about the user's status for the purposes of the data, but I can use the rest of the data and mention the 76% as part of the report text.

I'll leave this ticket open although perhaps it should now be used to track ensuring that the checkbox events are counted from now on. Though we do now have the ideal opportunity to feed them directly into the "reporter category" tag that I've been otherwise applying manually and save a lot of manual work in the future.

dantheta commented 5 years ago

That's cool. I think it might be worth clarifying what the "I am someone who cares about censorship" option really means.

I think it may have been intended to provide a third option if the owner/user/other options had been radio buttons, since the user would only be able to select one option. Since they are checkboxes the user can select multiple options, but is unlikely to select owner and user together.

In which case, what does the "someone who cares about censorship" option mean in terms of categorising the reports?

I'm also wondering if we should move the owner & user options to the second unblock screen, so that we have the option of capturing it on each request.

alexhaydock commented 5 years ago

That's cool. I think it might be worth clarifying what the "I am someone who cares about censorship" option really means. I think it may have been intended to provide a third option if the owner/user/other options had been radio buttons, since the user would only be able to select one option. Since they are checkboxes the user can select multiple options, but is unlikely to select owner and user together. In which case, what does the "someone who cares about censorship" option mean in terms of categorising the reports?

Well I can't answer for @JimKillock or the original intent here but honestly from my perspective the only function of the "Someone who cares about censorship" option is perhaps to separate out privacy activists, ORG supporters and staff from ending up lumped into a generic "Other" category. There are plenty of reports from people who fit into those groups who are neither users nor owners of the sites they're reporting for review.

That category is not of critical importance to presenting the data really, since the biggest points of interest we're noting so far relate to the self-identified owners or users of a site. It's just nice to not have a large "Other" category that doesn't make for great looking data on reports.

In terms of categorising the replies for which that option has been selected, I'd have tagged them with something along the lines of "Cares About Censorship" or "Censorship Activist". That's not a category I've created yet in the database since I have no real way of knowing (other than guesses) whether someone falls into that category. Although admittedly I could go back through and retroactively categorise ORG staff that way at least.

For what it's worth, I probably would have these as radio buttons personally. The "Reporter Category" in the database seems to only accept a single entry and I can't picture situations in which people selecting more than one option adds any additional useful info for us.

I'm also wondering if we should move the owner & user options to the second unblock screen, so that we have the option of capturing it on each request.

I think this is a good idea for sure. It's quite nice actually if you look through the data since you can see a few situations where an owner of a particular site has requested an unblock for their own site, and has subsequently stayed on the Blocked site and submitted a few more reports for other blocked sites while they've been there. But essentially what I'd take from that is that we can't count on the radio button/checkbox data being the same between multiple reports. Those users would be "Site Owner" for the first site they report, and should have the opportunity to adjust that setting for any subsequent reports they send to avoid being miscategorised as the owner of sites they don't actually own.

dantheta commented 5 years ago

These values are now being stored properly.