"Signals" and "Indicators"

[sandhawke]

Some people use the terms "signals" and "indicators" interchangeably. (Call this Option 1)

Other people seem to think "signals" are what machine-learning takes as input and "indicators" are what humans see and use in making trust decisions. (Call this Option 2)

Given that, the current draft goes with Option 2, figuring the Option 1 people can probably live with it. But maybe it's not that simple.

[connieimdialog]

hi, helping to move this discussion over to this space! I am one of those not sure about Option 2. In principle: i am more than fine with going with any working group definitions, understood internally, to facilitate common basis for discussion. super. but i think that option 2 might be unnecessarily complicated, and doesn't seem to work with other, larger, pre-existing definitions of indicators and could lead to more confusion.

• consider the notion of signal as opposed to noise, or the notion of indicators overall: so - my understanding has been that in some ways we don't actually know yet which factors are signals or indicators. In either case, something truly being a signal or indicator of credibility is something that needs to be established through data and evidence. What i have considered the indicator/signal VOCABULARY effort to be is a catalog of factors and the effort to isolate potential signals, potential indicators. we don't yet know if most of it is noise or noisy.

further here -- think about common parlance and clarity in terms of communication externally: when we talk about economic indicators/human rights indicators, we are not talking about every possible factor, we are talking about usually a set of ...factors that have been determined to show correlations with things that we care about in terms of growth/wellbeing

• mix of machine-human? i have a feeling that there will be a class of potential indicators/signals that will be both machine and human informed, so would be interested in considering a draft list of what is a signal versus an indicator. especially since some of this work in generating machine ability to do so rests on human classifications, perhaps supervised learning, etc.

but clearly there is a need so trying to understand it better: 1) instead, why not say "machine-generated indicator or signal" or "human-generated indicator or signal"; "hybrid indicator/signal?" in other words, what significant gain is there by classifying signals/indicator as relating to machines and humans? 2) can someone provide a short list of examples of what candidate indicators/signals would be? to make sure that machine-human classifications stay stable, etc.

in short, again, what i am asking is if this is introducing unnecessary complexity to an issue as i think it clashes with some ongoing conventions around those terms. happy to be persuaded though!

[jkranzler]

Hi, Jesse from TM here.

Just wanted to chime in with a few of my thoughts on the above. We're fairly agnostic on terminology over here (so take everything I say with a grain of salt), but as I see it indicators are the individual aspects of a site / publisher that lead to observations/outputs aka signals. So my opinion is similar to Sandro's, though I don't believe that signals are necessarily machine-driven. For example, an indicator might be that Politifact has identified a particular article on a site to be untrue. That indicator (along with detail around other articles on the site) could bubble up to a site-level signal about Politifact's overall fact-check percentage of a site (passed 85% of fact-checks, etc...)

I could be wrong in my assessment here, but I just figured any outside opinion might be helpful. With all that said, I would defer to Facebook, since I believe they have fairly nuanced definitions of these two. I've reached out to them to get their most recent definitions and will let you know when I hear back from them.

[jkranzler]

Just got additional clarification from FB:

Signals: data that platforms can use to make decisions behind the scenes Indicators: user-facing information that platforms can display directly to people in UI

[sandhawke]

Thanks @jkranzler...

That's clear enough, but what if there's no platform involved? Or what about features of the article that the platform simply passes through? Like, what do you call a cheap amateur layout, that a human simply sees and decides means a site is probably not credible. Is that a signal or an indicator? I was trying to extrapolate from Facebook's view -- where there is always a news feed in the loop -- but I'm not sure it remains coherent.

[connieimdialog]

Hi! i think i have a proposal that I'm going to work out in person with Mollie and others tomorrow hopefully.  :) More soon.

On 10/4/18 5:12 PM, Sandro Hawke wrote:

So what happens if there's no platform involved? What does you call a cheap amateur layout, that a human simply sees and decides means a site is probably not credible. Is that a signal or an indicator? I was trying to extrapolate from Facebook's view -- where there is always a news feed in the loop -- but I'm not sure it remains coherent.

[connieimdialog]

hiya! discussion with mollie and sandro here IRL has been productive. understanding the needs of platforms in concert with existing uses of terms like signal and indicator, we'd like a little longer to come back with some proposals for moving forward. so will do that and for now I believe @sandhawke will lift this section from the Community Group Report (who can confirm if i got that wrong!)

sandro, should we leave this open for the upcoming back-and-forth, or close? defer to you