Add activity feed to signed in landing page focused on people

[equivalentideas]

In a chat with @mlandauer a week ago, we proposed that the signed in landing page should support two main activities: finding out what's happening with the scrapers you're interested in and discovering new interesting data and scrapers to use and people to connect with/follow.

We decided that a feed of the latest updates about what's happening with your scrapers and other updates of interest could be a good way to support both these activities. By 'Activity feed' we mean something like the feed on GitHub's logged in homepage:

I think the first steps with this are working out what information appears in the feed that will be immediately useful to people and sketching out some ideas of how this could be displayed.

Does this cover what we took from our discussion @mlandauer ? It was a while ago now.

[equivalentideas]

What should go in the feed?

Here are some ideas:

• User creates scraper
• User forks scraper
• Scraper scraped 29 new rows of data from domain.com
• User downloaded data from scraper
• User watched scraper
• User watched User
• User fixed scraper
• User made 3 commits to scraper
• User manually ran scraper
• New user joined an organisation you watch

I think we'd want to start with a small number of simple types, then iteratively add new kinds of updates.

We want to try and make sure the items in this list are actually useful and relevant to users. There could be a number of filtering options for the feed, for example:

• activity of users and scrapers you are watching
• activity of users and scrapers owned by users who are near you (based on your github location)
• activity relating to topics you're interested in (would require some way to register what topics you were interested in and tag scrapers)
• all activity

Activity from things you are watching seems like a good first step, then the local based one seems the next most interesting. I don't think that that 'all activity' is very useful to users and the tags or interest based feed relies on a lot of data we don't currently have.

[mlandauer]

Some very initial thoughts/questions:

• The initial idea of what could go in the feed mostly consists of activity of a user or organisation. I think this is good. Why not for consistency keep it purely that way and remove "Scraper scraped 29 new rows of data from domain.com" from the list? Then, there's a very simple conceptual model for that list. It's stuff that people are doing.
• That list could potentially get very busy and noisy so I think the default view should only be to show the activity for users and organisations that you've specifically chosen and activity by random users on scrapers that you're watching. Otherwise you'll be shown activity of a user you don't know who they are for them doing something completely out of context. I think this could be extremely confusing.
• I don't think there should be a filter on the activity feed. Its main use is as a "background feed for serendipity" (what I mean is you won't actively go and look at it, it will be something that you see and get distracted by) and so it will just be there and should be useful. An other way of saying that is that the activity feed should be useful and meaningful. If the only way to get that to work is to add filters that probably means it will never work because I think it's unlikely that people would use a filter on an activity feed.
• Github has an activity feed which gets populated by the activity of users that you actively "follow". Why don't we add a "follow" action in morph that simply follows the user on github and we use the follow information from github to populate the activity list + we mix in activity from watched scrapers.
• Then, how do we make it clear to users the difference between "following" and "watching"? Right now, in morph we can "watch" scrapers and users/organisations. In Github, I think you can "watch" repos and "follow" users/organisations, which is nicer.
• For user actions on a scraper it would be easier to first implement that as extra information in the "scraper history" section of the scraper show page. There are some other tickets for those related ideas.

Anyway, just some things to think about and work out. @equivalentideas sketching some ideas for the activity feed will help.

[equivalentideas]

That's great @mlandauer I think you've outlined a nice simple approach to this.

I agree about the filters idea, I think I'm just getting excited about seeing activity by location, but that's a longer term sky-pie idea and this isn't the place for it :)

I think introducing the concept of following and doing it through github is a good idea—because you'd definitely want to follow people on morph you were following on github and it would be a drag to have to follow in both places.

The other nice thing about that is that you might just have joined morph, but you'll see a feed with activity from people your interested in.

For user actions on a scraper it would be easier to first implement that as extra information in the "scraper history" section of the scraper show page. There are some other tickets for those related ideas.

Yep that sounds good. Let's further refine the list of activity types after looking at some sketch ideas.

Then, there's a very simple conceptual model for that list. It's stuff that people are doing.

I really agree with this and that was the line I was following with some sketches. Really making this about the people by always having the avatar there. These were some ideas I did before that include a 'contributed to' but I agree we'd add that 'action on a scraper' stuff away for now.

What do you think about keeping a scraper activity one, but framing it as the action of a user:

I think new data added is a useful piece of information as it's a way to discover new data from your own scrapers even ( similar user need to #678 ).

[equivalentideas]

I'm doing some more sketches of what an activity item could be now.

[equivalentideas]

Here's a few more ideas for an activity item.

Firstly simplifying the text: 'user create scraper' instead of 'user create new scraper scraper':

Also in user lists we use the full name if they have one:

How could this all sit on the page?

We could have the feed take up the full width and leave whitespace to the side, or use that space as a sidebar and put something useful there (we could put the welcome new users block there):

Or we could make this a two column list:

I'm actually quite drawn to the two column idea, as there's more interesting activity that could catch your eye when you hit this page.

[equivalentideas]

Taking a step back to redescribed how this could work:

When you are signed in, on the landing page is a feed of activity from users you follow, so you can discover new scrapers, data and people that are interesting to you.

The users in the list are people you follow on Github. We will add a way to follow users within morph that syncs with Github.

The items in the feed are the actions users and organisations are taking:

• User created scraper
• User forked scraper (Not sure if we can do this one)
• User downloaded data from scraper
• User watched scraper
• User followed User
• User you follow joined morph.io

One thing to note is that you can't actually follow organisations on Github. That means you wouldn't see 'OpenAustralia Foundation downloaded data from planningalerts-scrapers/city-of-sydny'. We could automatically show activity by organisations you follow, but it does seem like confusing/dangerous territory to show stuff from users you haven't explicitly asked for as @mlandauer noted above. Hmmm.

Any thoughts on this @mlandauer this does seem like quite a bit of work to add the following feature, and then the bits and pieces required for the feed.

It does seem like it could be a big step forward for the social side of morph: connecting people and helping the learn about interesting new data/scrapers.

I do think by removing scraper activity we're not supporting peoples need to 'find out what's happening with the scrapers you're interested in' (from the issue description). Maybe there's a simple way we could reintroduce that, or maybe it's better served somewhere else, like your profile with something like #777 ?

[equivalentideas]

Rereading what you wrote above @mlandauer :

Then, how do we make it clear to users the difference between "following" and "watching"? Right now, in morph we can "watch" scrapers and users/organisations. In Github, I think you can "watch" repos and "follow" users/organisations, which is nicer.

I think this leads us down a path to have user stuff (following) and scraper stuff (watching) in separate places. I quite like that as it seems like a cleaner way to manage your scrapering/data. The activity feed can then be a place to find interesting stuff, and keeping an eye on scrapers (which is more a 'work' related function) can be done an effective way somewhere else.

Does that making sense? Relates to last par of comment above.

[mlandauer]

@equivalentideas working through all of that has been very helpful.

I think the idea of splitting the activity on scrapers you're watching from the activity of users you follow is a nice one. That you can't apparently "follow" organisations on github is I guess fine if the activity stream for those you follow is really confined to what real people are doing. So, no "organisation downloaded ..." because that's going be via an API, so not a real person. Also, no "scraper created 5 records" because that wasn't done by a person.

In terms of what feature is useful to which people - the activity feed for scrapers you're watching is more immediately useful because that supports people who are using morph.io for their own scrapers and it helps them get their job done.

The activity feed for users you're "following" is useful as the next stage. It's for people who are using morph.io now for their own scrapers and have invested enough in it that they are "following" people. Then, it allows them to expand their horizons and discover new things.

Neither of these features in their current incarnation are helpful to new users of morph.io because you won't be following anybody nor will they be watching any scrapers.

So, that does seem to me to make this feature a nice one but only really useful to very established users of morph.io and I don't think we're at the stage where those people (except maybe for us, oaf people) are limited by the functionality of morph.io.

So, I think we would be better off in the short term doing something like highlighting the "most active" users and/or highlighting the "most active" scrapers as a really crude place for morph.io newbies to poke around from.

Also, maybe it's time to help newbies more by writing some decent documentation ;-)

[equivalentideas]

So we're digging this idea of splitting up scraper and people activity. I think we break a scraper activity feed issue out of this and make this one about people, as they're two separate features supporting quite different needs. The people feed is about discovering interesting things and the scraper activity feed is about maintaining scrapers and using data.

So, that does seem to me to make this feature a nice one but only really useful to very established users of morph.io and I don't think we're at the stage where those people (except maybe for us, oaf people) are limited by the functionality of morph.io.

So, I think we would be better off in the short term doing something like highlighting the "most active" users and/or highlighting the "most active" scrapers as a really crude place for morph.io newbies to poke around from.

This 'most active' list could be the default that gets replaced as you follow people. Everyone gets activity from a small group, but once they are following enough people they only see stuff from people they follow.

Is that what you were thinking @mlandauer or a distinct separate place for most active people/scrapers (the explore page comes to mind as a place for that, something like #691 )?

The other thing to note is that many of the people signing up are following lots of people on Github (Also if you're someone very interested in scrapers and civic tech, you're probably following people who are too), so they could well have a few people with activity on morph already.

That's a kind of best case scenario though, you sign up and immediately see activity from people you know of, giving you a nice way in to explore and learn about scraping.

[equivalentideas]

^^^ made this issue about 'people' and split out the scraper activity feed into #779

[mlandauer]

I don't we need to work everything out at once. The important realisation for me was that we need to support the initial users of morph.io the most and for that a list of "most active" users is a useful place to start exploring.

Also, we shouldn't start by adding a feature that only supports people that are already following a lot of people on github some of who might be using morph.

What we do with the "active" users list later as we add more features (like following people on morph/github) is an open question and I don't think we should concern ourselves with that now.

So, let's just start with figuring out a nice way to show "active" users.