PostHog / posthog

🦔 PostHog provides open-source product analytics, session recording, feature flagging and A/B testing that you can self-host.
https://posthog.com
Other
21.62k stars 1.29k forks source link

Make saved reports a top-level feature #3408

Closed macobo closed 2 years ago

macobo commented 3 years ago

Is your feature request related to a problem?

Reports are useful for saving ad-hoc analysis. I want to:

We have a concept of saved reports already. You can access them by:

  1. Going to insights
  2. Clicking history
  3. (Optional) Giving a unnamed insight a name
  4. Clicking on "saved" tab

Describe the solution you'd like

Make reports a more top-level thing with:

  1. Default tab to "saved" insights, history being a different tab
  2. Provide tooling to manage reports - e.g. filtering by name, creator, when created, starring, etc.
  3. Allow adding reports directly from dashboards - e.g. by having a (+) button which when opened allows to filter saved reports.
  4. Add a way to open recent reports of same type to insights. Move all "saved funnels" to reports.

How filtering is handled by Heap

2021-02-19_17-33

Describe alternatives you've considered

Only using dashboards - makes it harder to reuse reports in dashboards, bad for discovery.

How mixp@nel does it:

At top of insights there are New, Save, Save as new buttons.

2021-02-19_17-36

Clicking open opens a modal for filtering these reports:

2021-02-19_17-37

Additional context

There's a downside to this: we'd be cluttering our side nav even more with "related" functionality.

cc @paolodamico @EDsCODE

Thank you for your feature request – we love each and every one!

paolodamico commented 3 years ago

Here's an initial stab at introducing saved insights as main functionality.

1. Saved insights Main way to access saved insights would be from the home page (if we keep it; alternatively this would just be a top-level menu item, in which case we would probably bundle other saved stuff).

2. Quick access Same as we have on dashboards, we allow users to quickly navigate between saved insights.

3. Insight history This would keep only the most recent analyses for easy recovery. We would default to filtering by analyses that each user ran themselves as it will probably be the most recurring use case.

paolodamico commented 3 years ago

Some mockups for the missing save insights experience.

Saving an existing insight

  1. We ask for confirmation because just clicking save would completely override previous configurations with no easy undo. saving

Saving new insight

save

Saved insight

saved
clarkus commented 3 years ago

I think this makes a lot of sense. There is a related case where work gets interrupted - sometimes you just need to put work down and come back to it later. The ability to save via a dedicated workflow would inspire some trust of the feature and add value to insights outside of dashboards.

clarkus commented 3 years ago

Transitioning here from https://github.com/PostHog/posthog/issues/4979

I've put together updated listing screens for insights. This includes some basic filtering and a new primary action for triggering the insight creation workflow.

All Insights-1 All Insights

Part of making this workflow feel more fluid / simple will be setting smart defaults for all required fields. In the scenario below, the insight has a meaningful, identifiable name that can be searched for later. This makes our save action immediate. Given this, we might consider a toast-style notification that informs a user of non-critical events like "insight saved successfully". When the user opts to save and add to a dashboard, we show the existing modular workflow for adding an insight to a dashboard.

Screen Shot 2021-07-14 at 4 16 44 PM Screen Shot 2021-07-14 at 4 11 16 PM

One outstanding thing to resolve is making starting the insight creation workflow a prominent and discoverable action. We want to make this list available for finding existing stuff, but we need to solve for that immediate "I need to explore my data now" task. This could be a big prominent action in the global navigation. Thoughts?

liyiy commented 3 years ago

I thought the all tab here

Screen Shot 2021-07-15 at 4 31 00 PM

would render us this current view below that options bar, but is it all insights saved instead?

Screen Shot 2021-07-15 at 4 30 16 PM

I thought having an extra options bar on top was a great way of adding saved insights discoverability without removing current visibility of our easy to access graphs. Is that new insight button meant to remove our current all graph types display and direct us to each one individually instead or is it just additional functionality?

clarkus commented 3 years ago

would render us this current view below that options bar, but is it all insights saved instead?

Yeah the intent is that this page primarily lists all saved insights into the categories all / yours / favorites / updated recently. Secondarily there will be a listing of insight history. This should be a distinct list.

Is that new insight button meant to remove our current all graph types display and direct us to each one individually instead or is it just additional functionality?

This would be a way to trigger insight creation. Right now when we go to insights, we're always in a mode where we're defining or creating something regardless of whether that work is saved. This change would create a distinct view for lists of insights with controls for making things easier to find. The existing insight view would be unchanged with the exception of funnels which has a "funnels saved to project" list. That would be deprecated and replaced by the insight list view.

liyiy commented 3 years ago

In terms of first insight creation workflow, I do like having all the insights always in a mode of creation/display 🤔 It makes it feel more intuitive to create something since it's all laid out visually for the user already. Adding a dropdown button in order to reach that page feels like we're adding another step to the process and obscuring a key discoverability moment? (when users just see all the charts and go at it, even if they don't quite know what they're looking at just yet).

Some of this saved insights stuff might be a good fit for the home page too, where we're already displaying some things like it?

But, curious to see what others think 😄

clarkus commented 3 years ago

@liyiy the idea is that you can click the button to trigger the creation flow as it is today, or you could click into the menu to jump to a specific insight type that you want to build. I'm not 100% happy with it either, so I'm chasing that as part of this goal:

One outstanding thing to resolve is making starting the insight creation workflow a prominent and discoverable action. We want to make this list available for finding existing stuff, but we need to solve for that immediate "I need to explore my data now" task. This could be a big prominent action in the global navigation. Thoughts?

Appreciate your feedback!

clarkus commented 3 years ago

I spent more time on the listing view. I introduce the concept of types into filtering. This is communicated via iconography now. Note that I'm trying some different icons here that aren't final. Let me know if you have thoughts on those.

We have enough inline actions here to justify an overflow menu. Here I've added an "add to dashboard" action that would trigger adding this insight to a dashboard. Next I am going to work on bulk actions that would allow you to remove multiple saved insights or add multiple insights to a single dashboard. There might be a tag editing case in there as well, but I'm still not sure tags make sense at the insight level or not.

Screen Shot 2021-07-16 at 12 18 02 PM
clarkus commented 3 years ago

One other note while I'm thinking about it... our nomenclature for insights is a bit all over the place as it relates to dashboards. In various places insights are referred to as "graphs", "insights", or the insight type "funnel" for example. We're also using the word "reports" while discussing internally. So my question is what do we want to call insights? :) /cc @macobo @paolodamico

timgl commented 3 years ago

I was trying to go into this direction with the history tab so definitely in favour of this. I am quite fond of the way I brought in the actual graphs as a way of super quickly identifying Insights, as 99% of them will be unsaved and thus have no description attached to them.

image

clarkus commented 3 years ago

@timgl I think that makes sense when you're trying to identify previous work that isn't identifiable via other attributes. I could see using that in a card layout for the insight list in the proposed changes above. When other attributes are available (name, desc, tags, author, etc) I think the table could offer some value. It really depends on context and how the user is identifying the insight.

One thing that is tripping me up is our different models between dashboards and insights. I think the core difference is the immediacy in which I am in that creation flow for insights. I click on insights and I see the creation / query view. I click on dashboards and I see a list of dashboards. The changes above propose we align insights with dashboards (I click insights and I see my list of insights).

We can still ensure quick and immediate access to creating an insight, but it would be triggered via something other than the insights item in the menu. Maybe there is a big prominent "new insight" button for the quick exploration case?

Let me know if this doesn't make sense... it feels really subtle and nuanced.

marcushyett-ph commented 3 years ago

I like this approach a lot - I actually wonder if it makes sense for this to replace or be part of "home" as I imagine its the first place most people will want to go? cc: @kpthatsme

As @timgl says the graphs are really helpful to understand quite a lot about the insight without digging - I think it'd be okay to show them on hover or something.

With the new insights dropdown for some reason it feels quite hard to differentiate between them with only the name - e.g. Whats the difference between a trend and perhaps, how retention is trending over time? Maybe its just a naming thing - but this potential confusion seems more pronounced in this experience.

clarkus commented 3 years ago

With the new insights dropdown for some reason it feels quite hard to differentiate between them with only the name - e.g. Whats the difference between a trend and perhaps, how retention is trending over time? Maybe its just a naming thing - but this potential confusion seems more pronounced in this experience.

Agree. I think secondary descriptions are going to be the best solution for making this more obvious. I'll incorporate that into these designs and post back here.

clarkus commented 3 years ago

Quick update to include insight descriptions and a card view option.

All Insights-1 All Insights

kpthatsme commented 3 years ago

@clarkus I really love this direction and this new Insights page and is almost exactly what I was hoping next steps of Project Home can lean into. I think the card view is super valuable as a big draw of this kind of page is being able to grab some data without clicking into the chart.

One idea I've had, which is more related to general nav, is the notion of something like an ever present "+ New" button in our UI that is the entry point to all content creation in PostHog. (we can discuss some of this tomorrow in our zoom!)

Clicking this button would open a quick little menu showing all the things that can be created in PostHog (I.e. Insight, feature flag, plugin pipeline, dashboard). I think an alternative of this could be for each of the sub-products mentioned, each has it's own +New button when you're on the actual sub-nav page – I think this is what you're going for here?

clarkus commented 3 years ago

One idea I've had, which is more related to general nav, is the notion of something like an ever present "+ New" button in our UI that is the entry point to all content creation in PostHog. (we can discuss some of this tomorrow in our zoom!)

Clicking this button would open a quick little menu showing all the things that can be created in PostHog (I.e. Insight, feature flag, plugin pipeline, dashboard). I think an alternative of this could be for each of the sub-products mentioned, each has it's own +New button when you're on the actual sub-nav page – I think this is what you're going for here?

I think that would be worth trying. I was working on some broader IA changes for #4268 and came up with the wireframe below. It has a creation actions inline where applicable. These could be added for other areas like plugins. I think that prominent creation makes a ton of sense, especially for insights. I could see adding in other creation options, but would probably want to make them lower priority than those core analysis actions.

settings-ia

clarkus commented 3 years ago

These are my most recent concepts for how we could rework the information architecture when creating an insight. The changes are mostly focused on clarifying the states for insights - editing/creating and reading. I've tried a lot of ideas for how we might better balance the query builder "configurator" and the chart, but I'm hesitant to make large layout changes without some testing.

Building a query or insight You would only see this query builder when building an insight or editing an existing insight. Note that building an insight does not imply saving an insight. I've placed prominent actions adjacent to the insight name for saving, adding to a dashboard, resetting the query and eventually more. The query builder shown is largely the same as we have today with some hierarchy changes and additional inline actions. Not detailed here, but there's way more work happening in #2760 that will show up here. Depending on the insight type, we'll have a variety of options in the right column that add functionality to the query.

Below the builder we see a full-width visualization of the insight. This would allow us to align insight layouts across types and probably reuse some components.

edit-insight

Insight detail view This view hides all the UI for editing. This page describes the insight, provides actions for adding to a dashboard or editing, and then shows the insight in full detail. Notably, insights contain more detail than dashboards. With this change a user could see something interesting on a dashboard, click through to the insight detail and do deeper exploration of the data.

All Insights

clarkus commented 3 years ago

@paolodamico do you think we should test this concept before implementing it? I can see the value this will add, but I'm still concerned about the workflow changes for creating insights. It could be a non-issue, but I am wondering if some quick testing could inform the direction for these changes.

marcushyett-ph commented 3 years ago

@clarkus I'll have a customer call today and will take them through it for some very initial feedback.

liyiy commented 3 years ago

I'm going to start on this, with the all/your insights page first. I think I'll also break it up into its own issue here :-) so we can iterate on it there if anything. Any thoughts, comments, suggestions?

paolodamico commented 3 years ago

Sorry for the delayed feedback here.

clarkus commented 3 years ago

I wonder if the tabs are the best way to filter insights. I see for instance some potential confusion when you click on "Your insights" with the created by filter. Maybe these should be preset filters?

I don't think tabs are critical. The critical thing is being able to find your own work. Secondary to that, finding pinned or favorite items would be valuable. Maybe just adding a filter for showing only favorites would be enough to drop tabs and move towards simple form controls for filtering.

I'm still hesitant of putting insight building behind an additional button. Maybe the new navigation designs can help, but in the meantime maybe we add a separate button to the navigation? Or called this view Home or Saved?

I am concerned about this as well, and think we should continue to test it. If we build the listing view, it really should align with the model we use for dashboards. It's established, familiar, and I think users are going to expect insight creation / saving to work the same way. I was exploring some secondary creation actions that could get us around this issue and that is when I got into the IA work at https://github.com/PostHog/posthog/issues/5346.

Already provided this feedback in a separate issue, but the build view is overwhelming. At least for now, I would keep it as it is, while we keep improving the new design.

+1

Another thing that I'm still unclear about is the whole saving insights experience, particularly saving existing insights (how would overwriting work vs. saving as a new insight). How about the "Save" vs. "Save & add to dashboard" buttons, as they're opening the same modal. Think it'll be key to really nail down the UX of this part.

I will put together an editing scenario to clarify this.

paolodamico commented 3 years ago

Perfect, sounds good on everything! Keep us posted on the insight saving process to share any feedback with you.

Re the insight creation process, I think your argument makes sense, my gut is that this will affect insights discovered and we'll have to balance the trade-off of how many insights are actually being saved (and used), but let's test out! We just have to make sure we have a solid way to measure the impact of this change. @liyiy let me know if I can help with this.

clarkus commented 3 years ago

After thinking through it more, I'm not totally user we need to do much to differentiate editing from creating. I've focused these illustrations just on the header as this contains the insight details we need to capture in order to make insights findable.

In the first header example, this is the initial state after triggering the insight creation flow. You can see default values everywhere. I've changed the button actions to "create" instead of "save" but I really don't think it's a critical change. In the second example, I am showing an edit / create state. There are non-defaults applied and there are prominently placed elements that reinforce the insight is in an unsaved state. The third header example shows the "read-only" view for insights. There are no save actions, just adding to dashboard.

Screen Shot 2021-08-03 at 2 04 38 PM

One thing to clarify with these changes is that in almost every case the save action should be immediate and pretty transparent. If we provide default values for required fields (insight name) and maintain unique IDs for each insight, it should be pretty straightforward to hit save once you have a valid query defined.

The action "save and add to dashboard" performs a save and then just triggers the add to dashboard modal. It's just an easy way to ensure the user saves their work before adding to the dashboard or updating existing dashboards. Also, we're making big changes to how users navigate to insights. That process of navigating to an insight will also reinforce the state of the insight. For example, I will have a pretty clear mental model if I click "edit" from a dashboard or on an insight detail page.

paolodamico commented 3 years ago

Feels intuitive, but then what if I want to save my changes but not overwriting the current insight but as a new insight?

Another pitfall I see is what happens if I accidentally save and overwrite something important? Think we might need to introduce an undo here so that the user doesn't have to go dumpster diving on the history.

One final thing that comes to mind is making sure the saving process is fully usable with the keyboard so it feels even faster, starting with using hotkeys to save. Thoughts?

clarkus commented 3 years ago

Feels intuitive, but then what if I want to save my changes but not overwriting the current insight but as a new insight?

We could introduce a save-as option, but there's also a duplication option from the listing views. A user could duplicate and then just save as a new item.

Another pitfall I see is what happens if I accidentally save and overwrite something important? Think we might need to introduce an undo here so that the user doesn't have to go dumpster diving on the history.

An undo or a revision history specific to the insight could help here. I think part of the value of these changes is that you'll have more confidence you're editing the right thing and not overwriting something unintentionally. I see the concerns, and while they're related, they're somewhat of an extension of this problem. Maybe we should see if it becomes a problem and file a follow up issue for revision histories?

One final thing that comes to mind is making sure the saving process is fully usable with the keyboard so it feels even faster, starting with using hotkeys to save. Thoughts?

I have divisive thoughts on hotkeys. I've been trying to better understand our implementation of hotkeys, and while I see their value, I feel like our current implementation is a bit heavy-handed in terms of communicating shortcuts. There are also somewhat massive accessibility concerns with hot keys. Generally, I think hot keys (or access keys) should be tied unreserved keys. I know we push for an app-like experience in the product, but it is still very much a browser with keyboard navigation features built into it. Generally, we should rely on those. They're the common context across sites because they're browser defaults.

Hotkeys or access keys can be helpful, but they shouldn't conflict with the default browser keyboard navigation methods, and they shouldn't be biased over other means of navigating (mouse cursor). We can still ensure it feels fast, but I think there's a fine line to walk with keyboards shortcuts and keybindings. I am still forming my opinion on all this as it relates to the PostHog app, so I'm not really ready to propose a different solution. I just know that the problem can be really complex depending on the context. Anyway, yeah we can make it feel fast :)

paolodamico commented 3 years ago
clarkus commented 3 years ago

Agreed, I think a save as option could be helpful. We could have an extra context menu like we just introduced for auto-refresh.

Here's a quick idea that rolls the "save as new insight" and "save and add to dashboard" actions into a submenu under the save action. I'm a bit concerned about discoverability for adding to dashboards, but at the same time I don't want a menu of a single option here. Thoughts? Are there any other related actions we might consider exposing while editing insights?

saveas

paolodamico commented 3 years ago

I think this works because even if you click Save only, you'll be able to add to dashboards from the modal, right?

clarkus commented 3 years ago

The modal would only fire when adding to a dashboard. Otherwise saves are immediate. This is based on having a default value for the insight name... in most cases I was just using "my insight " + a unique ID. That would give you something uniquely identifiable (as long as unique names are enforced or we have some other unique ID exposed). Descriptions and tags are optional, so as long as we have that name value saves should be immediate.

A general workflow could look like this:

Throughout saves you'd see toast notifications reinforcing the successful save or creation of new insights. Thoughts?

paolodamico commented 3 years ago

Oh okay! Cool indeed, makes saving even more seamless. I think what we could do then is: a) if it's the first time saving this insight, we have both buttons (as save as doesn't make too much sense in this context), b) if it's NOT the first time, we do the single button with the dropdown. Adding to dashboard discoverability is less important in this case. Wdyt?

clarkus commented 3 years ago

That sounds like a good approach to me.

clarkus commented 3 years ago

I reorganized and posted the latest designs to the PostHog App project in figma - https://www.figma.com/file/gQBj9YnNgD8YW4nBwCVLZf/PostHog-App?node-id=3116%3A34102

clarkus commented 3 years ago

Quick update to clarify the editing state for meta details like descriptions, tags, and names. https://www.figma.com/file/gQBj9YnNgD8YW4nBwCVLZf/PostHog-App?node-id=3245%3A33481

Previously the user would need to toggle the editing state for names and descriptions regardless of the insight page mode. If you're editing the insight, these fields should just be directly editable and saved along with the rest of the insight definition.

Screen Shot 2021-09-13 at 10 02 49 AM
kpthatsme commented 3 years ago

I really like what you have here @clarkus – I think it would add a ton of value to activation, it's basically a much better version of project home.

@paolodamico how far out do you think this is for you all to work on? (Just the saved insights page)

paolodamico commented 3 years ago

@alexkim205 is actually getting this over the line during the next sprint. @alexkim205 would we be able to include the explore tab here too?

kpthatsme commented 3 years ago

Oh that's super exciting– I think this will really help with activation.

alexkim205 commented 3 years ago

Actually I think @liyiy actually merged in her saved insights work today. It should be behind a FF for @posthog people and it's really cool. I'm picking up remaining work for saved insights though so feel free to play around with it and assign me to any bugs you might find

Twixes commented 2 years ago

I think we can consider this done, saved insights having been released now.

posthog-contributions-bot[bot] commented 2 years ago

This issue has 4509 words at 40 comments. Issues this long are hard to read or contribute to, and tend to take very long to reach a conclusion. Instead, why not:

  1. Write some code and submit a pull request! Code wins arguments
  2. Have a sync meeting to reach a conclusion
  3. Create a Request for Comments and submit a PR with it to the meta repo or product internal repo

Is this issue intended to be sprawling? Consider adding label epic or sprint to indicate this.