🍱 Interactive widgets

[juliushaertl]

Summary

Goal

Current link previews provide a simple, static preview but in lots of contexts it would be very helpful if users could work more interactively with the linked resource without navigating away to the actual app of it.

This is partly addressing a gap to competitor products like Notion, Microsoft Loop, Sharepoint when on the Collectives app, but also in other Nextcloud parts like Talk where team/discord can display more interactive widgets within the chat.

The goal is to add an additional way of rendering link preview widgets that provide more information, interactivity and possibly useful actions right away. The degree of what functions should be available for a widget may need further discussion on the individual apps.

Scope

• We provide a generic way that existing link preview widgets custom render function can be extended to provide an interactive way https://github.com/nextcloud-libraries/nextcloud-vue/issues/5090
• Using the interactive widget
  • Text/collectives will integrate using this interactive preview mode if manually inserted by the user https://github.com/nextcloud/text/issues/5270
  • Deck/notes will inherit this function from the text app automatically
• Providing an interactive widget
  • As per discussion with Jan we considered the following interactive widgets as the MVP, but need more specific design specs individually
  • Files/folder (e.g. pdf preview embedding)
  • calendar for shared calendars with month/day/week picker, single events https://github.com/nextcloud/calendar/issues/5701
  • deck full board, single card
  • tables full table exists, single rows, fully interactive https://github.com/nextcloud/tables/issues/803
  • youtube
  • spotify
  • maps
  • github

Out of scope

• Mobile app support

Tracking issues

• [x] https://github.com/nextcloud-libraries/nextcloud-vue/issues/5090
• [x] https://github.com/nextcloud/text/issues/5270
• [x] https://github.com/nextcloud/calendar/issues/5746
• [x] https://github.com/nextcloud/server/issues/42916
• [x] https://github.com/nextcloud/server/issues/42917
• [x] https://github.com/nextcloud/tables/issues/803
• [x] https://github.com/nextcloud/deck/pull/5630

Follow-ups

Not planned at the moment

• https://github.com/nextcloud/mail/issues/9253
• https://github.com/nextcloud/calendar/issues/5701
• https://github.com/nextcloud/contacts/issues/3790

## More detailed draft notes

- Deeper embedding of resources [old mockup](https://github.com/nextcloud-gmbh/office-team/assets/3404133/0ee749c5-7be2-4607-b154-5d2fbac2dd1c) - Add embedding (smart picker like) but not as link preview but real interactive widget - Should be based on the existing smart picker concept of inserting links - Should work across apps - For link preview providers (define what apps should implement this) - For displaying (e.g. talk or where else the link preview is used) - To clarify - How does the user or app decide if a simple or interactive preview is shown - How to handle this on mobile - Text/Collectives/Notes - Should this be different markdown syntax so users can decide based on the document usage what to display? - We got feedback that the link preview is sometimes not useful in its current approach of always rendering it if it is the only content of a paragraph - Reference: Tables already has an option like that to select the mode, but we should probably have a generic approach  Notion examples: ## Design spec I had a call with @jancborchardt to outline the implementation in more detail: - We should have different modes of link previews / widget renderings similar to Notion - Inline link (auto generate title and just show without a link preview) - Static link preview from open graph - Interactive widget if it makes sense - When inserting a link users should be able to select which option is used - default, preference should be depending on the link type but remember user preference - Should have a global setting for privacy opt out - Interactive link previews either show a collection or single item - Limit scope of interactivity to browsing, but some functions like create new entries still make sense - There need to be reasonable default sizing of widgets - Apps (bold as important/prio items) - **files/folder** - pdf preview embedding - possibly readonly for odt - photo/album - talk room/message - mail no preview https://github.com/nextcloud/mail/issues/9253 - contact maybe for shared addressbook https://github.com/nextcloud/contacts/issues/3790 - **calendar for shared calendars with month/day/week picker, single events** https://github.com/nextcloud/calendar/issues/5701 - single events like calendar quick view - full calendar embed - notes single note, possible category - collectives just simple preview - **deck full board, single card** - tasks single task, full task list - forms complete form, form results (interesting) - **tables full table exists, single rows, fully interactive** - external apps - **youtube** already embedded - **spotify** can be embedded - **maps** can be embedded - draw io - sketch picker - **github**/jira/openproject interactive list depending on the apps ## Implementation questions - Figure out how we can store different link preview formats in markdown for text @max-nextcloud - For displaying different link formats it could be beneficial to switch to a node instead of a mark - I did a quick PoC at https://github.com/nextcloud/text/compare/feat/link-node also covering auto generating the link title from link preview metadata - Link previews/smart picker @juliushaertl - Need a way to indicate that a widget or link preview supports interactive widgets - Rendering the widget needs to let integrating developers decide which one to show - Might be good to have https://github.com/nextcloud/text/issues/4024 for this

[jancborchardt]

Btw this is a bit of an advanced and fancy example of how Microsoft does it with their Loop components (thanks @marcoambrosini for sharing). Fully collaborative, including live cursor and avatars on top:

https://github.com/nextcloud/server/assets/925062/d4591e6b-f1e6-4dff-83e8-897ac92fd669

[juliushaertl]

Discussion with @max-nextcloud around markdown syntax:

We need three options:

inline link

[]()
[http://](http://)

regular preview

plain url as the only thing within a paragraph

https://nextcloud.com

interactive rendering

<https://nextcloud.com>

other options:

#<http>

@[max](mention://)

[test](https://example.com)<!-- -->

```preview

```

:::error
https://example.com
:::

[juliushaertl]

First draft for adding a flag for this to the reference rendering API https://github.com/nextcloud-libraries/nextcloud-vue/compare/feat/noid/interactive-widget

Tables app implementing it: https://github.com/nextcloud/tables/pull/new/feat/noid/interactive-widget

Still to figure out:

• Do we need size recommendations, restrictions (might be good if talk can enforce a maximum height but collectives could just render depending on required height)
• Do we need a generic way to pick and return different types?

[juliushaertl]

Main parts are done, remaining issues are tracked separately.