Version difference extraction

juliushaertl commented 1 year ago

When trying to summarize changes that happened since a file was last viewed or to compare the difference of versions in a overview we would need a way to extract the difference between two files in some structured way.

When brainstorming with @AndyScherzinger and @marcoambrosini today we came up with the following idea:

Generate a simple diff of relevant changes in a CRUD approach, e.g.
- Paragraph added/removed/updated with content
- Image inserted/removed
- Table added/updated/removed
- List added/updated/removed
To give the change more context we could extract the preceding heading as the section the change refers to
This way we could generate a changelog style output for any given two documents

As a first step we'd need to:

[ ] experiment with possible tools we could use to generate the diff
[ ] remove or group possibly irrelevant changes like single character/word changes
[ ] figure out a way to find matching section headings and connect them
[ ] try to represent them in structured data

If this works out fine we can align with the design team on how to further make use of this, some possible candidates:

Summary in collectives for changes since the page or collective was last viewed
- https://github.com/nextcloud/collectives/issues/311
- https://github.com/nextcloud/collectives/issues/599
Comparison view for versions in files
Maybe other places like activity or summary emails
This could be a first candidate for formulate input prompts for possible summaries that @marcelklehr is looking into

For implementation we may want approach it in a flexible API approach so that apps could provide their own extractors for file types which could be text for markdown and a different implementation in richdocuments for office files, though that is something to still evaluate in further detail after the first feasibility exploration.

juliushaertl commented 1 year ago

We probably could use https://commonmark.thephpleague.com/2.4/customization/abstract-syntax-tree/ and try to walk over the output from two versions to generate some sort of diff.

juliushaertl commented 1 year ago

https://github.com/swaggest/json-diff might be useful for this

juliushaertl commented 1 year ago

https://codepen.io/benbowes/pen/pVaMpv could be interesting to show changes inline in the version preview of text.

Have tiptap read only view implement ins/del tags
Pass the html diff of two markdown-it results to htmldiff and then to tiptap to have inline change highlight

Doesn't seem to be actively maintained though https://github.com/dfoverdx/htmldiff-js

juliushaertl commented 10 months ago

Summarizing different approaches that we've checked:

Generate HTML diff in the backend

Generate HTML diff in the frontend

https://github.com/dfoverdx/htmldiff-js since 2019
https://github.com/GitbookIO/react-rich-diff 7 years ago
https://github.com/sethtippetts/rich-text-diff 7 years ago
https://github.com/nono/mddiff 9 years ago experimental

Generating structured diff data

Use tiptap to display changes

https://github.com/nextcloud/text/issues/4805#issuecomment-1787641301 We would have the option to enable this after loading the document which could also be used to show differences when using setContent. This could be useful for showing differences with #4422 but setContent replaces the full document so we would need to calculate a set of transactions to get from state A to state B to apply only that and have actually only the changed parts as transactions
- Would need further research for that

nextcloud / text