androb commented 7 years ago

Implement a block for footnotes.

Prior pull requests: #2818

swissspidy commented 7 years ago

Interesting suggestion, thanks! Have you had anything in mind how this could be resolved UI-wise?

I mean, displaying footnotes is one thing. But you also need to add them to words/sentences/paragraphs somehow in the first place.

androb commented 7 years ago

It is a good question. It is an "insert inline at cursor" action which I do not believe we have modeled before.

Under the current UX regime, I believe it would belong alongside link on the inline toolbar that pops up over each block... but it doesn't feel like a commonly enough used function to occupy such prime real estate.

May be we keep it in the Insert drop-down but it behaves somewhat differently?

Perhaps @annaephox has some thoughts?

karmatosed commented 7 years ago

I really like this idea and think a lot of users would want this. I do wonder if it's a block or a plugin that goes with Gutenberg though. Worth exploring either way.

BoardJames commented 7 years ago

The tricky bit with footnotes is that they need to be coordinated across block boundaries. In my mind footnotes need 3 things:

Link to some external resource
Short description about the external resource (could be pulled from the external resource directly using the same techniques as for embeds)
Knowledge of their position in the overall document so they can be numbered.

If the editor has control over the whole document then they are easy. Just allow users to insert a link with a "footnote" toggle button that applies a style or a data attribute. Then before save of the entire document, you process the document to find all of the link tags that have the appropriate style or data attribute. You move the links to the footnote block of the document and then replace them in the edited block with a link to the entry in the footnote. On edit you do the reverse, you find all the footnote locations and reinsert the original link.

ellatrix commented 7 years ago

I'm pretty excited about footnotes. I think it should be supported in core, as it's something that really needs to be baked in and it's quite common for people to use.

Adding

We probably don't want to be adding too much UI when you're not using them. We'll need a button in the formatting toolbar for sure, but maybe we can introduce a "Footnotes" block, which is inserted at the bottom, and only with this block present the button in the formatting toolbar pops up.
There should be a shortcut. Probably [1] etc. (order should be autocorrected) and using an existing number links to an existing footnote. [#] (or any not yet existing number) to create a new item without having to know the correct number.
It would be nice if, once you click the button or complete the shortcut, a field slides in at the button that auto-focusses (you can continue typing from the shortcut), enter submits it and puts the caret back after the reference mark.

Editing

The reference mark [1] looks like a superscripted link, and when the caret goes in the boundaries, the footnote field slides in again at the bottom.
You can also go to the bottom and edit the footnotes in the footnote block. The reference mark links to the footnote, and the footnote links to the reference mark.

Deleting

I think for sanity, it should only be possible to delete the reference marks (which deletes the footnote). Of course you should be able to bring it back with undo.

At a state/reducer level, insertion and delete (or only post save if we don't want numbers/order in edit mode) will both have to look through all editable content to determine order.

ellatrix commented 7 years ago

Actually, we don't even have to loop through all editable areas, only look at the editable area the current one is in and sort. We could add the footnotes as properties to each block, and the footnotes block could put these together based on the block order.

ellatrix commented 7 years ago

Going to assign to look at in the near future.

aaronjorbin commented 7 years ago

I love footnotes, but I think this might be something that belongs as a property to the post, rather than as a block. Two thoughts:

1) How do we ensure if it is a block, that it can be locked to the bottom of the post (i.e. that there is never a case when footnotes are in the middle of a post) 2) If you are using multiple pages ( via  or whatever form that has taken in Gutenberg) in a post, that the footnotes appear on each page and are numbered correctly for that page.

dsifford commented 7 years ago

@iseulde I'd be happy to work with you on this.

Check out https://wordpress.org/plugins/academic-bloggers-toolkit/. I've been working on a pretty good academic citations plugin over the last 3 years (although, I am biased!).

There's quite a few hidden difficulties that come with managing references and footnotes that I'd imagine most people don't consider early on (I was one of those people). I'd be happy to discuss this in detail if this is something you'd be interested in pursuing.

Pushing a rather large codebase update to wordpress.org in the next week or two which has quite a bit of refactors necessary for gutenberg.

karmatosed commented 7 years ago

This is just a v1 but the basic design could look like this:

A simple option is added to the toolbar. We will have to iterate the graphic but after doing some research this is a good starting iconography for footnotes.

footnotes-1

Following on from creating a footnote, the footnote block would appear. This has no options, we can discuss if it should have options. I'm cautious this can be deleted even, if you have footnotes why would you delete? Isn't just remove a footnote the delete?

footnotes-2

Perhaps people would want a title, but for now this shows the minimum I think footnotes would make sense. I'm very cautious we do not spiral this feature, it could easily spiral.

jasmussen commented 7 years ago

Love those mockups, Tammie, I think they provide a solid basis and can help our minds grok the side effects and everything. The "deleting" thing is one issue that immediately pops up, but that seems solvable by looking at how other editors have done it — perhaps a "are you sure" prompt is sufficient.

The icon also is almost there, with a popout label I'm sure people will get it.

But one thing I keep coming back to is — it's very very clear that we need to be able to both add more buttons to the quick toolbar, and stay responsive.

This is how Google Docs does it:

more

This is how it could look for us:

quick toolbar alt

RayWaldo commented 6 years ago

I really NEED footnotes for my writing.

Also, they are NOT always related to external sources (that is normally a bibliography or end-notes). Often, writers use footnotes to provide additional information about a subject that is not necessary for the primary content but may be helpful to some readers. Please do not force them into ONLY using external sources.

dsifford commented 6 years ago

@RayWaldo When the gutenberg API stabilizes just a bit more (and also when I have the time) I plan on converting my plugin to gutenberg and include first-class footnote support. Can't give a time-frame for when that will precisely happen though.

johanneswilm commented 6 years ago

@dsifford Great work with your plugin!

I am working on Fidus Writer and I'm trying to make sure that copy-and-pasting from Fidus Writer to Wordpress works well. I noticed that it's really hard to make the kinds of internal links for footnotes you are currently producing, @dsifford, as <div>s are not allowed and <p>s are not allowed to have ids. So the only way one can make footnotes is by making them headlines - something like <h6>. That will look ok or not depending on the styling of <h6>.

The advantage of dsifford's work is really that the footnotes continue to work even if one uninstalls his plugin because he stops maintaining it, etc. . Yet that advantage goes away with Gutenberg, as far as I can tell.

One solution could be to allow id-attributes on <p>. There could be other solutions.

aaronjorbin commented 5 years ago

@mtias I put future back on this. If there is a different target for this, can you please move it to the appropriate one.

mtias commented 5 years ago

"Future" is correct, I misclicked while managing the Bonus Features so we could close it.

jcklpe commented 5 years ago

Just wanted to throw in my two cents, that I'd love to see this implemented, or at least something in the Gutenberg API (similar to how one can create one's own custom inline styles using the new Rich Text API) that makes it possible for individual theme/block developers to implement themselves.

I've been meaning for awhile to create a footnote feature in my personal blog theme using ACF or something of that sort, but I'm fully bought in on Gutenberg and found this page when trying to look up how I could make my own gutenberg compatible inline footnotes etc.

Also I'll echo previous sentiments that I don't think of footnotes as just for external sources. In fact, I would really like to implement my own footnotes solely as a means of adding extra information to or next to a piece of text. This could be a link externally, or a bit of extra info, or perhaps even an image or a video! When taken in it's most flexible and generalizable form, I think it can be thought of as being sort of like a hyperlink that triggers a modal container.

Could something be built off it with the Rich Text API? Like if the hyperlink is now using the Rich Text API, then a footnote can be thought of as "a hyperlink whose scope is to content already inside the page" essentially, right? I hope that makes sense and isn't just gibberish.

EDIT: I think you can kinda add ID elements on inline stuff using the Rich Text API @johanneswilm

dsifford commented 5 years ago

@jcklpe Working on this as we speak for my plugin. Citations are being prioritized, but footnotes should be done shortly after. It's looking like it should be done sometime early January if all goes well (doing this in my spare time).

@johanneswilm Sorry I missed your message before. RichText allows for custom data- attributes, which is what I'm going with for my implementation. Feel free to shoot me an email if you want to talk more on this.

ettoredn commented 5 years ago

+1000 for this to be implemented. Gutenberg has definitely been too much hyped for not having this feature implemented and field tested by WordPress 5.0.

jcklpe commented 5 years ago

@jcklpe Working on this as we speak for my plugin. Citations are being prioritized, but footnotes should be done shortly after. It's looking like it should be done sometime early January if all goes well (doing this in my spare time).

@johanneswilm Sorry I missed your message before. RichText allows for custom data- attributes, which is what I'm going with for my implementation. Feel free to shoot me an email if you want to talk more on this.

Hey I've also been thinking about diving into making my own custom Rich Text stuff for gutenberg. I just finished a mini course on building custom blocks. Seems pretty easy so far but I have no clue how to get started for Rich Text. I don't even really know where the documentation for it is? You have any ideas or suggestions @dsifford ?

Like I know how to include a richtext component into a new gutenberg block, but I thought there was specifically a new part of the RichText API for registering new inline styles that are accessible from any rich text component enabled block, right?

johanneswilm commented 5 years ago

@dsifford I don't think I can find your email anywhere. It's been a while now, but if I remember correctly, I found out that Gutenberg would strip out the kind of data- attributes you were using in your plugin, hence making it incompatible. At least I could not find any way to paste that kind of content into gutenberg and not have it be stripped out. But this may no longer be up to date, or it may only apply to paste-handling. Looking forward to the work of @jcklpe .

dsifford commented 5 years ago

@jcklpe I'm about ~30-40% done with the conversion of my plugin so you can check there for examples. You're correct that the docs are scant at the moment, so I had to just read through the Gutenberg source to figure it out. They've got a few open issues now though here to improve them so eventually they'll get better. PS: I'm in the wordpress slack group, so you can just message me there with questions and I'll try to do my best to help.

@johanneswilm That may have been the case in earlier iterations. But now you can register attributes and they'll carry through. I'm pretty sure that you still won't be able to just directly paste the content in though.... gotta do it using the APIs.

For the other lurkers: Plugin should absolutely be beta-ready in about a week or two, and should be release-ready absolutely by January.

johanneswilm commented 5 years ago

Ok, yeah so not having copy/paste access is somewhat problematic as that continues to be the #1 easiest method for 99.9% of users. Power users with extensive programming experience will be able to set up API access from some other site, etc. .

Just think about what you would do if a random friend asks you to just help them quickly transfer a document they have written in Google Docs or some other editor to make a new blog post in their Wordpress site. My guess is you'll try to copy-and-paste and then spend a few minutes cleaning the result up, rather than try hunting down plugins for Google Docs and Wordpress that enable API access and add menus to transfer the document in some other way.

dsifford commented 5 years ago

@johanneswilm There's gotta be sanity and security checks before allowing anybody to just arbitrarily paste in HTML, so they're doing the right thing.

If you needed to add copy and paste raw HTML functionality to the editor, you can do that pretty easily by adding in a modal dialog that accepts HTML in a textarea form. When the user clicks the submit button, the plugin can just sanitizes the potential bad stuff first, and then insert it programatically using the API.

But with all that said, I think it's probably possible just to copy and paste raw HTML into the "Code editor" of Gutenberg. If the text format was registered correctly, it should just pick up on that. Similar to the old TinyMCE editor.

johanneswilm commented 5 years ago

There's gotta be sanity and security checks before allowing anybody to just arbitrarily paste in HTML, so they're doing the right thing.

Sure. But there is no reason this should go as far as making it impossible to copy and paste footnotes. That script tags are being removed, inline CSS, etc. goes without saying. There should be a way for plugins to have a say in what is being stripped out.

If you needed to add copy and paste raw HTML functionality to the editor, you can do that pretty easily by adding in a modal dialog that accepts HTML in a textarea form. [...]

Sure. But that's similar to a car engineer telling a normal person "Wheelchair passengers cannot get in through the door. But if you remove one of the windows and use a crane, you can get the wheelchair in like that." For the car engineer who has a crane standing around at home and has replaced a million windows this seems easy as pie. The problem is the average user who will not be able to figure out to do anything beyond opening the door and trying to get in that way.

There is no good explanation for average users why footnotes are toxic and cannot be copied, whereas headlines can. And it's not impossible to program this either in a way that is both sane and user friendly.

dsifford commented 5 years ago

@johanneswilm I just tested pasting html directly into the visual editor and it seems to work for supported/registered elements (e.g. blockquotes).. So I think this conversation is largely not even valid.

I'd refer you to one of the core contributors to this project for further clarification just so I don't misspeak and add to the confusion.

johanneswilm commented 5 years ago

@dsifford Yes, so I opened this to figure out what can be pasted and what cannot. I hope to get an answer there eventually.

Something related came up in a break of this year's W3C Editing taskforce meeting at TPAC: Copy and pasting between Google Docs and Wordpress does not work that well, which is surprising given the number of users this will affect worldwide. One solution mentioned by the browser maker devs was that Wordpress could provide and consume its own mimetype format from the web clipboard. That way it could use a subset of HTML specifically targeting Wordpress when integrating pasted content. Another solution would be to organize a meeting/conference for difference web based editors and try to agree on some kind of standard transfer format (please invite me if such a meeting is to be held).

dsifford commented 5 years ago

Sweet! I'll be sure to follow along in that issue thread.

Good point Re: Google Docs and WordPress not pairing well. That's affected me personally dozens of times. (Specifically, the fact that Google Docs seems to want to add <span style="font-weight: 400"> to anything that has had bold toggled.

greatestview commented 5 years ago

Hey @dsifford, first of all: Thank you for your efforts so far. I’m very excited to see you plugin in action, since I was not able to build a plugin by myself (lack of format API documentation). :) Any news on your beta?

dsifford commented 5 years ago

Should be ready within the next week. I’ll chime back in here when that time comes.

nicopujol commented 5 years ago

I am writing an article in Gutenberg now, trying to add a footnote like(1) at the end of the post. Until the feature is ready, is there any workaround we can use in the text editor? I am not cut & pasting, writing from scratch. Thanks!

Update: for now I just used in the text with a standard paragraph at the bottom of the post with Source: XXX. It works for now.

dsifford commented 5 years ago

Beta is nearly ready. Would any of you like to try kicking the tires? Still might be somewhat buggy, but it seems to do the job so far.

Here's a small demo:

https://www.useloom.com/share/fef8ab0d7ba34186aa8129d1924844cc

(I figured out why the tooltip didn't generate properly in the demo... fixing that right now).

Edit: Here's what the citation tooltip looks like after fixing the bug...

nicopujol commented 5 years ago

Looks great on the demo! Do you have a link to the beta plugin in ZIP or uncompressed format to load in WordPress? I'll take it for a test drive.

jcklpe commented 5 years ago

Woo! I'm excited. Going to try the demo now. Should I just git clone the dev branch into my plugins folder?

dsifford commented 5 years ago

Sorry for the delay. Here ya go!

academic-bloggers-toolkit-5.0.0-beta.1.zip

If you guys could also post any bugs you might be able to turn up here, I'd be eternally grateful :pray:

Still a bit of housekeeping and various odds and ends to get to, so I won't be pushing this to wp.org for a little bit still. Maybe next weekend.

Edit: I should also mention that if you'd prefer to build from source, you can do that from the gutenberg branch of the repo, not master.

nicopujol commented 5 years ago

Thanks! Installed it tonight and inserting first footnote was a breeze. Need to read docs for more advanced use, but so far so good. Does it have any performance impact? Is everything generated cacheable?

dsifford commented 5 years ago

I've still neglected to update the docs for the new update, but the old ones should be close enough to get you on the right track.

Does it have any performance impact? Is everything generated cacheable?

This is one of, if not the, most important consideration I had when updating the plugin. Everything that I could possibly cache, I did. Also, the expensive functionality (notably, re-parsing out editor citations and footnotes for changes) only occurs after a user is done typing. You can see this by adding in a footnote/citation and simply using the backspace key to delete it out.. After you stop typing for a few seconds, or click somewhere else, the footnotes list/bibliography will update.

There's still a few perf things that need to be added, but nothing too huge.

nicopujol commented 5 years ago

Here is a test post containing a legacy footnote (MS WORD import, I believe) with a hyperlinked [1], and a footnote using your new plugin, showing as an asterix. https://bit.ly/2LRUZqI. Is there a way to make footnotes the same as the legacy example, i.e. its standard text size and with a hyperlink? Mobile users may find references easier to access and read that way.

dsifford commented 5 years ago

@nicopujol Sorry for the delay. It is possible to make it work in the way you describe, absolutely. There are some potential issues though that I should point out:

The most salient issue is the strict differentiation of footnotes from in-text citations. Some (of the now well over 4000 available citation styles there are to choose from) are styled precisely in the way you have listed (e.g. [1], [2], [1,4,5-9], etc..). So that's the reason I opted to go straight to using symbols.

The other issue with using the traditional anchor-style footnotes is with posts that are paged. Anchors just simply will not work in those instances.

After checking out your example page though, it is clear that the tap targets need to be made bigger on mobile; especially so when footnotes are positioned right next to a link. That's something I will probably work on.

Another thing (as I mentioned in my demo video): I do think there is some merit in allowing users to choose between symbols, numbers, and letters for their footnote format. That's something I'll likely look into after releasing the update. I think it should always default to the symbols though, just for safety with in-text citations.

To comment on your question Re: automatically converting legacy footnotes to this format: I don't think that's worth the effort. Too many potential variations to account for and the return frankly isn't really worth it for the amount of work it would require.

Hope that answers your questions. Good thoughts nevertheless. Again, I definitely think there is some progress to be made on tap targets. :+1:

Edit: Also, I'm wondering why it is that you are even using footnotes in the first place? Both of your examples cite sources. Wouldn't it be better to use citations for that?

nicopujol commented 5 years ago

Thanks @dsifford.

To comment on your question Re: automatically converting legacy footnotes to this format: I don't think that's worth the effort. Too many potential variations to account for and the return frankly isn't really worth it for the amount of work it would require.

Agree, no need to convert legacy citations and footnotes. Only thing is the numbering overlaps, when using a citation.

Hope that answers your questions. Good thoughts nevertheless. Again, I definitely think there is some progress to be made on tap targets. 👍

Great to hear the test was helpful.

Edit: Also, I'm wondering why it is that you are even using footnotes in the first place? Both of your examples cite sources. Wouldn't it be better to use citations for that?

Footnote was the first popup which came on the editor. I see now that citations are located on the editor's sidebar. I replaced the example above on the same post with a citation (web page type, since book did not allow including a URL).

dsifford commented 5 years ago

Yeah, unfortunately in those situations, you'd just need to replace your old citations with new ones using the plugin in order for them to be tracked.

Looking back at your test post I now also see that adding transition: all to link elements within the tooltips causes them to behave weirdly when the tooltip hides due to the transition still applying. I'll have to stub those out in the plugin as well.

dsifford commented 5 years ago

@nicopujol Did you try using the ISBN to insert your book citation? Assuming the book is indexed in google books, that is another alternative. If not, yeah, you'd have to add manually. I'll look into maybe adding a URL field to books.

jcklpe commented 5 years ago

@dsifford

Hey sorry for the delay in testing the plugin.

Unfortunately when I tried to activate the plugin I got this response:

Parse error: syntax error, unexpected '}', expecting identifier (T_STRING) in /home/jcklpe/jackalope.tech/wp-content/plugins/academic-bloggers-toolkit/php/admin.php on line 16

Some additional thoughts:

I know this is primarily an academic tool, but my biggest interest is in the footnotes functionality, and I'd love to see that expanded a bit more. Namely, I think it would be interesting to allow a wider diversity of content to be linked through footnotes. So right now it can only be used to display some text, but it could also be used hypothetically to show a video or image on mouseover. It seems to me that if one were to treat the individual footnotes as their own individual sub blocks you could possibly genericize the footnote content to the point where it could easily be adapted to almost any other kind of gutenberg child block type, even ones not created by you specifically. Just a thought.

dsifford commented 5 years ago

Whoops, sorry @jcklpe... I forgot to mention that I'm being super opinionated and am requiring PHP 7.2+ for the update since I work on this in my spare time and don't feel like supporting older PHP versions.

but it could also be used hypothetically to show a video or image on mouseover. It seems to me that if one were to treat the individual footnotes as their own individual sub blocks.

Interesting idea! Unfortunately, that's a bit outside the scope of the plugin I'm working on so I'll let others take the lead on pursuing something like that.

jcklpe commented 5 years ago

Totally, yeah I prefer to use php also. This is on my host, so I'll have to see if I can upgrade stuff.