Updating, deleting, and rearranging non-current entries

[domenic]

Because the navigation API is scoped to same-origin same-frame history entries, we have a lot more freedom to allow manipulating it, compared to the joint session history represented by window.history. In particular, we could allow rearranging or removing history entries, or even updating their URLs. This is not a security issue, because it only changes the behavior of the app with respect to itself: e.g., if you delete the previous app history entry, that's basically the same as doing window.onload = () => history.back() on the previous page (which, again, is part of your origin).

There's a few cases people have cited where this might be useful:

• Invalidating logically-deleted history entries, either by deleting them or maybe by changing them to point to a 404 page or something: https://mobile.twitter.com/juandopazo/status/1356689596805836802 /cc @juandopazo

• Implementing stack-like navigation patterns: https://mobile.twitter.com/buildsghost/status/1356661396717543424 and https://mobile.twitter.com/maxlynch/status/1356672792226394113 /cc @mlynch. This would require some bookkeeping by the application or framework, and wouldn't be provided out of the box, but I believe the main primitive necessary is rearranging existing entries.

... probably there are more? Let's use this issue to track discussion.

Updating non-current entries' URLs and states, as opposed to deleting/rearranging them, is related to #7, but I think the lower-level concerns there are somewhat orthogonal to this thread. That is, whatever conclusion #7 comes to for updating the current URL and state, will probably port over to non-current entries, if we decide that such modifications would be useful.

[juandopazo]

Hi! Very happy to see this!

There's few unique use-cases we've had to deal with in terms of navigation that lead us to build our own routing instead of relying on something like react-router. We found that we needed access to the whole history stack and we would've benefited from being able to rewrite it entirely.

Deleting items

A very common UI pattern is to have a list of items (emails, recipes, news articles) and be able to open those items into a view. This should naturally navigate. Deleting or invalidating these items will usually also cause a navigation. In most cases this navigates back to the list. To add headaches to our specific case, this is configurable and it can either go back to the list or forward to the next item in the list. This forces our navigation to have to always be "forward".

The issue is that if the user clicks the back button after performing the deletion, in the current state of affairs this will navigate to the previous item in the stack, which now points to something that doesn't exist. We're then forced to chose between either showing an error page -not a great UX- or ignoring the URL and looking back in time to the previous valid state.

With this approach of going back to the last valid state, if our history was [LIST] -> [ITEM] -> delete -> [LIST], clicking back will do nothing as it will just land in [LIST] again. This is not a great UX either.

What made it extra hard

Because of these issues with the lack of control over the browser history we had to treat every in-page navigation (even the ones that say "go back") as adding new browser history entries. This lead to answering the question "what does the in-page back button do?" to be hard to answer. In practice we had to keep our own history stack with a both an idea of an active index (similar to the browser's) and a pointer to where "back" is for each entry in the stack.

This worked, but it was tough. I don't believe users expect this behavior in a web app. Much less in a PWA without a browser chrome. It would be much better to just be able to navigate back in the browser's history. Then PWA navigation would feel very much like a native app (the in-app back buttons would do the same as the native back button). I'd like to see this as the goal of a new browser API.

Dialogs

Another similar case where manipulating the history would be beneficial to web apps is dialogs, particularly in Android. Android users are able to dismiss dialogs with the back button. Since we don't have options to customize dialogs, we end up creating them in HTML. But assigning them full history entries with their URL is not great as dismissing the dialog and clicking back would take the user back to the dialog. This is a case covered by stack navigation patterns if I'm not mistaken.

[domenic]

This is great feedback, thanks! I really appreciate the extra detail on the deleting items case.

So, let's say we added a appHistory.deleteEntry(key) function. Would that then allow you to give the ideal UX? Would it allow the in-page back button to use appHistory.back()? I'd be interested in hearing more about how it would change your codebase and UX.

On the subject of dialogs and the Android back button: our belief is that this is best handled via a dedicated separate proposal: https://github.com/WICG/proposals/issues/18 . Having to abuse the history API (either old or new) to implement the back-button-on-mobile-equals-Esc-on-desktop pattern is not great, and so by working on a separate API for that, we can avoid the complications there infecting the app history API.

[juandopazo]

Re: dialogs, makes sense! Let me take a look.

If I had a full CRUD set of methods for the app's history, I'd probably never write a history stack in JavaScript state again. I'd probably go back to treating the browser's history state as the source of truth. I believe this would likely be the position of open source framework authors as well. We should try to get them to chime in.

There would be some edge cases not covered. It's likely our UI would still be a function (appHistory, appState) => UI, but the cases where there is an inconsistency with the app state are going to be few and I'd treat them differently.

For example, right now when the user refreshes the browser in a certain route we give up recreating the history. If we could treat the app history as the source of truth, we could traverse it to build our UI. If we have a "stack of cards" navigation UI (the typical mobile app UI), when the user refreshes we could rebuild the stack of cards with empty states. Right now we give up and go back to the base URL.

[tbondwilkinson]

I think deleteEntry is very reasonable.

Trickier is deciding how to enable full modification of the stack.

I might suggest enabling some method to stash a full stack and/or replace it with another (or an existing)? But that's pretty radical. Domenic do you have any API suggestions of what that might look like?

[domenic]

I discussed this issue with some implementation folks this morning and the biggest takeaway was that we'd need to propose a more full model for what happens to iframes (including cross-origin iframes) in such modification scenarios. And, also how it'd impact the user's back button perception.

@jakearchibald is creating a collection of weird scenarios with iframes; once we have those then we can try to define what a deleteEntry() would do in each case, or what a stash-and-replace would do.

[Yay295]

If there's a way to delete entries there should probably also be a way to insert entries.

[tbondwilkinson]

I'm not convinced there needs to be a way to insert entries. Can you expound on what use case that would enable?

I think the ability to insert entries is just quite abusable.

[domenic]

Adding a quick note about the connection with the proposed history.clear() API: https://github.com/whatwg/html/issues/5744 .

[posva]

Regarding updating entries other than the current one. The readme currently says

It can be introspected or modified even for the non-current entry, e.g. using appHistory.entries()[i].getState().

https://github.com/WICG/app-history#attaching-and-using-history-state

Is this intended or is it a typo? Since there is AppHistory.updateCurrent() but no AppHistoryEntry.update().

TBH I'm struggling to find use cases for updating an entry rather than deleting it but it would definitely make things more flexible.

[domenic]

Thanks for catching! That is indeed a typo, and I'll fix it momentarily.

One thing that has come up in offline discussions about this is that adding a "forward pruning" API would be particularly easy (since this is already something browsers do when you do a push navigation). I'm curious to hear if people have a sense of how many use cases would be specifically solved by something like appHistory.deleteForwardEntries(), as opposed to needing to delete arbitrary entries by index.

[bathos]

We have a lot of “overlay” views used for previews (e.g. clicking rows in a table opens or updates the overlay) or data entry panels (new foo form, edit foo form). Semantically, these are dialogs — some are even modal — but they’re not transient dialogs. They need to be navigable (reload should not close them; users should be able to share links to them with colleagues). It’s also natural to expect the back button to close them. Because these overlay states are mostly independent from the “primary” view and should be openable over most other views without a definite hierarchical relationship, we adopted a conceit where the URL’s fragment identifier is treated like a URL’s pathname typically might be, but specifically for the overlay panels. For this reason we came to call them “fragment dialogs”.

While it was a given for various reasons that these needed to be navigable states and it was clear pretty quickly that the user intuition was that, after expanding one, the back button would close it again, the specific history behavior needed a bit more nuance to not end up just getting in people’s way: they don’t expect the back button to move them between these fragment dialog states. Again, picture the user clicking rows to quickly scan details for various items: the first click opens an the overlay, the next updates it in place, etc, but the back button at any point would return to the state prior to first opening the overlay. Likewise, when navigating from one major “area” of the app to another, it’s unexpected for back to return you to the prior area with the last fragment dialog still showing.

In general we can achieve this behavior neatly just by using replaceState for any (fragment dialog state) → (any) transition, but scenarios do sometimes arise where this isn’t sufficient: it turned out that expectations vary depending on the relationship between two preview states. The replaceState approach aligns with “I’m flipping through entries like it’s a card catalog,” but if the user clicks something in the fragment dialog to “drill down”, then they actually do expect back to return them to the prior fragment state rather than to close it.

(Apologies for how dense this explanation is, ugh! I promise I’m getting to stuff that relates to entry deletion — it’s hard to convey exactly what the use cases are for us and how this functionality could improve user experience without painting a lot of context. Or maybe I’m just bad at it :)

We can account for cases like this too: use pushState when it’s “drilling down” through the preview, use replaceState when it’s a “lateral” move. But remember earlier I mentioned that if we move to some entirely unrelated area of the application, folks expect the previews to be “closed” when they click back ... and you can see the trouble now probably: if they move from a “drilled down” fragment view to that other area of the app, there’s now two “fragment state history entries” at the head and just using replaceState will only “remove” the top one. This is a scenario where being able to delete an entry “in the past” from “the future” would help.

There are a few other cases that arise like this, e.g. if a user deletes the entity represented in the open preview or editor panel. It’s no longer a navigable state, so (after showing a confirmation), we traverse backwards. But we cannot remove the state we left, whose URL now describes a missing resource; if the user clicks forward at this juncture, they’ll get some kind “hey this is gone / not found” message. This is a scenario where being able to delete “future” entries from the “past” would help.

Re: updating non-current entries: I can think of use cases for this which aren’t already covered by sessionStorage, e.g. updating some bit of history-persisted formdata that the user is expected to probably return to (but which is specifically an “instance” of the form associated with that history entry, vs global state applicable to many forms). This is a fabricated example though: I suspect it has legit use cases, but can’t claim I’ve hit a problem before and thought “if only I could do [that]”.

[domenic]

Thanks, that is super helpful!

[tbondwilkinson]

There are a few scenarios I can think of where this would have value:

1. Dialog - people have mentioned this a number of times. There are UI components where a back navigation should close them, but a forward navigation should NOT open them. Similarly, a navigation to a different component would also close them and make the history state where they were open unnavigable. To support this a forward state should be deleted, and a back state should be deleted. Forward pruning could be used instead of back state deletion.
2. Globally changed state. Imagine a scenario where someone logs in and then goes through a bunch of user flow - account management, updating details, etc. These are all navigations. Now when that user logs out, you may want to delete all those history entries so that going back does not attempt to render information for a user that has already logged out.
3. Globally changed state. Imagine viewer of some type, where you could swap between different "documents", like AMP or an image carousel, where each "pane" had its own history stack. So users expect back/forward to update the pane content, but there are other UI elements on the page that swap the pane contents. When pane contents swap, you'd expect the history stack for the panes to be restored as it was before. To support this, we have to do either fully modifiable history entries, or some type of removing and re-adding entries that were previously added into history (and then removed). You could imagine an API for this is essentially appHistory.updateEntries([entry1, entry2, entry3]) where entries are previously serialized (and now defunct). This opens up all sorts of questions around how this type of API could be abused.
4. Globally update state. Other times, there's a piece of state that's common across multiple views. For instance, say there's some configuration option that is passed to a renderer for multiple views, but that is configurable globally. You'd want to go back through every history state that you serialized and update that option. For instance imagine a dark mode/light mode toggle. Users probably expect once they set dark mode that going back would NOT turn light mode back on, so previous views that were rendered should be updated when navigated back to.

[Yay295]

Globally update state.

I'd think session storage would be more appropriate for this.

[domenic]

I think there are several chunks of capabilities here. Roughly in order of simplest to more-tricky:

1. Deleting all future entries
2. Deleting all past entries
3. Deleting individual entries (past or future; cannot delete current)
4. Modifying the state of individual entries (past or future; current can already be updated)
5. Modifying the URL of individual entries (past or future; current can already be updated via a replace navigation)
6. Adding past or future entries

I don't see any real abuse concerns with (1)-(5). They are all easily accomplished by other hacky means today, mostly based around performing some modification once the user or site goes traverses back to the history entry in question. Note that a form of (2) was also discussed in https://github.com/whatwg/html/issues/5744#issuecomment-661997090.

I am a bit concerned about (5) from other perspectives. I'm not sure we have use cases for it, and in general I want to steer people away from "updates" and toward "navigations" as much as possible.

So I'd like to design an API for at least (1)-(4), and ideally with future extension points for (5) and (6) as needed. I'd like to design such an API now, even if we don't spec and implement it for a while, since it might affect the overall API shape (in particular this intersects with #1).

We probably need to make all of these operations async since the canonical form of the entries is stored in another process. We might be able to make them fake-sync, i.e. changing the current-process representation and then propagating the change to the other process in the background, but that seems a bit sketchy; e.g. if for whatever reason the propagation fails you'd end up in an inconsistent state.

Here's my initial API ideas:

• Assuming async:
  • While keeping entries() as an array:
  • await appHistory.deleteEntry(key) for (1), (2), (3)
  • Maybe await appHistory.deleteForwardEntries() and await appHistory.deleteBackEntries() for (1) and (2)
  • await appHistory.updateOther({ url, state }) for (4) and (5), playing off of appHistory.updateCurrent({ state }). (Note: updateCurrent() is sync, while updateOther() is async.) Probably start out with only { state }, i.e. (4), and introduce the url option later if we find use cases for (5).
  • await appHistory.insertBefore(index, { url, state }) for (6)? Is just url and state enough information to construct an entirely new history entry??
  • If we change entries to a maplike per #1:
  • await appHistory.entries.delete(key) for (1), (2), (3)
  • Maybe await appHistory.entries.deleteForward(), await appHistory.entries.deleteBack() for (1) and (2)
  • Probably still await appHistory.updateOther({ url, state })?
  • Inserting entries is a particularly bad fit for a maplike, as they have a mostly-unordered semantic in their public API despite being ordered under the hood for iteration purposes. I guess we could use await appHistory.insertBefore(key, { url, state })?
• If we want to make state/URL updating fake-sync
  • Remove appHistory.updateCurrent()
  • Introduce appHistoryEntry.update({ url, state })
  • Remove all the awaits for the deletion methods.

Conclusions:

• We need to figure out ASAP whether fake-sync is a good direction or bad since if we think it's good, we want to make immediate changes to updateCurrent(). I lean toward bad, but would be interested in double-checking with implementers; /cc @natechapin @smaug----
• I'm pretty convinced maplike is not good. It isn't future-extensible to (6). And if we decide against fake-sync, then having be async while get()/has()/keys()/etc. are sync. Plus we'd never add set() or clear() so it's a mostly-readonly maplike.

Marking as "Might block v1" until we resolve fake-sync vs. async.

[domenic]

@csreis pointed out a key reason why sync updateCurrent() and replace navigations that synchronously change the current entry are OK, whereas synchronous updates to non-current entries are tricky:

Updates to non-current entries are much more likely to fail due to conflicting modifications from other processes. Whereas the current entry is basically owned by the current process, and such conflicting modifications should never occur.

So in terms of my last message, this is a pretty solid reason to avoid fake-sync.

[domenic]

Conclusion: this is the proposed future API shape. Use cases (1)-(6) refer to the start of https://github.com/WICG/app-history/issues/9#issuecomment-973297995.

• Keep entries() as an array.
• await appHistory.deleteEntry(key) for use cases (1), (2), (3)
• Maybe await appHistory.deleteForwardEntries() and await appHistory.deleteBackEntries() for nicer versions of (1) and (2)
• await appHistory.updateOther({ url, state }) for (4) and (5), playing off of appHistory.updateCurrent({ state }). (Note: updateCurrent() is sync, while updateOther() is async.) Probably start out with only { state }, i.e. (4), and introduce the url option later if we find use cases for (5).

More speculative: await appHistory.insertBefore(index, { url, state }) for (6)? Is just url and state enough information to construct an entirely new history entry??

Removing "might block v1" based on this all being future-compatible with the current design.

[domenic]

Note: I think deleteForwardEntries() / deleteBackEntries() should only be able to delete entries that are visible in entries(), i.e. contiguous same-origin same-frame entries.

For deleteBackEntries() this is clearly necessary, as otherwise it trivially allows abusive back-trapping.

For deleteForwardEntries(), this is less powerful than the hack people are currently doing (of going back to a dummy entry and then pushing a new entry equivalent to the original entry). That will remove all future entries. But I think it's probably OK. I can't think of a great use case for cutting off forward button access to other origins, and if you want to remove same-origin iframe entries you can use iframe.contentWindow.appHistory.deleteForwardEntries() which is more explicit.

This also avoids a situation where, e.g. appHistory.canGoForward is false, and appHistory.entries().at(-1) === appHistory.current, but appHistory.deleteForwardEntries() has an effect. To me that would be unexpected.

[jeromebon]

I'm not convinced there needs to be a way to insert entries. Can you expound on what use case that would

We have one use case for inserting entries. Our sign-in flow is email -> names -> phone number, most of the time our users already filled some of these from other forms on our site. When the user open this sign-in flow we resume from the last incomplete step. Yet we still want the user to be able to edit previous filled steps.

For example, if user A filled it's email and names, but not is phone number, when he resume this sign-in we would like to push [email, names, phonenumber (active)] in the history stack, so that the back button allows him to edit his name or email. Right now we have implemented our own back button in the sign-in to allow edits: if the previous step is already in the history we call window.history.back(), otherwise we call history.replace(). But it feel strange that this back button behave differently from the browser's back button.

That's a small inconvenience for us, most users won't notice this asymmetrical behavior between our back button and the browser's back button.

[bathos]

@jeromebon I may be misunderstanding the use case you’ve described, but that sounds like a kind of “insertion” that’s already possible (and which would remain possible)? In History API terms, it sounds one would invoke pushState() 1-2 times when entering one of the application’s sign up flow states if the expected prior entries aren’t present, these being possibly preceded by a single replaceState so that the earliest entry “becomes” the email state’s entry. Am I picturing that right?

If so, I think that’s a narrower scenario than “generic” insertion at arbitrary offsets without corresponding navigations. I think that’s what “insert entries” referred to (I’m not 100% sure), and that kind of no-nav insertion would constitute a new capability.

[jeromebon]

@bathos You're right, but browsers prevent pushing too many entries in a row. For instance firefox throw "Too many calls to Location or History APIs within a short timeframe." if we attempt to push 4 entries in a row. Chrome's limit is high enough for our use case. I didn't test safari.

With insertions at arbitrary offsets we could do one push then multiple insertions at offset - 1. I assumed that would be less costly for the browser to overcome the current limit. To my understanding the intend is slightly different, pushState intent is to initiate a navitation, that's not what we want to do when we do 4 pushState in a row. Whereas insertion doesn't initiate a navigation, that's why I assumed the limit could be higher. But that might be out of scope and the issue belong to firefox's issue tracker?

[domenic]

Not quite. What browsers are doing with that limit is trying to prevent abusive back-trapping. Which, unfortunately, is not possible to distinguish from non-abusive back trapping like what you're doing...

So, from that perspective, there's no difference between multiple pushState and multiple insertions at -1. Both would need to be limited, as otherwise evil pages could break the back button.

[KevinDoughty]

Yes please.