Allow sessionId of form submissions to be configured per-render

[engram-design]

If you want to use the same form in multiple contexts, the same submission across all is used, unable to be configured.

For example you're rendering a form for event registration against a channel of entries. Being a multi-page form, you allow users to fill out details for multiple events at one time. Currently, Formie will "remember" the incomplete submission as the same submission for all events. Formie acts under the assumption that a single form has a single purpose/context, and therefore ties session data to that form.

To better understand how session data is stored, we use a key to save and fetch it. This'll often look like “formie:10542:submissionId”. So next time the page is loaded for the form you’re using, it’ll just see there’s at least one submission and use that.

I’d propose we introduce a way to provide your own unique ID: “formie:10542:XXXX:submissionId” where this might be the event ID.

[romainpoirier]

Do you have update on this?

I'm still stuck in cases where I need the user to be able to have multiple incomplete submissions per form. In my case, the user can submit multiple projects using multi-pages forms. The problem is that if he open the form again, he gets the data from the other incomplete submission instead of a blank new one.

[engram-design]

Not at the moment sorry, but after finalising Formie v2 I should be able to tackle this soon after.

[romainpoirier]

Please can you tell me more how are saved the session data? Is it in JavaScript (localStorage / sessionStorage), in PHP, or in the MySQL database (if so, in which table / column)?

I think we're also facing major session issues in cases where the user start, stop and get back to the form lately. Sometimes it's from a fresh new session, sometimes from another device / browser... It's like something is mixed between the session, and what is stored as an isIncomplete submission.

[engram-design]

It's PHP-session based, and not in the database. And that's also something I've considered from your other issues that something might be going on with the current session handling, or just down to how you're using forms for your site.

Rest assured, I want to address it properly after giving it some thought!

[romainpoirier]

Thank you. After checking the session file, I can see the data about Formie form, submission & page ID.

I also did this test:

1. Start to fill a new multi-pages form submission;
2. Stop in the middle of it;
3. Delete this incomplete submission from the database (craft_formie_submissions table);
4. Reload the form and/or close it: you are lock in the previous form tab where you were before the deletion;

So even is the submission doesn't exist anymore, the page tab position persists. It persists even if you leave completely the website.

Maybe this is the key reason why we've got unexpected behavior on multi-pages forms that were fill-in in multiple times. In our case, these forms usually got conditions on their tab, and we've seen workflow where some tabs were skipped.

Looks like something is working wrong with the incomplete / loaded submissions + sessions.

[engram-design]

Good point, there needs to be some better handling of stale sessions, particular if the underlying submission is gone. But it is indeed the point of Formie to enable easy resuming of incomplete submissions when reloading the page, or even if the browser crashes, so we certainly need to find a happy medium.

We're also toying with the idea of different session management options, to also disabling it entirely. But again, that's a larger for Formie v2.

As I've mentioned, will try to circle back to this for Formie v1 ASAP.

[engram-design]

Added a sessionKey for Formie v2. This also includes fixes for stale session data when the incomplete submission is destroyed.

To get this early, change your verbb/formie requirement in composer.json to:

"require": {
  "verbb/formie": "dev-craft-4 as 2.0.0-beta.16",
  "...": "..."
}

Then run composer update.

[romainpoirier]

Thank you Josh! Will test this out.

Do you have any ETA for an integration of his fix in Formie v1? Also, I can see that the 2.0.0-beta.16 is still a beta: do you have any ETA when a stable version will be released?

[engram-design]

Plan is to finalise the v2 beta this week after one or two things ticked off the milestone then will look at back-tracking what we can to Formie v1.

[engram-design]

Added in 2.0.0

[romainpoirier]

I've upgraded from 1.5.16 to 2.0.24, but haven't yet set the new sessionKey parameter.

I'm posting this here, as I've found an issue that is probably related to session changes since V1:

• Fill-in a multi-pages form, and stop in the middle;
• Connect to the same account in another browser and re-open the form;
• On opening the form from this session, you're redirect to the first page instead of last active page;
• You can't submit the page anymore and go forward: the submit (next) button is disabled;

Why the active page is not kept and the submit button is disabled in the second session?

[engram-design]

That would be a different issue to the original one, being able to tie an incomplete form submission to a user session isn't strictly possible at the moment. These would be different sessions on different browsers.

I'm not sure why you're getting the disabled submit button though.

[romainpoirier]

Thank you, so how can you define a safe unique sessionKey in case of multi-pages forms?

I don't figure out what to set as new key that will be kept until later edits of the submission. If I use a timestamp, or a random thing, I would get a different key each time the page is reload or the form opened again. Later, I could use the submission id as sessionKey, but this id is unknown until the first page submit.

In the doc example, the key value is entry.id, but how will it be different for each user? In my case, the form is not related to an Entry.

Furthermore, in my case of multi-pages forms, I found that setting the sessionKey is locking me on the first page. If I click next, I'm kept on this first page without any error showing up.

After checking the logs, it says that some required fields are not filled, but they're not in the current first page, they're in the following page(s). It looks like the session can't save if the whole form is not valid. I'm using page reload, no AJAX.

Deleting the sessionKey params make the form working again.

[engram-design]

So the docs example does cater for keeping things unique to an entry element. For example, the entry might be an event, but as you use that to populate the form, you want each event to be stored uniquely, so if the user starts the form submission for one event, but goes to another event to fill out the form - they need to be classified as two different submissions.

Going back to your original question: In my case, the user can submit multiple projects using multi-pages forms - what are your projects? I did assume those were entries you could tie them to.

As for maintaining a session across different browsers, that's not something strictly available with Formie at the moment (probably best to file a new issue), as that's more down to the session storage mechanism we use. That would be a higher-level session key, which I'm not quite sure is possible without messing up Craft's session.

However, it looks like there's an issue with using sessionKey in general anyway, for multi-page forms. I've pushed a fix for that, and to get this early, run composer require verbb/formie:"dev-craft-4 as 2.0.24". Do note that if you're using custom templates or template overrides, there are some template changes required https://github.com/verbb/formie/commit/7528639446054fb25d0045b5e22667163ad518af#diff-3306c454a993841ede24889322d129f7f3709604c6e2a9c1e0709683f703b831

[romainpoirier]

what are your projects? I did assume those were entries you could tie them to.

In the logged-in area, users can access a page that list all forms available. For each form, the users can submit multiples submissions in parallel. These forms are showing using a template that have the renderForm() method. These templates are accessible thanks to routes setup in config/routes.php: they belong to their template, not an entry.

In that case, what's the unique sessionKey value I could generate from the first page (brand new incomplete submission), store and keep for later use? I could use the form's handle instead of the entry.id, but this is not a unique key, so multiple parallel submissions of the same form would not share correctly.

As for maintaining a session across different browsers, that's not something strictly available with Formie at the moment (probably best to file a new issue)

Thank you, I've created the new #1371 issue.

I've pushed a fix for that, and to get this early, run composer require verbb/formie:"dev-craft-4 as 2.0.24"

Thank you, I'll be able to test it after getting more info about the unique sessionKey to use in case of no entry.id.

[romainpoirier]

One more question: is data retention still relevant using the sessionKey? Currently, my setup is Forever, and if I open a new submission while testing with a different session key, it fills in the fields with the previous one in session.

[engram-design]

Ah, I see what you mean. Yes, that's a little trickier. Could you use any portion of the URL or URL segments as the unique key? craft.app.request.segments | last for example, if your last segment determines what project it might be for.

As for data retention, that's purely for saved, completed submissions. Incomplete submissions are pruned according to plugin settings and Craft's garbage collection. It's a setting to help determine how long to keep submission elements around for.

if I open a new submission while testing with a different session key, it fills in the fields with the previous one in session.

Even after my latest update?

[romainpoirier]

Ah, I see what you mean. Yes, that's a little trickier. Could you use any portion of the URL or URL segments as the unique key?

Yes, I can, but this would not be unique. If you open twice the new submission form, the key would be the same. Also, newly created incomplete submission would get the same key as brand new submission. Is that ok to make it works as expected?

As for data retention, that's purely for saved, completed submissions.

Ok, so this has no incidence on the pre-completion of form fields? In my test, if I start completing a multi-pages form, then open a new one, the first page get the fields' values of the previous one, like if there's a retention.

Even after my latest update?

I have updated to 2.0.25 (using "verbb/formie": "dev-craft-4 as 2.0.25") instead of 2.0.24 as you released a new version these last days: is it ok with this one? If so, unfortunately it still doesn't work.

[engram-design]

In my test, if I start completing a multi-pages form, then open a new one, the first page get the fields' values of the previous one, like if there's a retention.

So just to re-iterate what I'm seeing on my end:

I have a URL which loads a form - https://formie.test?key=test1 and I use the key query param to set the sessionKey.

{% set form = craft.formie.forms({ handle: 'contactForm' }).one() %}

{{ craft.formie.renderForm(form, {
    sessionKey: craft.app.request.param('key'),
}) }}

I fill out the first page with values and submit, but don't complete the form. Navigating back to the first page shows my saved values.

In a new tab, I open https://formie.test?key=test2 and the form is blank, as I expect. If I open the original https://formie.test?key=test1 URL again in another tab, I can see my values are still there in the incomplete form. Opening this in a new browser session doesn't retain those values, but we've identified that in another issue and a feature request.

Is this not happening on your end?

[romainpoirier]

Yes, I can confirm that this is working, as the used keys here are unique.

But that's not my case:

• I can't pass an entry.id value as key (as the submission are not linked to entries), and even if they would be linked to entries, they would get the same key as you can submit multiple submissions per entry (in my case, per form);
• Currently, my path for creating a new submission is something like /my-forms/{form.handle} and after the first page is submitted, if you go back to the forms list (/my-forms), it's now displaying a link to the new submission to be able to edit it: this link is currently something like/my-forms/{form.handle}/{submission.id}-{submission.title}, and I'm loading the submission based on the {submission.id} in the URL path;
• When creating a new form, I still don't know what will be the {submission.id} and {submission.title} values, so I can't initialize the key to craft.app.request.segments | last: if I would do it on /my-forms/{form.handle}, this key would not be unique, as it would be used each time someone creates a new submission;
• I could make an incremented key on the link that target new submission form on the forms list (/my-forms) based on the total submitted submissions on this form (like /my-forms/{form.handle}?id={submissions.isIncomplete(*).count()+1)), but I see downside: if someone open multiple tabs the key would be the same, and if some entries are deleted (and the Trashed empty) maybe this would mess the increment and available keys;

So the challenge here seems to generate a unique key each time a form is opened to create a new incomplete submission, and be able to find this key easily later for edition.

[engram-design]

Gotcha, that all makes sense, and I see your predicament. The first step is to define what action constitutes a "new submission" to be generated. That's when the user is starting a brand new form submission, not editing, not loading next or previous pages, and not reloading from validation. Otherwise, we could add it to the template that renders the form, but I think we need to put it before that.

The only feasible way I can see that is by generating a unique identifier on the page that links to the new form (your /my-forms/{form.handle}?id={submissions.isIncomplete(*).count()+1) example).

What about generating a random() variable, or craft.app.security.generateRandomString(10) to use as the sessionKey?

So maybe a change in the route /my-forms/{form.handle}?key={{ randomKey }} and the rendering template can fetch that query param to use as the sessionKey? This does rely on the user messing around with the URL, but it'll also require us to save that value against the form itself, so you can use it later to spin up the same template to edit the incomplete submission with the same session data. I'd suggest populating a hidden field with this value. There are more advanced ways with setSnapshotData, but hidden fields seems the easiest - so long as you've not got lots of forms to add them to.

I'm open to looking at other, less complicated methods if you have any bright ideas as well! I know this seems like this should be a simple thing...

[romainpoirier]

What about generating a random() variable, or craft.app.security.generateRandomString(10) to use as the sessionKey?

Yes, this is a good alternative to the increment. I already thought about use a random key and store it in a hidden field of the first page. But as It's not retroactive to previously submitted submissions and because I need to edit each form to add this new hidden field, I was wondering if there's something more safe and automatic for that problem. As it seems not, I'll try this way.

I'm open to looking at other, less complicated methods if you have any bright ideas as well!

Would it be possible to automatically generate the random sessionKey each time a brand-new submission is submitted? Then, attaching this value to the submission's object (like the ID is generated) and making it available to query later?

[engram-design]

Would it be possible to automatically generate the random sessionKey each time a brand-new submission is submitted? Then, attaching this value to the submission's object (like the ID is generated) and making it available to query later?

Happy to look at adding that, but that's essentially the UID of the submission right now? A submission is only created after the first page is submitted, and never changed after that.

The trick is how do you go about fetching that value for your submission-edit page

[romainpoirier]

As you said, we already have a UID attached to the incomplete submission on first save. So when using the craft.formie.setCurrentSubmission(form, submission) / do submission.form.setSubmission(submission) afterward, would it be possible to automatically use the UID value as sessionKey?

[engram-design]

I've just pushed that addition to craft-4 if that'll help at all.

[romainpoirier]

Thank you, is there anything to set up with this update, like the sessionKey config or something in the URL parameters?

If not, unfortunately it doesn't seems to work because:

• If I open two tabs for new submission, the submission isn't yet loaded until each first page is submitted;
• Even after first page is submitted, I don't know how to load the current one;
• I can't query craft.formie.submissions.form(handle).user(currentUser).isIncomplete(true).one() as I don't know which id to select to get the one currently in view;
• The current submission id is neither in the URL, neither in the form object, neither in craft.app.getSession();
• I can't make an URL redirect to something like /my-forms/{form.handle}/{submission.id}-{submission.title} after the first page is submitted, as I don't know the current id to load the submission;

So it looks like I'm still stuck as commented here.

[engram-design]

Another approach which we've done before is defining a way to denote a new form vs editing a submission.

So in your launch-page for creating a new submission, you could have /my-forms/{form.handle}?new=true, which redirects to a form with the formie.renderForm() call.

{% set sessionKey = craft.app.request.getParam('new') %}
{% set form = craft.formie.forms.handle(handle).one() %}

{% do form.resetCurrentSubmission() %}

{{ craft.formie.renderForm(form) }}

The resetCurrentSubmission() call will discard any session-based submission, in case there is one. As such, it's starting the form submission from scratch.

The user caries on with the submission process like they normally would. They can refresh the page, go back, etc. If they decide to create another new form, they can, which will again remove their current submission from the form, starting anew.

So I assume you have an "incomplete submissions" page where you list these incomplete submissions for users to resume. Maybe /my-forms. I'm not sure how you tie submissions to a user, but there's many ways to do that via email, or a logged-in user. I'm making some assumptions here.

{% for submission in craft.formie.submissions.user(currentUser).isIncomplete(true).all() %}
    <a href="/my-forms/{{ submission.form.handle }}?id={{ submission.id }}">Edit {{ submission.title }}</a>
{% endfor %}

Which would render the same page, but this time it should populate the form with the incomplete submission:

{% set submissionId = craft.app.request.getParam('id') %}
{% set submission = craft.formie.submissions.id(submissionId).one() %}

{% do form.setSubmission(submission) %}

{{ craft.formie.renderForm(form) }}

Tell me if I'm way off base with your requirements here! But to summarise:

If I open two tabs for new submission, the submission isn't yet loaded until each first page is submitted;

So long as when you open your new tabs the resetCurrentSubmission() is called, they should be separate

Even after first page is submitted, I don't know how to load the current one;

Calling form.getCurrentSubmission() will do that if you need it.

[romainpoirier]

Thank you very much for your clear and complete answer.

As validated by the reduced test case in #1373, I can confirm it works (without the sessionKey).

However, when opening a new form, I don't know yet the submissionId. After submitting the first page, the id URL parameter is not added to the URL, so I'm forward back to the initial first page of the form, with a total reset because of {% do form.resetCurrentSubmission() %}.

What would be the recommended method to be able to load the submission as soon as the first page is submitted? Currently, this is working fine when editing the submission after getting its link (example: /test?handle=myForm&id=123456). But when opening a new form submission, the URL doesn't include the id (example:/test?handle=myForm).

form.getCurrentSubmission() could be a solution to that, but it looks like it's loading the incomplete submission from the session. If I open two tabs of incomplete submission, submit the first and the reload the second, I get the values of the first on the second.

To summarize, this won't work as the id is not pushed to the URL after first page submit:

{% if craft.app.request.getParam('id') is empty %}
    {% do form.resetCurrentSubmission() %}
{% else %}

And this won't work because the same form.getCurrentSubmission() value is shared across all brand-new submissions:

{% if craft.app.request.getParam('id') is empty and form.getCurrentSubmission() is empty %}
    {% do form.resetCurrentSubmission() %}
{% else %}

[engram-design]

I was going to suggest checking for the "new" param to only reset the submission if that exists. My example:

{% set isNew = craft.app.request.getParam('new') %}
{% set form = craft.formie.forms.handle(handle).one() %}

{% if isNew %}
    {% do form.resetCurrentSubmission() %}
{% endif %}

form.getCurrentSubmission() will return the in-session, and incomplete submission, but as you say, that's essentially site-wide, so running that in two different tabs will show the same content. That's why we started with the sessionKey to namespace that data. So, we're sort of back where we started.

Maybe I'll need to go back to the drawing board on this one and come up with another solution.

[romainpoirier]

Thank you. A quick and dirty hack would be to redirect to something like /test?handle=myForm&id=12345 just after the first page of /test?handle=myForm is submitted, but I don't find out how to retrieve the id that have just been created on submit.

[engram-design]

The problem is that the page is reloaded so we can’t inject the newly-created submission with route params. Otherwise it’d be trivial to include the submission to your twig templates. And anything to do with the redirect which you could modify will only be done on the final submission.

Which really leaves it to storing the submission in a session, which is what getCurrentSubmission() is for, that just doesn’t work for multiple “sessions” or instances of new submissions. It always assumes that you want one current submission per form, per session.

When looking into this before, it just need a reasonable refactor with how session storage works, as you can tell from your other filed issue.

[romainpoirier]

It looks like this could detect if the page is reloaded after submit (with Validate Form on Submit enabled), but it doesn't feel very safe: craft.app.request.referrer == craft.app.request.absoluteUrl.

If true, we could then retrieve the latest user's submission on reload (using craft.formie.submissions.form(handle).user(currentUser).isIncomplete(null).orderBy('dateCreated desc').one()) and then redirect to something like /test?handle={handle}&id={submission.id} to continue the form submission in a safe, isolated way (i.e. without applying {% do form.resetCurrentSubmission() %} on /test?handle={handle}).

[romainpoirier]

Update: I confirm that this hack is working. However, after the redirect, I'm forwarded to the first page of the form instead of the second. But forcing the redirect as below does the trick:

{%
    if craft.app.request.absoluteUrl != craft.app.request.referrer
    and form.getCurrentPage().sortOrder == 0
%}
    {% do form.setCurrentPage(pages[form.getNextPage().sortOrder]) %}
{% endif %}

[engram-design]

Yeah I've had mixed results relying on craft.app.request.referrer. For whatever reason, it doesn't always report back the referred page, which is why I haven't gone with that (using Controller::refresh()). But checking in your templates might be an option for your needs.

Glad that's working for the moment, I'll give it all some more thought to improve the flow.

[romainpoirier]

Yeah I've had mixed results relying on craft.app.request.referrer

Just curious to secure my hack in prod: what's the other results you were getting?

But checking in your templates might be an option for your needs.

After the redirect I've had to add a comparison between the new URL and the targeted URL, as the craft.app.request.referrer is lost during the redirect.

Glad that's working for the moment, I'll give it all some more thought to improve the flow.

Not 100% tested in live, but I hope it'll works. However, it's a dirty and unsafe solution as it occurs has a hack from Twig template. If you're able to find an effective solution from the Plugin, that would be more than welcome.