400 error when calling the refresh-tokens action

[stevefloat]

I am trying to refresh the CSRF tokens in my Formie form, using the details you describe here: https://verbb.io/craft-plugins/formie/docs/template-guides/cached-forms#csrf-token

However I'm hitting a few issues.

When attempting to fetch refreshed tokens as per your instructions from: http://mysite.local/actions/formie/forms/refresh-tokens?form=myFormHandle

I see the following:

1. If I'm not logged into the Control Panel I get a 403 (Forbidden) error
2. If I am logged into the Control Panel, I get a 404 (Not found) against this URL

I have tried to update my version of Formie from 1.4.5 to 1.5.7, however I get Composer errors for Symfony packages:

Problem 1
    - symfony/cache[v6.0.0, ..., v6.0.2] require php >=8.0.2 -> your php version (7.3; overridden via config.platform, actual: 7.3.9) does not satisfy that requirement.
    - Root composer.json requires verbb/formie 1.5.7 -> satisfiable by verbb/formie[1.5.7].
    - verbb/formie 1.5.7 requires symfony/expression-language ^5.3.0 -> satisfiable by symfony/expression-language[v5.3.0, ..., v5.4.0].
    - symfony/expression-language[v5.3.0, ..., v5.3.11] require symfony/cache ^4.4|^5.0 -> satisfiable by symfony/cache[v4.4.0, ..., v4.4.36, v5.0.0, ..., v5.4.2].
    - symfony/expression-language v5.4.0 requires symfony/cache ^4.4|^5.0|^6.0 -> satisfiable by symfony/cache[v4.4.0, ..., v4.4.36, v5.0.0, ..., v5.4.2, v6.0.0, v6.0.1, v6.0.2].
    - symfony/var-dumper v3.3.6 conflicts with symfony/cache v5.4.2.
    - symfony/var-dumper v3.3.6 conflicts with symfony/cache v5.4.0.
    ....

Are you able to provide any assistance? Many thanks for your help!

Form settings

• Multi-page form: No
• Submission Method: Ajax
• Client-side Validation: Yes
• Custom Form Templates: No

Additional info

• Plugin version: 1.4.5
• Craft version: 3.6.16
• Multi-site: Yes

Additional context

[engram-design]

So that endpoint was only added in 1.4.22 and also fixed in 1.4.24 so you're welcome to update to that version, rather than the 1.5.x line.

As for the composer issue, are you able to send through your composer.json and composer.lock files? There must be a conflicting package in there (likely symfony/expression-language via symfony/cache). This was only introduced in 1.5.x.

[stevefloat]

Thanks for getting back quickly, Josh.

I'll update to 1.4.28 and try that.

It would be good to bring everything right up to date, to 1.5.7. I've attached my composer.json and composer.lock files if you're able to take a look.

Note that I had to remove dukt/twitter when attempting to update Formie, as that requires Guzzlehttp/oauth-subscriber 0.4.0 where Formie required Guzzlehttp/oauth-subscriber 0.6.0. The symfony/cache conflict came after dukt/twitter was removed.

On 25 Jan 2022, at 04:16, Josh Crawford @.***> wrote:

So that endpoint was only added in 1.4.22 https://github.com/verbb/formie/blob/craft-3/CHANGELOG.md#1422---2021-10-22 and also fixed in 1.4.24 https://github.com/verbb/formie/blob/craft-3/CHANGELOG.md#1424---2021-11-06 so you're welcome to update to that version, rather than the 1.5.x line.

As for the composer issue, are you able to send through your composer.json and composer.lock files? There must be a conflicting package in there (likely symfony/expression-language via symfony/cache). This was only introduced in 1.5.x.

— Reply to this email directly, view it on GitHub https://github.com/verbb/formie/issues/727#issuecomment-1020794485, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMN7CBZF46VVRSNLVAOCKYTUXYPYTANCNFSM5MVBBR2Q. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you authored the thread.

[engram-design]

Yup, I'd recommend updating to 1.5.x if you can - but just in case something else is stopping you from updating.

Unfortunately, those attachments haven't come through (looks like you might've replied via email). But good to know about dukt/twitter I'll take a look at that - although I'm not sure how up to date Ben (from Dukt) is keeping that.

[stevefloat]

Thanks Josh - I had attached the Composer files to my email, however Github must have stripped them off.

As Github doesn't allow files with the extensions .json or .lock, I've zipped up and attached.

Thanks again for your time composer files.zip

[engram-design]

Thanks for that, so I've just added "verbb/formie": "dev-craft-3 as 1.5.7", (replacing the existing item), run composer update and that seems to have done the trick. Although I didn't really hit into the symfony dependancies you've mentioned, there was an issue with the dukt/twitter package.

If you'd like to give that a go on your end?

[stevefloat]

Hi Josh - that all worked. Plus no conflict with dukt/twitter when running the Composer update

To continue to make the Twitter plugin work on the frontend, I've used the work around highlighted in this post: https://github.com/dukt/twitter/issues/29

Looking back at why I initially saw the symfony errors, when I first tried to update Formie to 1.5.7 I was doing this via the Craft CLI using php craft update formie. Perhaps this was having a negative effect, somehow?

Manually updating the composer.json file with "verbb/formie": "dev-craft-3 as 1.5.7" and running composer update all worked without issue.

Regarding the version dev-craft-3, will the updates contained within this be rolled into the next release?

Thanks again for your help!

[engram-design]

Yep, I did just add a compatibility fix for dukt/twitter and you can expect to switch that dependancy back to 1.5.7 and continue using php craft update formie once the next version is released (this weekend)

[stevefloat]

Sorry Josh - just coming back to this.

Whilst the updates all went well and the forms are working following the update, I'm seeing "undefined" against every attribute when I attempt to edit the fields in all existing forms - see screenshot:

Have you seen this before following an update?

Thanks for your help.

[engram-design]

Huh - I've never seen something like that! Are there any JS errors in your console, when opening web developer tools?

Worth clearing your caches?

[stevefloat]

No JS errors in the console.

I've cleared all Craft caches at Utilities > Caches plus cleared my browser cache.

Also tried creating a new form incase it was just on existing forms created before the update, however the same happens on new forms too:

[engram-design]

That's really quite strange, I've never seen something like that happen even during development, or is something catastrophic were to happen. But something clearly is not playing well with Vue.

What browser are you using? Are you still on Craft 3.6.16, or on a later version?

[stevefloat]

I'm using Safari 14.1. Same thing happening Chrome - I'm on the latest version (Version 97.0.4692.99 (Official Build) (x86_64)) Same in Firefox - latest version (96.0.2)

Craft is still 3.6.16

Everything was working fine prior to starting this update. My initial set up was as per the composer files I first shared.

Attached are my updated composer files, so you can see what packages have been updated.

Appreciate your help!!

composer files updated.zip

[engram-design]

Ah, looks like there's an issue with Craft 3.6.x as I can replicate that. Strange, but must be something with Craft's bundled version of Vue (that Formie uses). Fixed for the next release. To get this fix early, change your verbb/formie requirement in composer.json to:

"require": {
  "verbb/formie": "dev-craft-3 as 1.5.7",
  "...": "..."
}

Then run composer update.

[stevefloat]

Thanks Josh. That looks like it's fixed it. I'll do some proper testing later however on first glance looks good.

[engram-design]

Fixed in 1.5.8

[clsse]

Hello @engram-design has this been fixed? I'm encountering the same issue.

"400: Couldn't save submission due to errors"

Formie: 1.5.11 Craft: 3.7.36

[engram-design]

@clsse Normally, a 400 error is a CSRF token mismatch error. Does your site use static or Craft's caching? If so, you'll want to look at refreshing the CSRF token

[clsse]

@engram-design I do have all of that in place, which is why I thought my issue matched the original poster of this issue.

[engram-design]

Hmm, can you verify if the token is being replaced in the DOM when the page is loaded? No network errors?

Are you using Vue or similar on the front-end? There is also the enableCsrfValidationForGuests config setting, which you could set to false - even just to test. Anonymous form submissions are a considered attack vector (for the most part)

[clsse]

@engram-design Thank you! I will dig a bit deeper re: check the token is being replaced, and look at the config setting as well. Really appreciate the tips!

I'm just using craftcms, no js framework.

[clsse]

Hi @engram-design , checking back in because I got the form error to go away, but now encountering a new issue (related to caching). Whenever the site has been cached and the form shows for the second time (cached), it triggers a "Changes you made may not be saved." alert window. And this is without any interaction with the form whatsoever.

The alert shows up every-time we click any link, so is obviously a huge issue! Please let me know if you have any thoughts here.

[clsse]

For reference here is my script block:

  <script src="https://cdn.polyfill.io/v2/polyfill.js?features=fetch,Promise"></script>

  <script>
    document.addEventListener('DOMContentLoaded', (event) => {
      let $form = document.querySelector('#{{ formieForm.formId }}');
      let $csrfInput = $form.querySelector('input[name="CRAFT_CSRF_TOKEN"]');

      fetch('/actions/formie/forms/refresh-tokens?form={{ formieForm.handle }}')
        .then(result => { return result.json(); })
        .then(result => {
          $csrfInput.outerHTML = result.csrf.input;
          let jsCaptcha = result.captchas.javascript;
          if (jsCaptcha) {
            $form.querySelector('input[name="' + jsCaptcha.sessionKey + '"]').value = jsCaptcha.value;
          }

          let duplicateCaptcha = result.captchas.duplicate;
          if (duplicateCaptcha) {
            $form.querySelector('input[name="' + duplicateCaptcha.sessionKey + '"]').value = duplicateCaptcha.value;
          }
        });
    });
  </script>

[engram-design]

So yep, that's to be expected somewhat, as technically the form content has changed. You can either turn this warning off with enableUnloadWarning, or you can refresh the cache of the form.

$form.form.formTheme.updateFormHash();

This is also outlined in the docs

[clsse]

@engram-design Thank you very much! Had missed that part of the docs, and it makes lots of sense.

[a-am]

The fix works in Chrome and Firefox however in Safari it is not working. Has anyone else run into this issue?

[engram-design]

This thread veered off into a few different separate issues @a-am which issue are you experiencing? Just that you're getting an unload warning when you haven't changed anything?

[a-am]

@engram-design I am referring to a cached pages returning an error due to csrf tokens not being updated. Chrome (version 108) and Firefox (version 108) seem to be working however Safari (16.2) it does not work. Here is our code:

{% do view.registerJsFile('https://cdn.polyfill.io/v2/polyfill.js?features=fetch,Promise', {position: 'POS_HEAD'}) %}
{% js at endBody %}
        // Wait until the DOM is ready
        document.addEventListener('DOMContentLoaded', (event) => {
            // Fetch the form we want to deal with
            let $form = document.querySelector('#{{ cform.formId }}');

            // Find the CSRF token hidden input, so we can replace it
            let $csrfInput = $form.querySelector('input[name="CRAFT_CSRF_TOKEN"]');

            // Fetch the new token for the form and replace the CSRF input with our new one
            fetch('/actions/formie/forms/refresh-tokens?form={{ cform.handle }}')
                .then(result => { return result.json(); })
                .then(result => { $csrfInput.outerHTML = result.csrf.input; });
        });
{% endjs %}

[engram-design]

Ah, gotcha. There should be zero difference when it comes to browsers here. I can't see an issue with the response back from the endpoint on Safari.

Are you able to confirm if the network response (in Dev tools) is reporting back correctly, with something like:

{
    "csrf": {
        "param": "CRAFT_CSRF_TOKEN",
        "token": "hleTqK_j4D01e0bl9HHXoWxfvkke-veKMLdODsf1cxRXNUIVXhOP4MUtpvvp0LBZZiE3lZ0jlsQDJ8Yfdrm4-gLzDFGDoEpQHQQNfidM2rg=",
        "input": "<input type=\"hidden\" name=\"CRAFT_CSRF_TOKEN\" value=\"hleTqK_j4D01e0bl9HHXoWxfvkke-veKMLdODsf1cxRXNUIVXhOP4MUtpvvp0LBZZiE3lZ0jlsQDJ8Yfdrm4-gLzDFGDoEpQHQQNfidM2rg=\">"
    }
}

I can confirm on my Safari (v16) that the response looks correct, and inspecting the HTML, the CSRF tokens are swapped out.

[a-am]

@engram-design I can confirm I receive the response in the Network tab. However when we submit we get this error in the Network tab:

{
    "name": "Bad Request",
    "message": "Unable to verify your data submission.",
    "code": 0,
    "error": "Unable to verify your data submission.",
    "status": 400
}

There is nothing in the logs about this submission.

We have Three text fields, one phone & one email field. One of the text fields is required. I can also confirm the return csrf input gets added to the the form. In addition, if I turn caching off for the page and remove this extra bit of code then the form works in Safari ¯_(ツ)_/¯

[engram-design]

If the CSRF token input is getting swapped out, I see no reason why that shouldn't work. The only instance this might be an issue is if you're using Vue, and updating the real DOM will not work, due to Vue placing the virtual DOM over the top of the real one.

Otherwise, I'm not sure. Can you look a the network request tab (again) to look at the payload being sent for the submission of the form? It should contain the correct CSRF token - the refreshed one, not the one on-load, if you can confirm that. Does seem quite odd that it's browser-specific though.

[a-am]

@engram-design One detail I forgot to mention is we have reCaptcha enabled. Could that be the culprit or does it's magic just happen on submit?

[engram-design]

Nah, it shouldn't matter, that's all just JS and client-side. If you can let me know what the payload looks like when the form is submitted, vs the HTML source when you inspect the page, it'd be good to know if there's any discrepancies.

[a-am]

The payload looks identical. Out of curiosity I disabled reCaptcha and it worked several times. Then reenabled it and it failed.

[engram-design]

Interesting, I can investigate. Just reCaptcha v3 (visible)?

[a-am]

@engram-design correct, reCaptcha v3 visible. Thank you.

[engram-design]

@a-am Formie 1 or 2? Ajax or Page Reload submission method?

I can't see any difference with using reCaptcha or not at the moment. The request payload looks like:

Content-Disposition: form-data; name="CRAFT_CSRF_TOKEN"
K9chToCUPAZhFeeajCHaXeOlYefjlltt1U4IT9kRzu3tD0S-aOkxxhyWYBbI8X1wBC2Xq-IYtjOpkAKCtdACGrg8XX6DZI2Uq1Ay0BDffpU=

Content-Disposition: form-data; name="action"
formie/submissions/submit

Content-Disposition: form-data; name="submitAction"
submit

Content-Disposition: form-data; name="handle"
contactForm

Content-Disposition: form-data; name="g-recaptcha-response"
03AD1IbLBfFd-sYOjTtdew5avf5J7-EQdHU0SgwRj7dkbiorPsEBSdhp0IhgF8djfDrTJMpQOWhm7a-8THX2sDowqPtyYZwToEN54_jChRQnXVyb5eHWTzc4HbDdevX_lY7PIW4EcVIBKa7kPTA6JgRV81uQCVeY1hdfvGlpRP4b3rWVyCZMMpwgcJPbOEQSsXmrs9NPrmKGb05v8RahmGv6nU1jIPzcHQw1eXVEgJ5OboeuDGeSrYwewthgRSfrNbJ8xpHLDUUx958daDnpvV_e-F7xcqd5qO3WNU2888HvyNQmSC_WKJtSwq51X8r-3gxbl-KPX0vTOzW5vBI1U0vTqz3H0YlK3RNtOKouxmhu0DO0adK4BbGuQ_GMrJHhcTF5kMkZLZI2BrZarYlh9_9XTAv9CABvSrUnthGCwzyzZLXshDchdSJPTeKaTEH_RewsI62-AVcvGUI_tZOMhaKlP9A24h0BiaQxrmQcbSAp_NLUFURrCOHFs

[a-am]

@engram-design Formie 2, Ajax.

[a-am]

@engram-design I have updated Formie to the latest version 2.0.39 and Craft CMS is on 4.5.9. We have a form on a page we are still using static page caching via NGINX FastCGI caching. We are still using Google reCaptcha v3. We have the form being handled by AJAX but I have tried toggling that off with same results. We are using the refresh token javascript from the documentation:

{% js  %}
   // Wait until Formie has been loaded and initialized
    document.addEventListener('onFormieInit', (event) => {
         // Fetch the Form Factory once it's been loaded
         let Formie = event.detail.formie;

         // Refresh the necessary bits that are statically cached (CSRF inputs, captchas, etc)
         Formie.refreshForCache('{{ cform.formId }}');
    });
{% endjs %}

I have made an observation that we continue to get 400 Unable to verify your data submission error on Chrome when the form's Validate Form on Submit however, it works in Firefox. When I toggle Validate Form on Submit off the form works in Chrome and doesn't work in Firefox. I am not sure why that would be the case but I can't get this to work on all browsers.

[engram-design]

Wondering if you might be able to share the site's URL via support@verbb.io so I can check it out?

[a-am]

@engram-design sent the url to the form to the email you posted.

[engram-design]

Thanks for that, we'll follow up via email with some questions. It seems like your action endpoint is being cached, so the refresh-tokens endpoint is serving you to same tokens each time.