Rendering of /app/users page gets stuck from version 3.11.1 on (bottom red progress bar stops at about 80%)

[kic68]

Cannot display /app/users page since version 3.11.1

• Passbolt Version: EXACT RELEASE VERSION OR COMMIT HASH, HERE.
• Platform and Target: -- Operating system: Debian Bullseye -- PHP: 7.4 -- Web server: nginx/1.18.0 -- Database server: MariaDB 10.1.48 -- etc.: Using docker image 3.11.1-1-ce-non-root from https://hub.docker.com/r/passbolt/passbolt/tags

What you did

I am opening this issue here (although using the docker image) as I am pretty sure it's an issue with a changed JS. Please continue reading.

I am trying to migrate our (long time) existing "VM" installation of Passbolt into a K8S installation. The "VM" version is 3.4.0, so I started setting up image 3.4.0-ce-non-root (community edition not running as root). After I configured all the environment variables and GPG files correctly, this was working perfectly.

I then wanted to directly migrate to version 3.12.0-3-ce-non-root and watched the backend migration running smoothly. Everything seems to work except that the /app/users page gets stuck during loading (that bottom red progress bar just stops at about 75% of page width).

So I tested every minor version from 3.4 upwards to see where the issue would begin - and it did start between 3.10.0-1-ce-non-root (all ok) and 3.11.1-1-ce-non-root (which all the following debugging output is coming from):

I have compared the content of the working and non-working /users.json, they are both identical except the first line:

diff broken_users.json good_users.json
1c1
< {"header":{"id":"69288ac9-4ed5-435e-b0a5-c5ec9de95c1f","status":"success","servertime":1681218464,"action":"d7bc9044-a64e-5421-a4d7-7a94eaa39d37","message":"The operation was successful.","url":"\/users.json","code":200,"pagination":{"count":169,"page":1,"limit":null}},"body":[{"id":"8495788c-8f73-4f6f-8947-92c74c6d1c22","role_id":"d1acbfc1-78d8-3e25-ad8b-7ab1eb0332dc","username":"xxx@xxx.xx","active":true,"deleted":false,"created":"2022-12-02T11:07:45+00:00","modified":"2022-12-05T10:2\ No newline at end of file
---
> {"header":{"id":"873c3ef5-3a46-48eb-b3cb-8bb4d4f617a7","status":"success","servertime":1681218632,"action":"d7bc9044-a64e-5421-a4d7-7a94eaa39d37","message":"The operation was successful.","url":"\/users.json","code":200,"pagination":{"count":169,"page":1,"limit":null}},"body":[{"id":"8495788c-8f73-4f6f-8947-92c74c6d1c22","role_id":"d1acbfc1-78d8-3e25-ad8b-7ab1eb0332dc","username":"xxx@xxx.xx","active":true,"deleted":false,"created":"2022-12-02T11:07:45+00:00","modified":"2022-12-05T10:2\ No newline at end of file

In the chrome browser's console it throws a caught error (not sure how to copy and paste this in a readable way):

app.js:2 Uncaught (in promise) 
{stack: 'EntityValidationError: Could not validate entity U…mhafipceonhjepacocaffmoppf/index.min.js:2:270214)', message: 'Could not validate entity User.', name: 'EntityValidationError', details: {…}}
details
: 
username
: 
{custom: 'The username is not valid.'}
[[Prototype]]
: 
Object
message
: 
"Could not validate entity User."
name
: 
"EntityValidationError"
stack
: 
"EntityValidationError: Could not validate entity User.\n    at te.validateObject (chrome-extension://didegimhafipceonhjepacocaffmoppf/index.min.js:2:13833)\n    at te.validate (chrome-extension://didegimhafipceonhjepacocaffmoppf/index.min.js:2:13552)\n    at new rr (chrome-extension://didegimhafipceonhjepacocaffmoppf/index.min.js:2:80972)\n    at chrome-extension://didegimhafipceonhjepacocaffmoppf/index.min.js:2:120875\n    at Array.forEach (<anonymous>)\n    at new Jr (chrome-extension://didegimhafipceonhjepacocaffmoppf/index.min.js:2:120845)\n    at vn.findAll (chrome-extension://didegimhafipceonhjepacocaffmoppf/index.min.js:2:224949)\n    at async vn.updateLocalStorage (chrome-extension://didegimhafipceonhjepacocaffmoppf/index.min.js:2:224529)\n    at async ic.exec (chrome-extension://didegimhafipceonhjepacocaffmoppf/index.min.js:2:270368)\n    at async ic._exec (chrome-extension://didegimhafipceonhjepacocaffmoppf/index.min.js:2:270214)"
[[Prototype]]
: 
Object

In the Firefox console the error looks like this:

Uncaught (in promise) 
Object { fileName: "moz-extension://fc6114f2-eca0-4f1d-8390-7e7c05e52ab6/index.min.js", lineNumber: 2, columnNumber: 11525, message: "Could not validate entity User.", name: "EntityValidationError", details: {…} }
columnNumber: 11525
details: Object { username: {…} }
username: Object { custom: "The username is not valid." }
custom: "The username is not valid."
<prototype>: Object { … }
<prototype>: Object { … }
fileName: "moz-extension://fc6114f2-eca0-4f1d-8390-7e7c05e52ab6/index.min.js"
lineNumber: 2
message: "Could not validate entity User."
name: "EntityValidationError"
<prototype>: Object { … }

The usernames are all normal email addresses, so I am not sure why one should be invalid.

Unfortunately my debugging skills in the frontend end here, so I would kindly ask for help.

What happened

The /app/users page stays empty in the main area (just the user groups on the left are displayed normally).

What you expected to happen

The /app/users page should still show the list of users normally.

I will be offline for 10 days now, but wanted to start a debugging thread - maybe someone of you sees something without needing more information from my side immediately.

Thank you very much. Torben

[claytontstevenson]

hey @kic68 thanks for the report! This is an issue we have identified and are already working on a fix. We have it tracked internally as PB-23949

It is related to a regex that was recently. Since you mentioned all of the usernames were normal email addresses my guess would be one of them is in caps and that is causing it as we have seen that as a cause of this

[kic68]

Hey Clayton, thanks for the very quick reply. I have just executed select username from users; on the DB server - and "unfortunately" in this case there's nothing in all caps. The most exotic characters are hyphen and dot. I cannot detect any special characters, also the lengths are not exceeding the "normal" ranges. Unfortunately I cannot disclose the data :(

I could also test it on this migrated instance once I am back from vacation in 10 days, if you let me know where this regex is located. These "compressed" JS files are just completely unreadable (and I am not a JS developer at all :) ).

Thanks anyways for looking into it. Best, Torben

[claytontstevenson]

Hey Torben, I believe this fix was previously released as part of 3.12.1 and now we are on 4.1.1. Have you had a chance to update since you posted this? If so has this issue been resolved or is it still present for you?

[kic68]

Hi, sorry, I was on vacation. The issue was still happening with 4.0.2-1-ce-non-root . So I'd rather keep this issue open.

[cedricalfonsi]

Hey @kic68,

This is quite unexpected.

Could you check if one of the emails doesn't validate with the following regex:

'/^[\p{L}0-9!#$%&\'*+\/=?^_`{|}~-]+(?:\.[\p{L}0-9!#$%&\'*+\/=?^_`{|}~-]+)*@(?:[_\p{L}0-9][-_\p{L}0-9]*\.)*(?:[\p{L}0-9][-\p{L}0-9]{0,62})\.(?:(?:[a-z]{2}\.)?[a-z]{2,})$/ui'

[kic68]

Hey @cedricalfonsi ,

thanks for the regex and sorry for the delay, vacation again :)

I have tested all email addresses in a little python script - and all are matching your regex. We only accept company addresses and they are (ASCII) first.last@company.de

Best, Torben

[cedricalfonsi]

Hey @kic68,

This is odd, here is the javascript which validates emails on the client side: https://github.com/passbolt/passbolt_styleguide/blob/67330b3353b4e812ce084b2164baa3cfa1e86e18/src/shared/lib/Validator/IsEmailValidator.js#L6

Regex should be similar, but it might be interpreted differently, I created a jsfiddle you could use to test your users usernames: https://jsfiddle.net/6w1t4sgf/22/

[kic68]

Hey Cedric,

thanks a lot for the testing setup - I am definitely not familiar enough with JS to do this myself.

Unfortunately it seems all email addresses ("username") seem to be ok... the output is

    ☁️ "Running fiddle"
    "lalala"
    true
    true
    true
    true
   ... (many more in between)
    true
    true
    true

Best, Torben

[cedricalfonsi]

There is one API configuration this error could be imputed too, but it's very unlikely that you changed it. Can you confirm there is no email regex override in the output of https://PASSBOLT_DOMAIN/settings.json.

You are looking for the following information:

            "email": {
                "validate": {
                    "regex": REGEX_VALUE
                }
            },

On another hand can you also confirm you are using the latest version of the browser extension? It should be 4.3.1 (or 4.3.0 on edge). You can find the API and browser extension versions by placing your mouse hover the heart in the bottom right corner of the application.

Can you provide us with your browser flavor and version also? It will help to reproduce if you are running an exotic one :)

[kic68]

Good morning Cedric,

no, there is no email key in the output.

I need to set up a test environment to reproduce the issue again with the latest versions of passbolt (we haven't dared to upgrade anymore since then), but when I opened the issue this was happening in both Firefox and Chrome in their latest versions on a Mac. Also I was on the latest version of the plugin (I had to reinstall it a few times while switching between environments).

Best, Torben

[kic68]

Good evening Cedric,

I have dumped the production data, set up a new test environment with exactly that data - and it was working with newer versions. You were totally right, there is an email key in settings.json - but it only appears in the newer (broken for me) versions. The regex is set like this in production: "email":{"validate":{"regex":"\/.*@companyname\\.de\/"}} - and it is not set in the test environment where all versions work. I wanted to prevent colleagues from registering with their private addresses, but obviously I am doing it the wrong way. I will try to fix and upgrade production beginning of next week!

Thanks a lot for your help and getting me into the right direction!

Have a nice weekend! Best, Torben

[kic68]

Hi Cedric,

I can confirm that the issue goes away if you define the regex value correctly and completely:

  PASSBOLT_EMAIL_VALIDATE_REGEX:
    value: '/.*@(company\.de|company\.com|anothercompanyname\.de)/'

I have migrated our existing VM passbolt instance to k8s and, during that process, also added this regular expression. Unfortunately I did not know this would also affect already existing accounts with a domain we would rather not want to have and therefore was missing in the regex definition.

Adding all existing domain names now allows me to upgrade to 4.0. I am then running into the "Too many redirects" issue with 4.1, but this is something different and I will now debug this.

Thanks a lot for your help! Torben

[claytontstevenson]

Hey Torben,

For the "Too many redirects" issue you'll likely need to turn the option for force SSL off as this is frequently the cause of that issue when you have a proxy in front handling SSL. If that doesn't solve it for you feel free to head over to our forum and we can troubleshoot it there with you since that error is usually more environmental than code related.