salesagility / SuiteCRM-Core

SuiteCRM - Open source CRM for the world
https://www.suitecrm.com
GNU Affero General Public License v3.0
176 stars 123 forks source link

Enable Notifications​ Unable to login after 8.4.2 upgrade #398

Open serhiisamko091184 opened 10 months ago

serhiisamko091184 commented 10 months ago

Issue

Summary:

  1. LDAP was used.
  2. After update, auth fails 2.1 504 gateway timeout - timeout thresholds increased 2.2 Login credentials incorrect, please try again.
  3. moved the .env.local config (posted above) somewhere else, so only .env is in effect (which has no LDAP config): 3.1 Fails authentication - Login credentials incorrect
  4. Opened .env to analyse it, has the whole empty LDAP section UNCOMMENTED. So I comment it. 4.1 500 Error (posted above) Impossible to login neither with previous and working LDAP configuration, NOR without it.

logs/prod/prod.log

[2023-11-16 16:06:36] php.INFO: User Deprecated: Since api-platform/core 2.7: Use "ApiPlatform\Metadata\Resource\Factory\ResourceMetadataCollectionFactoryInterface" instead of "ApiPlatform\Core\Metadata\Resource\Factory\ResourceMetadataFactoryInterface". {"exception":"[object] (ErrorException(code: 0): User Deprecated: Since api-platform/core 2.7: Use \"ApiPlatform\\Metadata\\Resource\\Factory\\ResourceMetadataCollectionFactoryInterface\" instead of \"ApiPlatform\\Core\\Metadata\\Resource\\Factory\\ResourceMetadataFactoryInterface\". at /var/www/html/suitecrm/sales/vendor/api-platform/core/src/Serializer/AbstractItemNormalizer.php:130)"} []
[2023-11-16 16:06:36] php.INFO: User Deprecated: Since api-platform/core 2.7: Use an implementation of "ApiPlatform\Api\IriConverterInterface" instead of "ApiPlatform\Core\Api\IriConverterInterface". {"exception":"[object] (ErrorException(code: 0): User Deprecated: Since api-platform/core 2.7: Use an implementation of \"ApiPlatform\\Api\\IriConverterInterface\" instead of \"ApiPlatform\\Core\\Api\\IriConverterInterface\". at /var/www/html/suitecrm/sales/vendor/api-platform/core/src/Hydra/Serializer/CollectionNormalizer.php:62)"} []
[2023-11-16 16:06:36] php.INFO: User Deprecated: Since api-platform/core 2.7: Use "ApiPlatform\Metadata\Resource\Factory\ResourceMetadataCollectionFactoryInterface" instead of "ApiPlatform\Core\Metadata\Resource\Factory\ResourceMetadataFactoryInterface". {"exception":"[object] (ErrorException(code: 0): User Deprecated: Since api-platform/core 2.7: Use \"ApiPlatform\\Metadata\\Resource\\Factory\\ResourceMetadataCollectionFactoryInterface\" instead of \"ApiPlatform\\Core\\Metadata\\Resource\\Factory\\ResourceMetadataFactoryInterface\". at /var/www/html/suitecrm/sales/vendor/api-platform/core/src/Hydra/Serializer/PartialCollectionViewNormalizer.php:50)"} []
[2023-11-16 16:06:36] php.INFO: User Deprecated: Since api-platform/core 2.7: Use "ApiPlatform\Metadata\Resource\Factory\ResourceMetadataCollectionFactoryInterface" instead of "ApiPlatform\Core\Metadata\Resource\Factory\ResourceMetadataFactoryInterface". {"exception":"[object] (ErrorException(code: 0): User Deprecated: Since api-platform/core 2.7: Use \"ApiPlatform\\Metadata\\Resource\\Factory\\ResourceMetadataCollectionFactoryInterface\" instead of \"ApiPlatform\\Core\\Metadata\\Resource\\Factory\\ResourceMetadataFactoryInterface\". at /var/www/html/suitecrm/sales/vendor/api-platform/core/src/Hydra/Serializer/CollectionFiltersNormalizer.php:53)"} []
[2023-11-16 16:06:36] php.INFO: User Deprecated: Since api-platform/core 2.7: Use an implementation of "ApiPlatform\Api\IriConverterInterface" instead of "ApiPlatform\Core\Api\IriConverterInterface". {"exception":"[object] (ErrorException(code: 0): User Deprecated: Since api-platform/core 2.7: Use an implementation of \"ApiPlatform\\Api\\IriConverterInterface\" instead of \"ApiPlatform\\Core\\Api\\IriConverterInterface\". at /var/www/html/suitecrm/sales/vendor/api-platform/core/src/JsonLd/Serializer/ObjectNormalizer.php:44)"} []
[2023-11-16 16:06:36] request.CRITICAL: Uncaught PHP Exception Symfony\Component\Ldap\Exception\LdapException: "Could not initiate TLS connection: Can't contact LDAP server" at /var/www/html/suitecrm/sales/vendor/symfony/ldap/Adapter/ExtLdap/Connection.php line 162 {"exception":"[object] (Symfony\\Component\\Ldap\\Exception\\LdapException(code: 0): Could not initiate TLS connection: Can't contact LDAP server at /var/www/html/suitecrm/sales/vendor/symfony/ldap/Adapter/ExtLdap/Connection.php:162)"} []
[2023-11-16 16:06:36] php.INFO: User Deprecated: Since api-platform/core 2.7: Use "ApiPlatform\Metadata\Resource\Factory\ResourceMetadataCollectionFactoryInterface" instead of "ApiPlatform\Core\Metadata\Resource\Factory\ResourceMetadataFactoryInterface". {"exception":"[object] (ErrorException(code: 0): User Deprecated: Since api-platform/core 2.7: Use \"ApiPlatform\\Metadata\\Resource\\Factory\\ResourceMetadataCollectionFactoryInterface\" instead of \"ApiPlatform\\Core\\Metadata\\Resource\\Factory\\ResourceMetadataFactoryInterface\". at /var/www/html/suitecrm/sales/vendor/api-platform/core/src/Serializer/SerializerContextBuilder.php:41)"} []
[2023-11-16 16:06:36] php.INFO: User Deprecated: Since api-platform/core 2.7: Use "ApiPlatform\Metadata\Resource\Factory\ResourceMetadataCollectionFactoryInterface" instead of "ApiPlatform\Core\Metadata\Resource\Factory\ResourceMetadataFactoryInterface". {"exception":"[object] (ErrorException(code: 0): User Deprecated: Since api-platform/core 2.7: Use \"ApiPlatform\\Metadata\\Resource\\Factory\\ResourceMetadataCollectionFactoryInterface\" instead of \"ApiPlatform\\Core\\Metadata\\Resource\\Factory\\ResourceMetadataFactoryInterface\". at /var/www/html/suitecrm/sales/vendor/api-platform/core/src/Serializer/SerializerFilterContextBuilder.php:40)"} []
[2023-11-16 16:06:36] php.INFO: User Deprecated: Since api-platform/core 2.7: The listener "ApiPlatform\Core\EventListener\ReadListener" is deprecated and will be replaced by "ApiPlatform\Symfony\EventListener\ReadListener" in 3.0. {"exception":"[object] (ErrorException(code: 0): User Deprecated: Since api-platform/core 2.7: The listener \"ApiPlatform\\Core\\EventListener\\ReadListener\" is deprecated and will be replaced by \"ApiPlatform\\Symfony\\EventListener\\ReadListener\" in 3.0. at /var/www/html/suitecrm/sales/vendor/api-platform/core/src/Core/EventListener/ReadListener.php:66)"} []
[2023-11-16 16:06:36] php.INFO: User Deprecated: Since api-platform/core 2.7: Use "ApiPlatform\Metadata\Resource\Factory\ResourceMetadataCollectionFactoryInterface" instead of "ApiPlatform\Core\Metadata\Resource\Factory\ResourceMetadataFactoryInterface". {"exception":"[object] (ErrorException(code: 0): User Deprecated: Since api-platform/core 2.7: Use \"ApiPlatform\\Metadata\\Resource\\Factory\\ResourceMetadataCollectionFactoryInterface\" instead of \"ApiPlatform\\Core\\Metadata\\Resource\\Factory\\ResourceMetadataFactoryInterface\". at /var/www/html/suitecrm/sales/vendor/api-platform/core/src/Symfony/EventListener/DeserializeListener.php:63)"} []
request.CRITICAL: Uncaught PHP Exception Symfony\Component\Ldap\Exception\LdapException: "Could not initiate TLS connection: Can't contact LDAP server" at /var/www/html/suitecrm/sales/vendor/symfony/ldap/Adapter/ExtLdap/Connection.php line 162 {"exception":"[object] (Symfony\\Component\\Ldap\\Exception\\LdapException(code: 0): Could not initiate TLS connection: Can't contact LDAP server at /var/www/html/suitecrm/sales/vendor/symfony/ldap/Adapter/ExtLdap/Connection.php:162)"} []

Expected Behavior

Possible to login with LDAP config or without after update.

Actual Behavior

After update from 8.4.1 to 8.4.2 it's impossible to login under any user.

Possible Fix

Steps to Reproduce

Initially described: https://community.suitecrm.com/t/unable-to-login-after-8-4-2-upgrade/90970/6

Context

Update from 8.4.1 to 8.4.2 version

Your Environment

jack7anderson7 commented 9 months ago

https://community.suitecrm.com/t/unable-to-login-after-8-4-2-upgrade/90970/20?u=jack_anderson

rmendes-13 commented 8 months ago

I have upgraded to 8.5.0

SuiteCRM Finalize Upgrade 
============

Running: legacy-post-upgrade
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: Interlace handling should be turned on when using png_read_image
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: Interlace handling should be turned on when using png_read_image
libpng warning: Interlace handling should be turned on when using png_read_image
libpng warning: Interlace handling should be turned on when using png_read_image
libpng warning: Interlace handling should be turned on when using png_read_image
step: legacy-post-upgrade | status: done
Post Upgrade process complete
Running: add-new-config-entries
step: add-new-config-entries | status: done
Added new config entries to config file
Running: metadata-merge
step: metadata-merge | status: done
Metadata successfully merged
Running: run-migrations
Warning! SuiteCRM is now using angular 16.
Warning! 'defaultExt' and any extension that contains frontend changes will need to be rebuilt. For defaultExt you can build using `yarn run build:defaultExt`
Warning! 'defaultExt' has been disabled. It may need to be rebuilt.
Warning! Extensions other than 'defaultExt' will need the following files manually updated:
- extensions/your-extension/app/tsconfig.app.json
- extensions/your-extension/app/webpack.config.js
- extensions/your-extension/config/extension.php
step: run-migrations | status: done
Successfully run migrations
Running: finalize-clear-symfony-cache
step: finalize-clear-symfony-cache | status: done
Successfully cleared cache
Warning! Please re-set permissions after this upgrade step has completed

Unable to login.

I get this error:

[2024-01-23 15:38:11] request.CRITICAL: Uncaught PHP Exception Symfony\Component\Ldap\Exception\LdapException: "Could not complete search with dn "", query "" and filters "*". LDAP error was [2] Protocol error." at /var/www/html/suitecrm/vendor/symfony/ldap/Adapter/ExtLdap/Query.php line 133 {"exception":"[object] (Symfony\\Component\\Ldap\\Exception\\LdapException(code: 0): Could not complete search with dn \"\", query \"\" and filters \"*\". LDAP error was [2] Protocol error. at /var/www/html/suitecrm/vendor/symfony/ldap/Adapter/ExtLdap/Query.php:133)"} []

And this the configuration:

###> LDAP CONFIG ###
LDAP_HOST='idserver.domain.tld'
LDAP_PORT=389
LDAP_ENCRYPTION=tls
LDAP_PROTOCOL_VERSION=3
LDAP_REFERRALS=false
LDAP_DN_STRING='dc=domain,dc=tld'
LDAP_QUERY_STRING='uid={username}'
LDAP_SEARCH_DN='uid=bind-user,cn=sysaccounts,cn=etc,dc=domain,dc=tld'
LDAP_SEARCH_PASSWORD='bind-password'
###< LDAP CONFIG ###
###> LDAP AUTO CREATE CONFIG ###
LDAP_AUTO_CREATE=enabled
LDAP_PROVIDER_BASE_DN='cn=accounts,dc=domain,dc=tld'
LDAP_PROVIDER_SEARCH_DN='uid=bind-user,cn=sysaccounts,cn=etc,dc=domain,dc=tld'
LDAP_PROVIDER_SEARCH_PASSWORD='bind-password'
LDAP_PROVIDER_DEFAULT_ROLES=ROLE_USER
LDAP_PROVIDER_UID_KEY='(uid={username})'
LDAP_PROVIDER_FILTER='(memberOf=cn=sales,cn=groups,cn=accounts,dc=domain,dc=tld)'
###< LDAP AUTO CREATE CONFIG ###

Since the LDAP CONFIG section at .env was empty and uncommented, I commented all the LDAP the config there, since I created a .env.local for the LDAP config.

After making the changes to the .env file, I tried to login again.

I get this:

Login credentials incorrect, please try again.

And zero output on any log. Thank you.

piotrgitt commented 4 months ago

@rmendes-13 Unfortunately debugging ldap authentication in SuiteCRM is BIG pain in the ass, logs won't tell you much or even anything if something goes wrong, but maybe i don't know something here. AD logs from microsoft also are a bit cryptic, maybe for me. It's also hard to find specific informations, knowledge on the internet, even in Symphony itself documentation. I won't even comment about actual SuiteCRM8 LDAP documentation which is just WRONG and misleading in many places, this is just hiliarous (i think whole documentation for suitecrm in 8+ version is very poor). For user autocreation i have found better documentation which helped me more in DIFFERENT programming language, for different software, on some old blank page from 15 years ago, so answer yourself.

At first, my LDAP server is on Microsoft Server with Active Directory. SuiteCRM8.5/8.6 stands on new Debian, with PHP8.2. So maybe, if your config is different, tell me, becouse it could mean my config won't work for you.

Overall I think you have a bad configuration. Some variable values are different in my configuration, and my configuration was not working on suitecrm8.5 when i had certains variables set just as you. Now my setting work as well in 8.5 as in 8.6. I wil give you some hints.

  1. If you finalized your CRM actualization from root user (./bin/console suitecrm:app:upgrade-finalize) and not from server user (like apache2's www-data user) you NEED to modify permissions for whole CRM instance. For clear head just do it after update, reset your permissions: https://docs.suitecrm.com/8.x/admin/installation-guide/downloading-installing/#_2_3_set_permissions find . -type d -not -perm 2755 -exec chmod 2755 {} \; find . -type f -not -perm 0644 -exec chmod 0644 {} \; find . ! -user www-data -exec chown www-data:www-data {} \; chmod +x bin/console

    a) You could also try rebulid default Extension in console yarn run build:defaultExt, you could have there some configuration for ldap, but you should not have (i don't have), so it should not matter much.

  2. Clear your cache for prod and qa environments. Best delete all files by hand from ./cache/prod/ and ./cache/qa/. Remember, dont touch legacy cache (legacy "cache" is not real cache). You can additional do: ./bin/console cache:pool:clear. You can also restart apache2 now and then for making shure php reads some changes.

  3. .env file - i have there whole ldap configuration, not on .env.local. On ldap.local i have only configured one line, one variable - "DATABASE_URL". So maybe you should move ldap config for .env file, but maybe it will work in .env.local

  4. For time of testing/debugging try LDAP without encryption and without LDAP user auto creation. So hash every line on ###> LDAP AUTO CREATE CONFIG ### and change LDAP_ENCRYPTION=tls for LDAP_ENCRYPTION=none.

  5. Make sure you have some ldap user already created in suitecrm, which have same username in LDAP server and have external_auth_only=1 set in database (this is database field).

  6. Here is my config for just LDAP authentication, paste it and modify for you in .env file, or try in .env.local:

###> symfony/framework-bundle ###
APP_ENV=prod
#TRUSTED_PROXIES=127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
#TRUSTED_HOSTS='^localhost|example\.com$'
###< symfony/framework-bundle ###

###> nelmio/cors-bundle ###
CORS_ALLOW_ORIGIN=^https?://(localhost|127\.0\.0\.1)(:[0-9]+)?$
###< nelmio/cors-bundle ###

###> AUTH_TYPE ###
AUTH_TYPE=ldap
###< AUTH_TYPE ###

###> LDAP CONFIG ###
LDAP_HOST='17.56.160.20'    ### fake IP of course here. Best practice is set hostname of ldap server here. IP for sake of testing
LDAP_PORT=389        ### left default 389     
LDAP_ENCRYPTION=none     
LDAP_PROTOCOL_VERSION=3        ### left default   
LDAP_REFERRALS=false        ### left default
LDAP_DN_STRING='ou=Users,dc=organization,dc=local' 
LDAP_QUERY_STRING='sAMAccountName={username}'
LDAP_SEARCH_DN='CN=bind,OU=Berlin,OU=IT_department,DC=organization,DC=local' 
LDAP_SEARCH_PASSWORD='***********************'  
###< LDAP CONFIG ###

a) LDAP_ENCRYPTION set none, for sake of testing, b) LDAP_DN_STRING - this is crucial, and i think you have error here. Unfornately you need to set some organizational unit (ou) here for crm or symphony able to find user in your LDAP server instance. Unfornately you can set only one organizational unit here, so you need to have all users under one organizational unit 'tree'. You can have subfolders for one organizational unit (ou) but still you need to pass this into LDAP_DN_STRING. In my config all users (all who need have access ldap login for certain systems) in AD are under certain subfolders (or sub-organizational units, you name it), but all subfolders are under 'users' organizational unit. That's why i have ou=Users in LDAP_DN_STRING variable, and that should be enough. I can't find any information about this weird behavior, but this behavior is not explicit for suitecrm. I had already a few experiences with softwares that can't find user in AD without passing organizational unit in query. Maybe it's overall symfony underdevelopment in older symphony versions, becouse suitecrm8.5, 8.6 use symfony 5.2.14. In roadmap they claim symfony will be upgraded in 8.7 suitecrm version (https://suitecrm.com/suitecrm-roadmap/, https://suitecrm.com/suitecrm-monthly-roundup-november-2023/), so maybe it will erase this problem, c) LDAP_QUERY_STRING - also crucial. sAMAccountName is field for username of users in Microsoft Server Active Directory, and {username} is field in SuiteCRM software. If using Microsoft AD Ldap server - use this variable for query, d) LDAP_SEARCH_DN - also crucial. need to pass whole path to bind user in domain. "bind" is his username (also described in domain by sAMAccountName field)., e) LDAP_SEARCH_PASSWORD - check if have correct password to bind user, of course crucial.

  1. After setting these parameters you should clear cache. ./bin/console cache:pool:clear should be enough, but for me in qa env it worked even without cache clearing not sure why :)

  2. Try login into suitecrm with ldap account. If all works correctly, you can move to next step and set auto user creation. My config:

> LDAP AUTO CREATE CONFIG

LDAP_AUTO_CREATE=enabled LDAP_PROVIDER_BASE_DN='ou=Users,dc=organization,dc=local' LDAP_PROVIDER_SEARCH_DN='CN=bind,OU=Berlin,OU=IT_department,DC=organization,DC=local' LDAP_PROVIDER_SEARCH_PASSWORD='**' LDAP_PROVIDER_DEFAULT_ROLES=ROLE_USER LDAP_PROVIDER_UID_KEY='' LDAP_PROVIDER_FILTER=''

< LDAP AUTO CREATE CONFIG

> SAML CONFIG

SAML_AUTO_CREATE=disabled SAML_USERNAME_ATTRIBUTE=uid SAML_USE_ATTRIBUTE_FRIENDLY_NAME=true

< SAML CONFIG


a) LDAP_PROVIDER_BASE_DN - also dont forget about base organizational unit (ou=Users), overall same as in ldap config
b) LDAP_PROVIDER_SEARCH_DN - bind, same as in ldap config
c) LDAP_PROVIDER_SEARCH_PASSWORD - bind password, same as in ldap config 
d) LDAP_PROVIDER_DEFAULT_ROLES - i did pass some variable ROLE_USER, didn't implement much into that, so no knowledge here
e) LDAP_PROVIDER_UID_KEY - i left it blank, with just two sigle quotes
f) LDAP_PROVIDER_FILTER - also left blank with two quotes
g) SAML config is not tested so i have it disabled. Configure and play with it at your own responsibility ;)

9. For LDAP AUTO CREATE CONFIG work you need to configure yaml file in ldap service. You can do it in some extension but i did it in default config services. Path to file: _./config/services/ldap/ldap.yaml_

content of file:

parameters: ldap.autocreate.extra_fields_map: givenName: first_name sn: last_name mail: email1 ldap.extra_fields: [ 'givenName', 'sn', 'mail']


This is yaml file, so remember about spaces and formatting. I'm not sure if I describe these correct, but it worked for me. Legend:
a) "givenName" - is AD field for first name. "first_name" is suitecrm field
b) "sn" - is AD field for surname. "last_name" is suitecrm field 
c) "mail" - is AD field where we have email saved for users. "email1" is suitecrm field

That should be all.  After configuring user autocreation should login to your suitecrm instance and have automatically set first name, surname and email for sending some notification or reports.
If i helped you, please give me feedback if it worked. Wish you luck ;)
rmendes-13 commented 2 months ago

Hi @piotrgitt thank you for your message and I'm sorry the delay in providing some feedback. Thank you for providing such a lengthy explanation for LDAP.

Ok so I overcame the issue with LDAP auth, actually I had to rewrite the base, filters and so on, apparently the way it was being ingested before no longer worked so that was it.

But in the meanwhile we simply abandoned SuiteCRM. We really wished this project would get some proper TLC, but apparently whose behind it refuses to add something as simple as VAT to the Core. This is supposed to be a CRM software with a sales module that does not know VAT from core. On the Quotes the values for TAX are calculated based on label

Also I've participated a lot on the forums and there is just too much disorganisation, disengagement from the community and the requests, and a bunch of other stuff just made me lose all interest in Suite. Was fun while it lasted. Cheers.

piotrgitt commented 2 months ago

@rmendes-13 No problem ;) I'm just curious what CRM did you choose to move on? We also are sometimes concidering to move to another CRM software, howewer we don't have problem with missing VAT module.

rmendes-13 commented 2 months ago

We tried a couple and so far the one that's stading out is ERPNext. We tried the docker for a go around and now I'm setting up a server with it using bench and external db server which corresponds to our prod setups.