Unable to open certain Contacts (Leads)

[PublyaDev]

[//]: # ( Invisible comment: IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII Before you create the issue: IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII Search for similar report among other reported issues. Learn how to troubleshoot at https://www.mautic.org/docs/en/tips/troubleshooting.html Use drag&drop to attach images or other files )

Bug Description

The issue started when we were unable to open certain contacs. It returns Internal Server Error 500. I happens with some contacts, not all.

The logs have nothing as far as I know, but I'll put what appeared there in the logs section of this report.

Also checking mysql returned nothing in terms of results as comparing a working contact with a broken one yielded nothing:

+------+----------+----------+--------------+---------------------+------------+---------- -------+---------------------+-------------+------------------+-------------+------------- ---+---------------------+--------+---------------------+----------+--------------+------- --------------+-------------------------+-------+-----------+----------+------------------ +----------+----------------------------------+-----------+--------+----------+----------+ ---------------+-------------------+---------+-------------------+---------+------+------- -----------+------------------+-------------+---------+----------+------------+----------- -+-----------+----------+-------+---------+----------------+------------+ | id | owner_id | stage_id | is_published | date_added | created_by | created_b y_user | date_modified | modified_by | modified_by_user | checked_out | checkedout by | checked_out_by_user | points | last_active | internal | social_cache | date_i dentified | preferred_profile_image | title | firstname | lastname | company
| position | email | phone | mobile | address1 | address2 | city | state | zipcode | timezone | country | fax | prefer red_locale | attribution_date | attribution | website | facebook | foursquare | googleplus | instagram | linkedin | skype | twitter | setornaempresa | leadsource | +------+----------+----------+--------------+---------------------+------------+---------- -------+---------------------+-------------+------------------+-------------+------------- ---+---------------------+--------+---------------------+----------+--------------+------- --------------+-------------------------+-------+-----------+----------+------------------ +----------+----------------------------------+-----------+--------+----------+----------+ ---------------+-------------------+---------+-------------------+---------+------+------- -----------+------------------+-------------+---------+----------+------------+----------- -+-----------+----------+-------+---------+----------------+------------+ 221 | NULL | 1 | 1 | 2019-05-28 19:52:09 | NULL |
| 2019-06-02 02:07:02 | NULL | | NULL | NU LL | NULL | 0 | 2301-09-18 15:57:52 | a:0:{} | a:0:{} | 2019-0 5-28 19:52:09 | gravatar | NULL | Daniela E | Medina | Claro Chile S.A. | NULL | | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | Chile | NULL | NULL
| NULL | NULL | NULL | NULL | NULL | NULL
| NULL | NULL | NULL | NULL | NULL | NULL | | 7804 | NULL | NULL | 1 | 2019-10-11 05:33:31 | NULL | NULL
| 2019-11-26 17:17:20 | NULL | | NULL | NU LL | NULL | 20 | 2019-11-26 17:17:20 | a:0:{} | a:0:{} | 2019-1 0-17 02:51:13 | gravatar | NULL | Elias | NULL | Samba
| Diretor | | | NULL | NULL | NULL | Caxias do Sul | Rio Grande do Sul | NULL | America/Sao_Paulo | Brazil | NULL | NULL
| NULL | NULL | NULL | NULL | NULL | NULL
| NULL | NULL | NULL | NULL | Marketing | NULL | +------+----------+----------+--------------+---------------------+------------+---------- -------+---------------------+-------------+------------------+-------------+------------- ---+---------------------+--------+---------------------+----------+--------------+------- --------------+-------------------------+-------+-----------+----------+------------------ +----------+----------------------------------+-----------+--------+----------+----------+ ---------------+-------------------+---------+-------------------+---------+------+------- -----------+------------------+-------------+---------+----------+------------+----------- -+-----------+----------+-------+---------+----------------+------------+

ID 221 is opening fine, ID 7804 won't open.

My mautic installation has an integration with hubspot, where id 221 came from the integration and id 7804 converted from a creative.

Editing a contact opens the editing page and saving modified information works, but still returns a 500 error on opening the contact page.

Q	A
Mautic version	2.15.3
PHP version	7.2
Browser	Firefox

Steps to reproduce

1. Have mautic Integrated with hubspot
2. Have some contacts be added through integration and other means like converting on a creative
3. Try and open a contact

Log errors

[2019-11-27 09:50:10] mautic.WARNING: PHP Warning - require(/var/www/html/mautic/app/cache/prod/doctrine/orm/Proxies/CGMauticStageBundleEntityStage.php): failed to open stream: No such file or directory - in file /var/www/html/mautic/vendor/doctrine/common/lib/Doctrine/Common/Proxy/AbstractProxyFactory.php - at line 209 {"className":"Mautic\StageBundle\Entity\Stage","classMetadata":"[object] (Doctrine\ORM\Mapping\ClassMetadata: Doctrine\ORM\Mapping\ClassMetadataInfo@0000000028fdfe1a00000000062db91f)","proxyClassName":"Proxies\CG\Mautic\StageBundle\Entity\Stage","fileName":"/var/www/html/mautic/app/cache/prod/doctrine/orm/Proxies/CGMauticStageBundleEntityStage.php"} [] [2019-11-27 09:50:10] mautic.ERROR: PHP Notice: require(): Failed opening required '/var/www/html/mautic/app/cache/prod/doctrine/orm/Proxies/CGMauticStageBundleEntityStage.php' (include_path='.:/usr/share/php') - in file /var/www/html/mautic/vendor/doctrine/common/lib/Doctrine/Common/Proxy/AbstractProxyFactory.php - at line 209 [] [] [2019-11-27 09:50:10] mautic.ERROR: Symfony\Component\Debug\Exception\FatalErrorException: Notice: require(): Failed opening required '/var/www/html/mautic/app/cache/prod/doctrine/orm/Proxies/CGMauticStageBundleEntityStage.php' (include_path='.:/usr/share/php') - in file /var/www/html/mautic/vendor/doctrine/common/lib/Doctrine/Common/Proxy/AbstractProxyFactory.php - at line 209 [] [] [2019-11-27 09:50:10] mautic.CRITICAL: Uncaught PHP Exception Symfony\Component\Debug\Exception\FatalErrorException: "Compile Error: require(): Failed opening required '/var/www/html/mautic/app/cache/prod/doctrine/orm/Proxies/CGMauticStageBundleEntityStage.php' (include_path='.:/usr/share/php')" at /var/www/html/mautic/vendor/doctrine/common/lib/Doctrine/Common/Proxy/AbstractProxyFactory.php line 209 {"exception":"[object] (Symfony\Component\Debug\Exception\FatalErrorException(code: 0): Compile Error: require(): Failed opening required '/var/www/html/mautic/app/cache/prod/doctrine/orm/Proxies/CGMauticStageBundleEntityStage.php' (include_path='.:/usr/share/php') at /var/www/html/mautic/vendor/doctrine/common/lib/Doctrine/Common/Proxy/AbstractProxyFactory.php:209)"} [] [2019-11-27 09:50:48] mautic.CRITICAL: Uncaught PHP Exception InvalidArgumentException: "The string a:11:{s:10:"page_title";s:76:"Agradecimento: Minicurso de Midia Programatica - Publya - Midia Programatica";s:13:"page_language";s:5:"pt-BR";s:13:"page_referrer";s:0:"";s:8:"page_url";s:66:"/";s:7:"counter";s:1:"0";s:10:"resolution";s:8:"1440x900";s:15:"timezone_offset";s:3:"180";s:8:"platform";s:8:"MacIntel";s:12:"do_not_track";s:7:"unknown";s:7:"adblock";s:5:"false";s:11:"fingerprint";s:32:"75615dea4d7440774941b7d80fcf8457";} contains an object." at /var/www/html/mautic/app/bundles/CoreBundle/Helper/Serializer.php line 32 {"exception":"[object] (InvalidArgumentException(code: 0): The string a:11:{s:10:\"page_title\";s:76:\"Agradecimento: Minicurso de Midia Programatica - Publya - Midia Programatica\";s:13:\"page_language\";s:5:\"pt-BR\";s:13:\"page_referrer\";s:0:\"\";s:8:\"page_url\";s:66:\"/\";s:7:\"counter\";s:1:\"0\";s:10:\"resolution\";s:8:\"1440x900\";s:15:\"timezone_offset\";s:3:\"180\";s:8:\"platform\";s:8:\"MacIntel\";s:12:\"do_not_track\";s:7:\"unknown\";s:7:\"adblock\";s:5:\"false\";s:11:\"fingerprint\";s:32:\"75615dea4d7440774941b7d80fcf8457\";} contains an object. at /var/www/html/mautic/app/bundles/CoreBundle/Helper/Serializer.php:32)"} [] [2019-11-27 09:53:07] mautic.CRITICAL: Uncaught PHP Exception InvalidArgumentException: "The string a:11:{s:10:"page_title";s:76:"Agradecimento: Minicurso de Midia Programatica - Publya - Midia Programatica";s:13:"page_language";s:5:"pt-BR";s:13:"page_referrer";s:0:"";s:8:"page_url";s:66:"/";s:7:"counter";s:1:"0";s:10:"resolution";s:8:"1440x900";s:15:"timezone_offset";s:3:"180";s:8:"platform";s:8:"MacIntel";s:12:"do_not_track";s:7:"unknown";s:7:"adblock";s:5:"false";s:11:"fingerprint";s:32:"75615dea4d7440774941b7d80fcf8457";} contains an object." at /var/www/html/mautic/app/bundles/CoreBundle/Helper/Serializer.php line 32 {"exception":"[object] (InvalidArgumentException(code: 0): The string a:11:{s:10:\"page_title\";s:76:\"Agradecimento: Minicurso de Midia Programatica - Publya - Midia Programatica\";s:13:\"page_language\";s:5:\"pt-BR\";s:13:\"page_referrer\";s:0:\"\";s:8:\"page_url\";s:66:\"/\";s:7:\"counter\";s:1:\"0\";s:10:\"resolution\";s:8:\"1440x900\";s:15:\"timezone_offset\";s:3:\"180\";s:8:\"platform\";s:8:\"MacIntel\";s:12:\"do_not_track\";s:7:\"unknown\";s:7:\"adblock\";s:5:\"false\";s:11:\"fingerprint\";s:32:\"75615dea4d7440774941b7d80fcf8457\";} contains an object. at /var/www/html/mautic/app/bundles/CoreBundle/Helper/Serializer.php:32)"} [] [2019-11-27 09:53:10] mautic.CRITICAL: Uncaught PHP Exception InvalidArgumentException: "The string a:11:{s:10:"page_title";s:76:"Agradecimento: Minicurso de Midia Programatica - Publya - Midia Programatica";s:13:"page_language";s:5:"pt-BR";s:13:"page_referrer";s:0:"";s:8:"page_url";s:66:"/";s:7:"counter";s:1:"0";s:10:"resolution";s:8:"1440x900";s:15:"timezone_offset";s:3:"180";s:8:"platform";s:8:"MacIntel";s:12:"do_not_track";s:7:"unknown";s:7:"adblock";s:5:"false";s:11:"fingerprint";s:32:"75615dea4d7440774941b7d80fcf8457";} contains an object." at /var/www/html/mautic/app/bundles/CoreBundle/Helper/Serializer.php line 32 {"exception":"[object] (InvalidArgumentException(code: 0): The string a:11:{s:10:\"page_title\";s:76:\"Agradecimento: Minicurso de Midia Programatica - Publya - Midia Programatica\";s:13:\"page_language\";s:5:\"pt-BR\";s:13:\"page_referrer\";s:0:\"\";s:8:\"page_url\";s:66:\"/\";s:7:\"counter\";s:1:\"0\";s:10:\"resolution\";s:8:\"1440x900\";s:15:\"timezone_offset\";s:3:\"180\";s:8:\"platform\";s:8:\"MacIntel\";s:12:\"do_not_track\";s:7:\"unknown\";s:7:\"adblock\";s:5:\"false\";s:11:\"fingerprint\";s:32:\"75615dea4d7440774941b7d80fcf8457\";} contains an object. at /var/www/html/mautic/app/bundles/CoreBundle/Helper/Serializer.php:32)"} [] [2019-11-27 09:56:44] mautic.CRITICAL: Uncaught PHP Exception InvalidArgumentException: "The string a:11:{s:10:"page_title";s:76:"Agradecimento: Minicurso de Midia Programatica - Publya - Midia Programatica";s:13:"page_language";s:5:"pt-BR";s:13:"page_referrer";s:0:"";s:8:"page_url";s:66:"/";s:7:"counter";s:1:"0";s:10:"resolution";s:8:"1440x900";s:15:"timezone_offset";s:3:"180";s:8:"platform";s:8:"MacIntel";s:12:"do_not_track";s:7:"unknown";s:7:"adblock";s:5:"false";s:11:"fingerprint";s:32:"75615dea4d7440774941b7d80fcf8457";} contains an object." at /var/www/html/mautic/app/bundles/CoreBundle/Helper/Serializer.php line 32 {"exception":"[object] (InvalidArgumentException(code: 0): The string a:11:{s:10:\"page_title\";s:76:\"Agradecimento: Minicurso de Midia Programatica - Publya - Midia Programatica\";s:13:\"page_language\";s:5:\"pt-BR\";s:13:\"page_referrer\";s:0:\"\";s:8:\"page_url\";s:66:\"/\";s:7:\"counter\";s:1:\"0\";s:10:\"resolution\";s:8:\"1440x900\";s:15:\"timezone_offset\";s:3:\"180\";s:8:\"platform\";s:8:\"MacIntel\";s:12:\"do_not_track\";s:7:\"unknown\";s:7:\"adblock\";s:5:\"false\";s:11:\"fingerprint\";s:32:\"75615dea4d7440774941b7d80fcf8457\";} contains an object. at /var/www/html/mautic/app/bundles/CoreBundle/Helper/Seria

[//]: # ( Invisible comment: Please check for related errors in the latest log file in [mautic root]/app/log/ and/or the web server's logs and post them here. Be sure to remove sensitive information if applicable. )

[dcweeks]

This is being caused by unescaped colons (maybe also other special characters) in the page title. The JS tracking code is not sanitizing this nor, apparently, is the code that is ingesting the data or the code that is using it (it should be able to figure out a colon is not a problem in that location — it is not a problem in the URL field).

unescaped colon after VIDEO in "query" field in page_hits mautic table. It is also in url_title field but doesn't seem to matter there.

a:12:{s:10:"page_title";s:92:"VIDEO: Machete Wielding Suspect Chasing Cop is Shot in Hollywood - Daily News";s:13:"page_language";s:5:"en-US";s:13:"page_referrer";s:0:"";s:8:"page_url";s:98:"https://XXXXXXXX/news/officer-involved-shooting-closing-down-streets-in-hollywood-update/";s:7:"counter";s:1:"0";s:16:"mautic_device_id";s:23:"bgffvqb56z9vs3gb8x0zrg4";s:10:"resolution";s:7:"667x375";s:15:"timezone_offset";s:3:"480";s:8:"platform";s:6:"iPhone";s:12:"do_not_track";s:7:"unknown";s:7:"adblock";s:5:"false";s:11:"fingerprint";s:32:"e6c4628e0db578cfd756be300362ca26";}

I spot checked a half dozen users with the colons in the page title in their hits and sure enough, they all failed to open. Removed the colon manually from one user and it fixed the problem.

[PublyaDev]

Might have closed it a little bit too early, even though I understand the problem I do not know how to apply it... Where is this information to be changed? Since I can't open the profile of any user which has a problem. I checked in the database and the pages table is empty...

[dcweeks]

This is a bug and we need someone to tag it to be addressed. The system needs to handle a colon in this spot like it does for the URL but it does not.

[dcweeks]

for a temporary fix of corrupted records, you would look for colon in "query" field in page_hits mautic table. It is also in url_title field but doesn't seem to matter there. You need to remove the colons or escape them. But the problem will continue as long as new hits are added with more colons

[PublyaDev]

Got it, thanks! Will circumvent this for now by scheduling a script to escape all colons in this field every day... Thanks again! Have a wonderful day!

[PublyaDev]

What is the correct way to escape a colon in mysql? For the record and anyone having this problem as well I'm attempting to run this query daily as a workaround: UPDATE page_hits SET query = replace(query,":"."\\:");

[PublyaDev]

Update: Tested replace with \: and ;, both made it work, but now there are other errors in the log: [2019-12-02 16:37:41] mautic.NOTICE: PHP Notice - unserialize(): Error at offset 0 of 427 bytes - in file /var/www/html/mautic/app/bundles/CoreBundle/Helper/Serializer.php - at line 39 {"serializedString":"a;11;{s;10;\"page_title\";s;57;\"Publya - Trading Desk Especializada em Midia Programatica\";s;13;\"page_language\";s;5;\"pt-BR\";s;13;\"page_referrer\";s;0;\"\";s;8;\"page_url\";s;19;\"https;//publya.com/\";s;7;\"counter\";s;1;\"0\";s;10;\"resolution\";s;8;\"1366x768\";s;15;\"timezone_offset\";s;3;\"180\";s;8;\"platform\";s;5;\"Win32\";s;12;\"do_not_track\";s;7;\"unknown\";s;7;\"adblock\";s;5;\"false\";s;11;\"fingerprint\";s;32;\"3ac3e85f38fc078e2149ea3ac7cdee84\";}","options":{"allowed_classes":false}} []

[dcweeks]

You might want to chase that down as a separate issue, assuming escaping the colon fixed the issue with being able to open contacts. That was a critical exception and this other is only a warning. There is one well defined bug here for a developer to address about the unescaped colon and inability to open contacts. So unless this warning is related then it probably needs to be defined separately. I'm new and not an expert on process and procedures around here but that is my guess.

[PublyaDev]

Oh ok, thanks again! Have an awesome week!

[stevedrobinson]

Seeing the same issue.

[stevedrobinson]

Here's the exact SQL Query I'm putting in a cron job until this bug can be fixed:

update page_hits set query = regexp_replace(query,"(.*\"page_title\";s:\\d+:\"[^\\\\\"]*):(.*)","\\1\\\\:\\2") WHERE query regexp "\"page_title\";s:\\d+:\"[^\\\\\"]*:[^\"]*\"";

and if anyone is curious how to do this securely, you'll need to create a mysql configuration file (my.cnf below) with your mautic database credentials in it and then an sql file with the above code in it and then schedule the following command on your server:

cat /path/to/script/page_title_fix.sql | /usr/bin/mysql --defaults-file=/path/to/config/my.cnf

Don't forget to set the permissions on my.cnf to 600 so that only your cron user has access

[tiagorodriguesdev]

Same here

[npracht]

Hi @PublyaDev @dcweeks @stevedrobinson @TiagoRodrigues could you test https://github.com/mautic/mautic/pull/8266

[moerazem]

I tested the patch #8266 and I can confirm it fixed this issue.