Contacts no longer being displayed

[darose]

I'm not sure when or how it broke, but the Contacts in my roundcube installation are no longer working:

• When I compose a new message, and start to type a contact's name in order to invoke auto-complete of their email address, it no longer auto-completes.

• When I look in my Contacts, my contacts are listed. But the entries contain no data.
  

Both of these pieces of functionality were definitely working until fairly recently.

The only clue I'm seeing as to what might be wrong is messages like the following in the roundcubemail/errors log file:

[09-May-2019 22:34:32 UTC] PHP Warning: preg_replace(): Compilation failed: escape sequence is invalid in character class at offset 8 in /usr/share/webapps/roundcubemail/program/lib/Roundcube/rcube_vcard.php on line 547

I'm on Arch Linux, running the following versions of roundcube and plugins: roundcubemail 1.3.9-1 roundcubemail-plugin-chbox 0.2.4-2 roundcubemail-plugin-contextmenu 2.3-1 roundcubemail-plugin-jquery-mobile 1.1-1 roundcubemail-plugin-keyboard-shortcuts-ng 0.9.4-1 roundcubemail-plugin-markasjunk2 1.11.1-1 roundcubemail-plugin-mobile 0.4.8-1

Any idea what the issue might be, and how to fix?

[alecpl]

Try with disabled all plugins. What PHP version? Could you show lines 530-550 in rcube_vcard?

[darose]

php 7.3.5

Disabling all plugins didn't remove problem.

Relevant code is as follows:

/**
 * Normalize vcard data for better parsing
 *
 * @param string vCard block
 *
 * @return string Cleaned vcard block
 */
public static function cleanup($vcard)
{
    // convert Apple X-ABRELATEDNAMES into X-* fields for better compatibility
    $vcard = preg_replace_callback(
        '/item(\d+)\.(X-ABRELATEDNAMES)([^:]*?):(.*?)item\1.X-ABLabel:(?:_\$!<)?([\w() -]*)(?:>!\$_)?./s',
        array('self', 'x_abrelatednames_callback'),
        $vcard);

    // Cleanup
    $vcard = preg_replace(array(
            // convert special types (like Skype) to normal type='skype' classes with this simple regex ;)
            '/item(\d+)\.(TEL|EMAIL|URL)([^:]*?):(.*?)item\1.X-ABLabel:(?:_\$!<)?([\w() -]*)(?:>!\$_)?./si',
            '/^item\d*\.X-AB.*$/mi',  // remove cruft like item1.X-AB*
            '/^item\d*\./mi',         // remove item1.ADR instead of ADR
            '/\n+/',                 // remove empty lines
            '/^(N:[^;\R]*)$/m',      // if N doesn't have any semicolons, add some
        ),
        array(
            '\2;type=\5\3:\4',
            '',
            '',
            "\n",
            '\1;;;;',
        ), $vcard);

    // convert X-WAB-GENDER to X-GENDER
    if (preg_match('/X-WAB-GENDER:(\d)/', $vcard, $matches)) {
        $value = $matches[1] == '2' ? 'male' : 'female';
        $vcard = preg_replace('/X-WAB-GENDER:\d/', 'X-GENDER:' . $value, $vcard);
    }

    return $vcard;
}

[alecpl]

I cannot reproduce with PHP 7.3.5.

I thought it's #6398, but it actually was a different warning. Could you check if it works when you replace /^(N:[^;\R]*)$/m with /^(N:[^;\r\n]*)$/m?

[darose]

Looks like that fixed it! Tnx!

[alecpl]

Need to be fix upstream ;) Reopened.

[alecpl]

I'm trying to reproduce, but with no luck. It may be a PHP bug. What platform are you using? What php package version precisely? Could you provide output of php -i | grep -i pcre?

[alecpl]

Looks like indeed \R is not allowed in character class. Fixed.

[darose]

I'm trying to reproduce, but with no luck. It may be a PHP bug. What platform are you using? What php package version precisely? Could you provide output of php -i | grep -i pcre?

As I wrote above:

• Arch Linux

• php version 7.3.5

• roundcubemail version 1.3.9

$ php -i | grep -i pcre
Configure Command =>  './configure'  '--srcdir=../php-7.3.5' '--config-cache' '--prefix=/usr' '--sbindir=/usr/bin' '--sysconfdir=/etc/php' '--localstatedir=/var' '--with-layout=GNU' '--with-config-file-path=/etc/php' '--with-config-file-scan-dir=/etc/php/conf.d' '--disable-rpath' '--mandir=/usr/share/man' '--without-pear' '--enable-cgi' '--enable-fpm' '--with-fpm-systemd' '--with-fpm-acl' '--with-fpm-user=http' '--with-fpm-group=http' '--enable-embed=shared' '--enable-bcmath=shared' '--enable-calendar=shared' '--enable-dba=shared' '--enable-exif=shared' '--enable-ftp=shared' '--enable-intl=shared' '--enable-mbstring' '--enable-shmop=shared' '--enable-soap=shared' '--enable-sockets=shared' '--enable-sysvmsg=shared' '--enable-sysvsem=shared' '--enable-sysvshm=shared' '--enable-zip=shared' '--with-bz2=shared' '--with-curl=shared' '--with-db4=/usr' '--with-enchant=shared,/usr' '--with-freetype-dir=/usr' '--with-gd=shared,/usr' '--with-gdbm' '--with-gettext=shared' '--with-gmp=shared' '--with-iconv=shared' '--with-imap-ssl' '--with-imap=shared' '--with-kerberos=/usr' '--with-ldap=shared' '--with-ldap-sasl' '--with-libzip' '--with-mhash' '--with-mysql-sock=/run/mysqld/mysqld.sock' '--with-mysqli=shared,mysqlnd' '--with-openssl' '--with-password-argon2' '--with-pcre-regex=/usr' '--with-pdo-dblib=shared,/usr' '--with-pdo-mysql=shared,mysqlnd' '--with-pdo-odbc=shared,unixODBC,/usr' '--with-pdo-pgsql=shared' '--with-pdo-sqlite=shared,/usr' '--with-pgsql=shared' '--with-pspell=shared' '--with-readline' '--with-snmp=shared' '--with-sodium=shared' '--with-sqlite3=shared,/usr' '--with-tidy=shared' '--with-unixODBC=shared,/usr' '--with-xmlrpc=shared' '--with-xsl=shared' '--with-zlib' '--enable-pcntl'
pcre
PCRE (Perl Compatible Regular Expressions) Support => enabled
PCRE Library Version => 10.33 2019-04-16
PCRE Unicode Version => 11.0.0
PCRE JIT Support => enabled
PCRE JIT Target => x86 64bit (little endian + unaligned)
pcre.backtrack_limit => 1000000 => 1000000
pcre.jit => 1 => 1
pcre.recursion_limit => 100000 => 100000

[alecpl]

For posterity, I tested with PHP 7.3.5 on Ubuntu.

PCRE (Perl Compatible Regular Expressions) Support => enabled
PCRE Library Version => 10.31 2018-02-12
PCRE Unicode Version => 10.0.0
PCRE JIT Support => enabled
PCRE JIT Target => x86 64bit (little endian + unaligned)
pcre.backtrack_limit => 1000000 => 1000000
pcre.jit => 1 => 1
pcre.recursion_limit => 100000 => 100000
pcre => enabled

[pc-rabo]

I thought it's #6398, but it actually was a different warning. Could you check if it works when you replace /^(N:[^;\R]*)$/m with /^(N:[^;\r\n]*)$/m?

I have the same issue and above regex change fixed it too. Archlinux php 7.3.5, roundcube 1.3.9

[ianbarton]

Just tried altering the regexp on Arch with PHP 7.3.6 roundcube 1.3.9 and it's still broken for me. Can anyone else confirm this?

[darose]

It's you. Works for me with those exact package numbers.

[ianbarton]

OK, it's almost certainly me being stupid:) I edited the function, as shown below, then for good measure rebooted the server. What am I doing wrong?

Roundcube is in: /srv/http/webmail/program/lib/Roundcube/rcube_vcard.php public function load($vcard, $charset = RCUBE_CHARSET, $detect = false)

`$vcard = pregreplace(array(
// convert special types (like Skype) to normal type='skype' classes with this simple regex ;)
'/item(\d+).(TEL|EMAIL|URL)([^:]?):(.?)item\1.X-ABLabel:(?:\$!<)?([\w() -])(?:>!\$_)?./si',
'/^item\d.X-AB.$/mi', // remove cruft like item1.X-AB
'/^item\d./mi', // remove item1.ADR instead of ADR
'/\n+/', // remove empty lines
'/^(N:[^;\r\n])$/m', // if N doesn't have any semicolons, add some

        ),   `

[darose]

I didn't have to touch the load function. I had to touch the cleanup function. This is how it reads:

    public static function cleanup($vcard)
    {
        // convert Apple X-ABRELATEDNAMES into X-* fields for better compatibility
        $vcard = preg_replace_callback(
            '/item(\d+)\.(X-ABRELATEDNAMES)([^:]*?):(.*?)item\1.X-ABLabel:(?:_\$!<)?([\w() -]*)(?:>!\$_)?./s',
            array('self', 'x_abrelatednames_callback'),
            $vcard);

        // Cleanup
        $vcard = preg_replace(array(
                // convert special types (like Skype) to normal type='skype' classes with this simple regex ;)
                '/item(\d+)\.(TEL|EMAIL|URL)([^:]*?):(.*?)item\1.X-ABLabel:(?:_\$!<)?([\w() -]*)(?:>!\$_)?./si',
                '/^item\d*\.X-AB.*$/mi',  // remove cruft like item1.X-AB*
                '/^item\d*\./mi',         // remove item1.ADR instead of ADR
                '/\n+/',                 // remove empty lines
                '/^(N:[^;\r\n]*)$/m',      // if N doesn't have any semicolons, add some
            ),
            array(
                '\2;type=\5\3:\4',
                '',
                '',
                "\n",
                '\1;;;;',
            ), $vcard);

        // convert X-WAB-GENDER to X-GENDER
        if (preg_match('/X-WAB-GENDER:(\d)/', $vcard, $matches)) {
            $value = $matches[1] == '2' ? 'male' : 'female';
            $vcard = preg_replace('/X-WAB-GENDER:\d/', 'X-GENDER:' . $value, $vcard);
        }

        return $vcard;
    }

[ianbarton]

Thanks. Still doesn't work for me. I'll try archiving my current installation and re-install from scratch.

[amishmm]

@ianbarton the fix worked for me too.. so it must be something wrong somewhere in your settings.

Arch linux - PHP 7.3.6 - Roundcube 1.3.9

[ianbarton]

Thanks all. I have just installed the fix on my backup server and works. So there must be something on my main server that is causing the problem.

[amishmm]

Make sure you have not disabled autocomplete in config files. Normally when something doesnt work we try to alter settings and when fix is found - we completely forget to revert the settings to normal. (human tendency)

[ianbarton]

Finally solved this, but not sure what caused the problem. Tried exporting my contacts to as vcard and they were all blank. Ran mysqlcheck which didn't report any problems. Browsed the contacts table with phpmyadmin and I could see all my contact data. If I tried adding back an existing contact using Roundcube, I got a warning about it already existing as I expected.

Finally deleted all my contacts using Roundcube and then added them back. All now working correctly.. Probably an attack by the gremlins.

[linuxuser424]

Hi @ all,

I had the same problem on Debian with php 7.3 from sury.org (#6864). The fix works here, too.

php -i | grep -i pcre pcre PCRE (Perl Compatible Regular Expressions) Support => enabled PCRE Library Version => 10.33 2019-04-16 PCRE Unicode Version => 11.0.0 PCRE JIT Support => enabled PCRE JIT Target => x86 64bit (little endian + unaligned) pcre.backtrack_limit => 1000000 => 1000000 pcre.jit => 1 => 1 pcre.recursion_limit => 100000 => 100000

Thank you.

[GeorgSchlisio]

Can confirm issue on Archlinux with RC 1.3.9 and PHP 7.3.7. Tweaking program/lib/Roundcube/rcube_vcard.php:539 to show \r\n instead of \R fixed the problem for me.

[aommundsen]

I'm trying to reproduce, but with no luck. It may be a PHP bug. What platform are you using? What php package version precisely? Could you provide output of php -i | grep -i pcre?

It seems the issue is only visible when running PCRE2 10.33, but when downgrading to 10.32 it works: https://forum.directadmin.com/showthread.php?t=58745