Can't swap contacts between address books

[phiL0co]

Steps to reproduce

1. Create a second adress book
2. Open a contact and switch the contact to the new address book (choose new address book from list)
3. Close the contact and reopen it -> contact's address book is the old one again.

Expected behaviour

The address book should remain the previously selected

On the phone (after a sync) a "switched" contact should only be in one address book, the address book that is selected in Nextcloud's contact app.

Actual behaviour

The address book always switches back to the first one.

BUT: When I sync the new address book on my phone, suddenly the contact seems to be in both address books, according to the phone. In the browser view, the contact remains in the old address book. So it seems somethings is happening in the background.

After syncing with the phone at some point the contact appears twice in Nextcloud contacts app, one contact in the old address book, the other one in the new address book.

Server configuration

Operating system: Ubuntu 18.04

Web server: Apache

Database: MySQL 5.7.28

PHP version: 7.2.24

Nextcloud version: (see Nextcloud admin page) 17.0.2

Contacts version: (see Nextcloud apps page) 3.1.6

Updated from an older Nextcloud or fresh install: Updated

Signing status: What does this mean?

Login as admin user into your Nextcloud and access http://example.com/index.php/settings/integrity/failed paste the results here.

No errors have been found.

List of activated apps: Accessibility 1.3.0 Activity 2.10.1 Auditing / Logging 1.7.0 Bookmarks 2.3.4 Calendar 2.0.0 Checksum 0.4.3 Collaborative tags 1.7.0 Comments 1.7.0 Contacts 3.1.6 Default encryption module 2.5.0 Deleted files 1.7.0 Draw.io 0.9.4 Federation 1.7.0 File sharing 1.9.0 First run wizard 2.6.0 Gallery 18.4.0 Log Reader 2.2.0 Markdown Editor 2.2.0 Metadata 0.10.0 Monitoring 1.7.0 Nextcloud announcements 1.6.0 Notifications 2.5.0 ocDownloader 1.7.5 Password policy 1.7.0 PDF viewer 1.6.0 Privacy 1.1.0 RainLoop 6.0.4 Recommendations 0.5.0 Right click 0.15.1 Share by mail 1.7.0 Support 1.0.1 Suspicious Login 3.0.1 Tasks 0.11.3 Text 1.1.1 Theming 1.8.0 Two-Factor TOTP Provider 4.1.2 Update notification 1.7.0 Usage survey 1.5.0 Versions 1.10.0 Video player 1.6.0 Viewer 1.2.0

If you have access to your command line run e.g.:
sudo -u www-data php occ app:list
from within your Nextcloud installation folder

Nextcloud configuration:

If you have access to your command line run e.g.:
sudo -u www-data php occ config:list system
from within your instance's installation folder

or

Insert your config.php content here
Make sure to remove all sensitive content such as passwords. (e.g. database password, passwordsalt, secret, smtp password, …)

Are you using external storage, if yes which one: local/smb/sftp/... no

Are you using encryption: yes/no yes

Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/... Only CalDav and CardDav if thats a user-backend

LDAP configuration (delete this part if not used)

With access to your command line run e.g.:
sudo -u www-data php occ ldap:show-config
from within your Nextcloud installation folder

Without access to your command line download the data/owncloud.db to your local
computer or access your SQL server remotely and run the select query:
SELECT * FROM `oc_appconfig` WHERE `appid` = 'user_ldap';

Eventually replace sensitive data as the name/IP-address of your LDAP server or groups.

Client configuration

Browser: Firefox 68.4.1 esr / Firefox 72

Operating system: Windows 10 / Manjaro Linux

CardDAV-clients: DAVx5

[republicus]

I didn't notice, but confirming it on my end now.

[skjnldsv]

I will need a screenshot of your network requests. Before you do anything, still on the development tools, there is a tab called network. Click on it and then the xhr filter. Do your action and screenshot the network log like this:

I will also need your browser console log to investigate this issue. Open your console, reload your page and/or do the action leading to this issue and copy/paste the log in this thread.

How to access your browser console (Click to expand)

# Chrome - Press either CTRL + SHIFT + J to open the “console” tab of the Developer Tools. - Alternative method: 1. Press either CTRL + SHIFT + I or F12 to open the Developer Tools. 2. Click the “console” tab. # Safari - Press CMD + ALT + I to open the Web Inspector. - See Chrome’s step 2. (Chrome and Safari have pretty much identical dev tools.) # IE9 1. Press F12 to open the developer tools. 2. Click the “console” tab. # Firefox - Press CTRL + SHIFT + K to open the Web console (COMMAND + SHIFT + K on Macs). - or, if Firebug is installed (recommended): 1. Press F12 to open Firebug. 2. Click on the “console” tab. # Opera 1. Press CTRL + SHIFT + I to open Dragonfly. 2. Click on the “console” tab.

[phiL0co]

Hi!

Thx for looking into this! I hope this is what you need! I made the screenshots right after I changed the address book!

Best regards

[skjnldsv]

Surprisingly I see a MOVE request that returned 200, meaning it was successful :thinking:

How many addressbook do you have? Can you maybe do me a screencast of what is happening live?

Does it do that for all contacts? Does it do it for other users?

[phiL0co]

Yes, the contact gets moved to the other address book, but when I return to the contact the old address book is shown. If I reload the page, the contact suddenly appears twice, once in the old, once in the new address book. So if I'd want to change lots of contacts, I'd have to do a "delete run" afterwards, which is pretty cumbersome.

I hope this clarifies things.

I have two address books.

Let me know if you still need that screencast, could be problematic as I don't want to make my contacts public...

edit: I tried with several contacts. All showed the same behaviour.

I'll try it with another user!

[skjnldsv]

Can you try with another new addressbook too?

Let me know if you still need that screencast, could be problematic as I don't want to make my contacts public...

Sure, let's postpone until it's really necessary (you can send me privately so your data is safe)

[phiL0co]

You mean create a contact in a new address book and swap him to a second new address book?

[phiL0co]

I did as described before and now it works as expected:

• Correct address book is shown when I deselect the contact and go back
• No duplicats

[phiL0co]

I tried to export my "problematic" address book and import it into a new address book. During the import of 464 contacts, 181 where shown as faulty.

After the import, i tried changing address books (both new) for several contacts. The result was, that some contacts could be swapped as intended, others where problematic (as described initially, with duplicats etc). So it seems to be contact-specific after all, probably the faulty contacts have the "bad swapping"?!?!

[phiL0co]

It seems that all contacts with a birthday entry "swap bad" and all contacts without a birthday entry swap as intended.

Could this be the reason???

[hanzi]

I've encountered this issue before as well. Forgot to look into it.

If I remember correctly, the issue had something to do with calendar events for a contact (so yes, birthday, but also anniversaries).

My best guess is that the DAV backend treats a moved contact like a new one and tries to create a new calendar event using the UID of that 'new' contact. But since a calendar entry with this UID already exists, it fails. Or something like that.

[phiL0co]

Hey, @skjnldsv!

Anything new concerning this issue? Let me know if you need additional info!

Best regards

[skjnldsv]

Hey sorry! I was crazy busy :) I received your mail but did not had the time to look into it yet :(

[ghost]

This issue has been automatically marked as stale because it has not had recent activity and seems to be missing some essential information. It will be closed if no further activity occurs. Thank you for your contributions.

[phiL0co]

Issue not solved yet. Plz don't close this issue! Thx!

[hrst]

Also affected by this, this is a massive problem as it generates duplicates in the database and does not show birthdays of affected contacts in the calendar, without any warning that something might be wrong.

[ghost]

This issue has been automatically marked as stale because it has not had recent activity and seems to be missing some essential information. It will be closed if no further activity occurs. Thank you for your contributions.

[phiL0co]

Not solved yet... Thx!

[skjnldsv]

This is my monthly reminder xD :see_no_evil:

[skjnldsv]

@phiL0co can you reset my password & test account please? Cannot access it it seems :thinking:

[phiL0co]

I didn' create a test account for you. At least not yet... ;-) I can create one if that can help you...

[skjnldsv]

Damn, I mixed up the issues xD

[skjnldsv]

Okay, so can yuo try the nextcloud/contacts#1579 test release there please? Then do be a screenshot of the error in the browser log again after you cleared your cache and triggered the issue.

Same as https://github.com/nextcloud/contacts/issues/1431#issuecomment-576639643, it was what I need

[phiL0co]

I hope this is what you need!

[skjnldsv]

Click the 400 error in the network tab and copy the response here please :)

[phiL0co]

[skjnldsv]

The response from the response tab please :grin: Thank for taking the time btw, really helps!

[phiL0co]

No problem, thank YOU!

[skjnldsv]

So you have a duplicate :) Is your contact already existing in the other addressbook?

[phiL0co]

No, he isn't. The thing is, when I try to switch the contact it's not switched but a duplicate is created. The contact then exists twice, one for each adress book.

[skjnldsv]

Okay, you see the url you currently have, it should be something like yourdomain.com/apps/contacts/group/addressbookid~contactsuid

Change the addressbookid from default (it seems to be that) to the other one. Does it display a contact? (you can screenshot the very bottom of the headers tab for me as well as the params tab too please?)

[blip-unicorn]

@skjnldsv No intention of disrespect but did you notice that it complains about duplicate in calendar, not in contacts.

As stated above this problem only seems to happen with contacts that have birthday information.

[skjnldsv]

No worries! :) I indeed assumed that it was a message error, but it might be because of the birthday calendar :thinking: @georgehrke ?

[georgehrke]

Yes, that looks Birthday calendar related. Let's move this to the server.

[phiL0co]

Okay, you see the url you currently have, it should be something like yourdomain.com/apps/contacts/group/addressbookid~contactsuid

Change the addressbookid from default (it seems to be that) to the other one. Does it display a contact? (you can screenshot the very bottom of the headers tab for me as well as the params tab too please?)

I can click on the duplicate that's not in the default address book, then the URL switches to what you described. But it's the other way around, first the contact ID (cryptic) and then the name of the address book. (.../index.php/apps/contacts/Alle Kontakte/d88aebce-592d-46f0-a1ff-e11db67f28d0~default)

I noted, that the contact ID is the same in each address book: .../index.php/apps/contacts/Alle Kontakte/d88aebce-592d-46f0-a1ff-e11db67f28d0~default

.../index.php/apps/contacts/Alle Kontakte/d88aebce-592d-46f0-a1ff-e11db67f28d0~jeph_gemeinsam

That's what the 200-entry says in headers:

Nothing in the params box!

(I didn't switch a contact this time because to do what you asked I had to use the duplicate (to make the URL switch). And as long as that exists, I can't copy the contact again. I then get an error message like "contact already exists...". )

[phiL0co]

Another thing I should add that I didn't know in my initial opening post:

When I switch a contact from one address book to another, at first it seems the switch was successful. The contact is only shown once in the new address book. But when I reload the page the contact then appears twice in the list, once in every address book.

So the sync with the phone is definitely not the reason for the duplicates, which I suspected at that time.

[cweiske]

On Nextcloud 19.0.1 the error message is different:

MOVE /remote.php/dav/addressbooks/users/admin/contacts/12DE8587-2859-4349-A5D5-7EEB89051A71.vcf

400 Bad Request

<?xml version="1.0" encoding="utf-8"?>
<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">
  <s:exception>Sabre\DAV\Exception\BadRequest</s:exception>
  <s:message>Calendar object with uid already exists in this calendar collection.</s:message>
</d:error>

Exception stack trace:

{
    "app": "webdav",
    "method": "MOVE",
    "url": "/remote.php/dav/addressbooks/users/admin/contacts/12DE8587-2859-4349-A5D5-7EEB89051A71.vcf",
    "message": {
        "Exception": "Sabre\\DAV\\Exception\\BadRequest",
        "Message": "Calendar object with uid already exists in this calendar collection.",
        "Code": 0,
        "Trace": [
            {
                "file": "/home/cweiske/dev/tools/nextcloud/apps/dav/lib/CalDAV/BirthdayService.php",
                "line": 384,
                "function": "createCalendarObject",
                "class": "OCA\\DAV\\CalDAV\\CalDavBackend",
                "type": "->"
            },
            {
                "file": "/home/cweiske/dev/tools/nextcloud/apps/dav/lib/CalDAV/BirthdayService.php",
                "line": 125,
                "function": "updateCalendar",
                "class": "OCA\\DAV\\CalDAV\\BirthdayService",
                "type": "->",
                "args": [
                    "*** sensitive parameters replaced ***"
                ]
            },
            {
                "file": "/home/cweiske/dev/tools/nextcloud/apps/dav/lib/AppInfo/Application.php",
                "line": 133,
                "function": "onCardChanged",
                "class": "OCA\\DAV\\CalDAV\\BirthdayService",
                "type": "->"
            },
            {
                "file": "/home/cweiske/dev/tools/nextcloud/3rdparty/symfony/event-dispatcher/EventDispatcher.php",
                "line": 264,
                "function": "OCA\\DAV\\AppInfo\\{closure}",
                "class": "OCA\\DAV\\AppInfo\\Application",
                "type": "->",
                "args": [
                    "*** sensitive parameters replaced ***"
                ]
            },
            {
                "file": "/home/cweiske/dev/tools/nextcloud/3rdparty/symfony/event-dispatcher/EventDispatcher.php",
                "line": 239,
                "function": "doDispatch",
                "class": "Symfony\\Component\\EventDispatcher\\EventDispatcher",
                "type": "->"
            },
            {
                "file": "/home/cweiske/dev/tools/nextcloud/3rdparty/symfony/event-dispatcher/EventDispatcher.php",
                "line": 73,
                "function": "callListeners",
                "class": "Symfony\\Component\\EventDispatcher\\EventDispatcher",
                "type": "->"
            },
            {
                "file": "/home/cweiske/dev/tools/nextcloud/lib/private/EventDispatcher/SymfonyAdapter.php",
                "line": 84,
                "function": "dispatch",
                "class": "Symfony\\Component\\EventDispatcher\\EventDispatcher",
                "type": "->"
            },
            {
                "file": "/home/cweiske/dev/tools/nextcloud/apps/dav/lib/CardDAV/CardDavBackend.php",
                "line": 668,
                "function": "dispatch",
                "class": "OC\\EventDispatcher\\SymfonyAdapter",
                "type": "->"
            },
            {
                "file": "/home/cweiske/dev/tools/nextcloud/3rdparty/sabre/dav/lib/CardDAV/AddressBook.php",
                "line": 147,
                "function": "createCard",
                "class": "OCA\\DAV\\CardDAV\\CardDavBackend",
                "type": "->"
            },
            {
                "file": "/home/cweiske/dev/tools/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php",
                "line": 311,
                "function": "createFile",
                "class": "Sabre\\CardDAV\\AddressBook",
                "type": "->"
            },
            {
                "file": "/home/cweiske/dev/tools/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php",
                "line": 135,
                "function": "copyNode",
                "class": "Sabre\\DAV\\Tree",
                "type": "->"
            },
            {
                "file": "/home/cweiske/dev/tools/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php",
                "line": 167,
                "function": "copy",
                "class": "Sabre\\DAV\\Tree",
                "type": "->"
            },
            {
                "file": "/home/cweiske/dev/tools/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php",
                "line": 641,
                "function": "move",
                "class": "Sabre\\DAV\\Tree",
                "type": "->"
            },
            {
                "file": "/home/cweiske/dev/tools/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php",
                "line": 89,
                "function": "httpMove",
                "class": "Sabre\\DAV\\CorePlugin",
                "type": "->"
            },
            {
                "file": "/home/cweiske/dev/tools/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
                "line": 474,
                "function": "emit",
                "class": "Sabre\\DAV\\Server",
                "type": "->"
            },
            {
                "file": "/home/cweiske/dev/tools/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
                "line": 251,
                "function": "invokeMethod",
                "class": "Sabre\\DAV\\Server",
                "type": "->"
            },
            {
                "file": "/home/cweiske/dev/tools/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
                "line": 319,
                "function": "start",
                "class": "Sabre\\DAV\\Server",
                "type": "->"
            },
            {
                "file": "/home/cweiske/dev/tools/nextcloud/apps/dav/lib/Server.php",
                "line": 325,
                "function": "exec",
                "class": "Sabre\\DAV\\Server",
                "type": "->"
            },
            {
                "file": "/home/cweiske/dev/tools/nextcloud/apps/dav/appinfo/v2/remote.php",
                "line": 35,
                "function": "exec",
                "class": "OCA\\DAV\\Server",
                "type": "->"
            },
            {
                "file": "/home/cweiske/dev/tools/nextcloud/remote.php",
                "line": 167,
                "args": [
                    "/home/cweiske/dev/tools/nextcloud/apps/dav/appinfo/v2/remote.php"
                ],
                "function": "require_once"
            }
        ],
        "File": "/home/cweiske/dev/tools/nextcloud/apps/dav/lib/CalDAV/CalDavBackend.php",
        "Line": 1066,
        "CustomMessage": "--"
    },
    "version": "19.0.1.1"
}

[cweiske]

The problem is as follows:

1. When a vcard changes, "onCardChanged" is called on the BirthdayService, so that the birthdate can be stored in the calendar.
2. BirthdayService::updateCalendar checks if the calendar entry already exists with CalDavBackEnd::getCalendarObject($calendarId, $objectUri). That one says "no, does not exist", because the $objectUri is at the new calendar.
3. BirthdayService::updateCalendar tries to create a new calendar entry with CalDavBackEnd::createCalendarObject($calendarId, $objectUri, $calendarData->serialize()). That fails, because createCalendarObject first checks if an calendar entry with that UID (not uri!) already exists, and throws an exception here.

I am not sure what the right solution would be here. Step 2 could use the UID to check if the calendar entry exists instead of the URI, but that may have side effects I do not know of.

[cweiske]

The problem is not that the card is changed; moving a card to another address book means that 1. a new card is created and 2. the old card is deleted. This explains the duplicate address book entries after the crash.

This also means that we cannot see that a card is being moved; we must handle the duplicate calendar entry problem before the old one gets deleted.