Contact Import Error: VCard object with uid already exists in this addressbook collection.

[schast]

Describe the bug

Error on importing already existing vcards.

Steps to reproduce

1. Go to contacts -> contact settings
2. Click on contact import
3. Upload File (Toni_Testeroni.vcf):

   BEGIN:VCARD
   VERSION:3.0
   N:Testeroni;Toni;;;
   FN:Toni Testeroni
   TEL;TYPE=home,voice:+43 123 456789
   TEL;TYPE=cell,voice:+43 664 123456789
   ADR;TYPE=home:;;Straße 123-456;Ort;;1234;Land
   URL;TYPE=home:http://toni.testeroni.com
   X-SOCIALPROFILE;TYPE=FACEBOOK:FacebookUser
   X-SOCIALPROFILE;TYPE=SKYPE:SkypeUser
   X-SOCIALPROFILE;TYPE=LINKEDIN:LinkedInUser
   X-SOCIALPROFILE;TYPE=INSTAGRAM:InstagramUser
   X-SOCIALPROFILE;TYPE=MASTODON:MastodonUser
   BDAY:1990-01-01
   EMAIL;TYPE=home:toni.testeroni@mydomain.com
   REV:20240222T081751Z
   UID:urn:uuid:9130b79c-9f59-456e-a96b-bf633fac5882
   END:VCARD

4. The vcard is successfully imported the first time it is created.
5. Upload the card again (the error also occurs with the REV field updated to the current timestamp).
6. See error in Browser Debug Console (Networking Tab)
   • Request:

     PUT https://NEXTCLOUD-SERVER-URL/remote.php/dav/addressbooks/users/NEXTCLOUD-USERNAME/NEXTCLOUD-CALENDAR-NAME/F69E2CA2-A5F4-443D-9478-392AC94847E7.vcf

   • Response:

     <?xml version="1.0" encoding="utf-8"?>
     <d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">
     <s:exception>Sabre\DAV\Exception\BadRequest</s:exception>
     <s:message>VCard object with uid already exists in this addressbook collection.</s:message>
     </d:error>

Expected behavior

Existing vcards shlould be updated.

Actual behavior

Error on import for existing contacts.

Contact version

5.5.2

Operating system

Ubuntu 22.04, Nextcloud Server 28.0.2

PHP engine version

PHP 8.1

Web server

Apache (supported)

Database

MariaDB

Additional info

Without the UID field, the contact will be created twice!

[LionelHoudelier]

Same problem ! Since when does it appear for you?

STRANGE:

1. export a contact vcf file via web UI: afilenameofcontactexport.vcf If the file name in the url is the same of a contact export file, it works with this command:
2. with curl -X PUT -u "$user":"$pass" "https://${cloud}/remote.php/dav/addressbooks/users/$user/$addressbook/afilenameofcontactexport.vcf" -T "$xfile" You can change the content for any contact UID. It works. but with another name, it does not.
3. with curl -X PUT -u "$user":"$pass" "https://${cloud}/remote.php/dav/addressbooks/users/$user/$addressbook/" -T "$xfile" it sends the error "VCard object with uid already exists in this addressbook collection"

[LionelHoudelier]

SOLUTION : extract the UID of file : uid=$(grep UID "$xfile" | sed 's/.*://') add it to the url + ".vcf" then it works PS: we have to have added first the new contacts with this procedure, so that the vcard file name on dav/users/$user/$addressbook is the uid like "uid.vcf", then the change is possible.

[LionelHoudelier]

Here is a full script to add AND update contacts from one file with some vcard:

#!/bin/bash
# YOU NEED TO ADD YOUR CONTACTS WITH THIS SCRIPT TO BE ABLE TO MODIFIY IT AFTERWARDS.
# OR YOU NEED TO GET THE UID OF THE CONTACT AND INSERT IT IN THE URL PATH TO MODIFY AN OLD CONTACT
#### ENTER YOUR DATAS ##########
# folder to parse the file
temp_folder='/home/...'
# folder where is the vcf file
folder='/home/...'
filename='VCARD_FILE.vcf'
user='XXX'
pass='XXXXX'
addressbook='XXXX'
cloud='your.nextcloud.url'
##########################################################
filepath="$folder/$filename"
url="https://${cloud}/remote.php/dav/addressbooks/users/$user/$addressbook"
echo $url
# parse the file in multiple random files name
mkdir -p $temp_folder
rm -f $temp_folder/*
echo "$folder"
inotifywait --format="%w%f" -m "$folder" -e moved_to |
    while read file; do
  if [[ "$file" == *".vcf" ]]
  then 
    awk -v temp_folder="$temp_folder" 'BEGIN {
          RS="END:VCARD";
          }
          {
          command = "echo -n '$temp_folder'/$(pwgen 20 1).vcf"
          command | getline filename
          close(command)
          print $0 "END:VCARD" > filename
          close(filename)
          }' "${filepath}"
    for xfile in "$temp_folder"/*.vcf; do
      if grep -q "BEGIN:" "$xfile"; then
        uid=$(grep UID "$xfile"  | sed 's/.*://')
        echo "UID : " "$uid"
        curl -X PUT -u "$user":"$pass" "$url/$uid.vcf" -T "$xfile"
        echo "ok"
        rm "$xfile"
      fi
    done
  fi
done
rm -f $temp_folder/*

[xnardo]

LionelHoudelier It's an excellent idea, I modified the code and it still doesn't work for me, what could be wrong?

#!/bin/bash

#### ENTER YOUR DATAS ##########
# folder to parse the file
temp_folder=/home/.....
# folder where is the vcf file
folder='/home/...'
filename='VCARD_FILE.vcf'
user='XXX'
pass='XXXXX'
addressbook='XXXX'
cloud='your.nextcloud.url'

##########################################################
filepath="$folder/$filename"
url="https://${cloud}/remote.php/dav/addressbooks/users/$user/$addressbook"
echo $url

# parse the file in multiple random files name
mkdir -p "$temp_folder"
rm -r "$temp_folder"/*

inotifywait --format="%w%f" -m "$folder" -e moved_to |
    while read file; do
        if [[ "$file" == *".vcf" ]]; then 
            awk -v temp_folder="$temp_folder" 'BEGIN {
                RS="END:VCARD";
            }
            {
                command = "echo -n '\''" temp_folder "/$(pwgen 20 1).vcf'\''"
                command | getline filename
                close(command)
                print $0 "END:VCARD" > filename
                close(filename)
            }' "${filepath}"

            for xfile in "$temp_folder"/*.vcf; do
                if grep -q "BEGIN:" "$xfile"; then
                    uid=$(grep UID "$xfile" | sed 's/.*://')
                    echo "UID : $uid"
                    curl -X PUT -u "$user:$pass" "$url/$uid.vcf" -T "$xfile"
                    echo "ok"
                    rm "$xfile"
                fi
            done
        fi
    done

rm -f "$temp_folder"/*

[LionelHoudelier]

Hi. Have you installed inotify and awk? Please attach the output of the command before an after modifying the "filename".

[xnardo]

Hello LionelHoudelier, thanks for answering!. Yes, all dependencies are installed. Here I send you the details:

With your script

• Console output after I modify the test.vcf file.

Setting up watches.
Watches established.
awk: line ord.:2: fatal: cannot open file "/home/dx/Downloads/update-contact/scripttest.vcf" for reading: No such file or directory
grep: /home/dx/Descargas/update-contact/script/tmp/*.vcf: The file or directory does not exist

The tmp folder does exist in the path.

• My script configuration

#### MY CONFIGURATION ##########
# folder to parse the file
temp_folder='/home/dx/Descargas/update-contact/script/tmp'
# folder where is the vcf file
folder='/home/dx/Descargas/update-contact/script'
filename='test.vcf'
user='user'
pass='xxxx'
addressbook='clientes'
cloud='cloud.example.com'

[LionelHoudelier]

• You replaced filepath="$folder$filename" by filepath="$folder/$filename" => OK. Thanks. I changed it too.
• Your errors are not because of the temp folder but of the folder path because if the folder and/or filepath is wrong, then the parsing function gives an error for the temp folder. It seems you did not save the changes as you tested it because your output shows update-contact/scripttest.vcf instead of script/test.vcf = folder and filename are merged without separator.
• You replaced command = "echo -n '$temp_folder'/$(pwgen 20 1).vcf" by command = "echo -n '\''" temp_folder "/$(pwgen 20 1).vcf'\''" => WRONG. Reverse this change to my version, then it should work.

[xnardo]

Thanks, now i have the next mistake:

Setting up watches.
Watches established.
UID :  027a0284-01a9-41b0-880d-d73c1d3308dd
curl: (3) URL using bad/illegal format or missing URL
ok

And the file tmp have one document.vcf with this inside only:

END:VCARD

[LionelHoudelier]

put echo "$url/$uid.vcf" " " "$xfile" and give the output check that your PASS, USERNAME, and URL/UID do not contain special characters that make the url false. If it is the case, put \ before them, but i am not sure it works. The resting file in the tmp is not a problem. I don't know how to avoid that by parsing better (allready tried).

[xnardo]

• output

UID :  027a0284-01a9-41b0-880d-d73c1d3308dd
curl: (3) URL using bad/illegal format or missing URL
.vcf    /home/dx/Descargas/update-contact/script/tmp/iegah1airitaikie5Ohz.vcf/027a0284-01a9-41b0-880d-d73c1d3308dd

[LionelHoudelier]

Strange edited answer. your string $url/$uid.vcf must be correct. Check it. https://cloud.excample.com/remote.php/dav/addressbooks/users/**user**/clientes/**0**.vcf : is user correct? why 0.vcf? Be sure of your nextcloud url... (with or without "nextcloud" in it)

[xnardo]

I have run it with your original code and the modifications, but it still does not work, I don't think it is the password, but there is the format in which they are (They are application passwords), because I have active in 2fa. Here I leave the complete code. Thanks in advance.

• Output

Setting up watches.
Watches established.
UID :  027a0284-01a9-41b0-880d-d73c1d3308dd
curl: (3) URL using bad/illegal format or missing URL
https://cloud.example.com/remote.php/dav/addressbooks/users/christian/general/02.vcf    /home/dx/Descargas/update-contact/script/tmp/eishee2Viah9sov7NeiF.vcf

• Code

#!/bin/bash
#### ENTER YOUR DATAS ##########
# folder to parse the file
temp_folder='/home/dx/Descargas/update-contact/script/tmp'
# folder where is the vcf file
folder='/home/dx/Descargas/update-contact/script'
filename='test.vcf'
user='user'
pass='xxxx-xxxx-xxxx-xxxx'
addressbook='general'
cloud='cloud.example.com'
##########################################################
filepath="$folder/$filename"
url="https://${cloud}/remote.php/dav/addressbooks/users/$user/$addressbook"
echo $url
# parse the file in multiple random files name
mkdir -p $temp_folder
rm -f $temp_folder/*
echo "$folder"
inotifywait --format="%w%f" -m "$folder" -e moved_to |
    while read file; do
  if [[ "$file" == *".vcf" ]]
  then 
  awk -v temp_folder="$temp_folder" 'BEGIN {
        RS="END:VCARD";
        }
        {
        command = "echo -n '$temp_folder'/$(pwgen 20 1).vcf"
        command | getline filename
        close(command)
        print $0 "END:VCARD" > filename
        close(filename)
        }' "${filepath}"
      for xfile in "$temp_folder"/*.vcf; do
        if grep -q "BEGIN:" "$xfile"; then
          uid=$(grep UID "$xfile"  | sed 's/.*://')
          echo "UID : " "$uid"
          curl -X PUT -u "$user":"$pass" "$url/$uid.vcf" -T "$xfile"
          echo "$url/$uid.vcf" "  " "$xfile"
          rm "$xfile"
        fi
      done
  fi
done
command = "echo -n '$temp_folder'/$(pwgen 20 1).vcf"

        command | getline filename
        close(command)
        print $0 "END:VCARD" > filename
        close(filename)
        }' "${filepath}"
# upload the files
      for xfile in "$temp_folder"/*.vcf; do
        if grep -q "BEGIN:" "$xfile"; then
      for xfile in "$temp_folder"/*.vcf; do
        if grep -q "BEGIN:" "$xfile"; then
          uid=$(grep UID "$xfile"  | sed 's/.*://')
          echo "UID : " "$uid"
          curl -X PUT -u "$user":"$pass" "$url/$uid.vcf" -T "$xfile"
          #echo "$url/$uid.vcf" "  " "$xfile"
          rm "$xfile"
        fi
      done
  fi
done
rm -f $temp_folder/*

[m0uH]

I had the same problem as @xnardo . I was able to fix the problem with the script. You might try the following script:

#!/bin/bash
#### ENTER YOUR DATAS ##########
# folder to parse the file
temp_folder='/home/...'
# folder where is the vcf file
folder='/home/...'
filename='test.vcf'
user='user'
pass='xxxx-xxxx-xxxx-xxxx'
addressbook='general'
cloud='cloud.example.com'
##########################################################
filepath="$folder/$filename"
url="https://${cloud}/remote.php/dav/addressbooks/users/$user/$addressbook"
echo $url
# parse the file in multiple random files name
mkdir -p $temp_folder
rm -f $temp_folder/*
echo "$folder"
inotifywait --format="%w%f" -m "$folder" -e moved_to |
    while read file; do
  if [[ "$file" == *".vcf" ]]
  then
  awk -v temp_folder="$temp_folder" 'BEGIN {
        RS="END:VCARD";
        }
        {
        command = "echo -n '$temp_folder'/$(pwgen 20 1).vcf"
        command | getline filename
        close(command)
        print $0 "END:VCARD" > filename
        close(filename)
        }' "${filepath}"
      for xfile in "$temp_folder"/*.vcf; do
        if grep -q "BEGIN:" "$xfile"; then
          uid=$(grep '^UID:' "$xfile"  | sed -E 's/.*:([a-zA-Z0-9_\-]+).*/\1/g' | xargs)
          if [ $? -eq 0 ]
          then
            echo "UID --> $uid"
            echo "URL --> $url/$uid.vcf  FILE --> $xfile"
            curl --fail-with-body -X PUT -u "$user":"$pass" "$url/$uid.vcf" -T "$xfile"

            if [ $? -eq 0 ]
            then
              echo "ok"
              rm "$xfile"
            else
              echo "curl failed for $xfile"
            fi
          else
            echo "getting uid failed for $xfile"
          fi
        fi
      done
  fi
done

Unfortunately I still get the following response for some of the contacts I'd like to import:

<?xml version="1.0" encoding="utf-8"?>
<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">
  <s:exception>Sabre\DAV\Exception\BadRequest</s:exception>
  <s:message>Calendar object with uid already exists in this calendar collection.</s:message>
</d:error>

[LionelHoudelier]

@m0uH

• Yes, I did a mistake by editing my post. I will correct it. Thanks.
• You are almost at success. You need to add first the contact as a NEW contact WITH this process (see https://github.com/nextcloud/contacts/issues/3821#issuecomment-2140900135) . You cannot modify a contact that has been created without this process. After CREATION with this process, you can modify it and should not get this message any more.
• You can also IMPORT your contacts vcards. Add the vcards contents to your VCARD_FILE.vcf AND for each BEGIN/END VCARD sections add the filename like "UID:yourfilenamewithoutextension" then it shoud be able to update them without deleting/adding them first.

[LionelHoudelier]

@xnardo

• PLEASE edit and delete the complete code if it is the same as mine OR write only the changed line with ... before and after
• My code is not the problem anymore. It is with curl. (have you done apt update && apt upgrade) It is CERTAINLY an incompatible character in your username/password or url strings.
• Take a look at these posts : https://support.zendesk.com/hc/en-us/articles/4408819734426-API-error-curl-3-URL-using-bad-illegal-format-or-missing-URL https://github.com/curl/curl/issues/7763 https://apple.stackexchange.com/questions/453973/curl-url-using-bad-illegal-format-or-missing-url https://github.com/curl/curl/issues/11129
• In short: -- No @ or & or other symbols -- Spaces must replaced with %20 -- maybe a problem with numericals in first place in the url. -- maybe an encoding problem. You can test your url string on dedicated websites. ==> test with another url / pass / username to see a string make a difference. If it does not work, you have to inspect your strings in the curl command and look on other forum to find which one does not fit to curl. ==> if you solve it, please tell us the problem/solution to help other people.

[m0uH]

The problem with the bad/illegal URL was due to some whitespace being added to the UID by the grep / sed command.

Making sure it's only the UID (without any additional whitespace characters, cannot tell which were added, but they also garbled the output), did fix the problem:

uid=$(grep '^UID:' "$xfile" | sed -E 's/.*:([a-zA-Z0-9_\-]+).*/\1/g' | xargs)

The modification to the grep part makes sure, we only get lines with the UID, not such containing UID somwhere within the line (did happen for me for some embedded profile pictures). xargs is being used to "trim" the UID, as unfortunately the modified sed part did not fix the problem. May be sed adds some line ending charcters to its output?

[LionelHoudelier]

@m0uH Great thanks for the sed addition and the others changes, even if i did not need them in my case. I hope it helps @xnardo

[xnardo]

Thank you very much! I've just been able to check.

It finally started, but it's not modifying the data, here's the error:

The problem only occurs when the contact has been originally created in Nextcloud. If the contact is created in another app or imported directly, it updates!

UID --> 5e5f77bc-7596-4eca-929f-23d1fb89e6db
URL --> https://cloud.example.com/remote.php/dav/addressbooks/users/user/contacts/5e5f77bc-7596-4eca-929f-23d1fb89e6db.vcf  FILE --> /home/dx/Descargas/update-contact/script/tmp/ailaec6OCeeGah0aiR8U.vcf
<?xml version="1.0" encoding="utf-8"?>
<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">
  <s:exception>Sabre\DAV\Exception\BadRequest</s:exception>
  <s:message>VCard object with uid already exists in this addressbook collection.</s:message>
</d:error>
curl: (22) The requested URL returned error: 400
curl failed for /home/dx/Descargas/update-contact/script/tmp/ailaec6OCeeGah0aiR8U.vcf

[LionelHoudelier]

Cause : your contact already exist with another file name for it (.csv) on the dav server. That's why it cannot create the file with xfile name otherwise would be the contact double registered.

Solution 1: Retrieve this file name from the dav server (Export the contact) and use it as filename in the URL and xfile) OR delete the uid.csv in the URL but it should not work.

Solution 2: Delete your contact and relaunch your script. This recreates your contact with the uid param as filename. Then you can easily update it.