search

pimutils / vdirsyncer

📇 Synchronize calendars and contacts.
https://vdirsyncer.pimutils.org/
Other
1.53k stars 159 forks source link

An "unsafe character" (+) seems to be derailing the sync process when syncing with Google. #865

Open xplreitr opened 3 years ago

xplreitr commented 3 years ago

Before you submit bug reports: https://vdirsyncer.pimutils.org/en/stable/contributing.html

Things to include in your bugreport:

[pair calendar_test] a = "my_calendar_local" b = "my_calendar_remote" collections = ["from a", "from b"]

[storage my_calendar_local] type = "filesystem" path = "/calendardata/test_calendar/" fileext= ".ics"

[storage my_calendar_remote] type = "google_calendar" token_file = "/calendardata/.google_auth/" client_id = "supersecretclientid" client_secret = "supersercretsecret"

* Use `vdirsyncer -vdebug` for debug output. The output is sensitive, but
  please attach at least the last few lines before the error (if applicable),
  censored as necessary. This is almost always the most useful information.

There was a debugging note right before the error regarding an "unsafe character" which is the + symbol translated into %2b. I included snippets of the debug output surrounding the offending character. Ellipses show gaps (that I would think are fairly irrelevant) cut out.

debug: Using 1 maximal workers. Syncing calendar_test/calendar_name debug: PROPFIND https://apidata.googleusercontent.com/caldav/v2/calendar_name/events/ debug: {'User-Agent': 'randomnumbers.apps.googleusercontent.com', 'Content-Type': 'application/xml; charset=UTF-8', 'Depth': '1'} debug: b'<?xml version="1.0" encoding="utf-8" ?>\n \n \n \n \n \n </D:prop>\n </D:propfind>\n ' debug: Sending request... debug: 207 debug: {'Date': 'Mon, 01 Feb 2021 23:05:42 GMT', 'Pragma': 'no-cache', 'Content-Type': 'text/xml; charset=UTF-8', 'Cache-Control': 'no-cache, no-store, max-age=0, must-revalidate', 'Expires': 'Mon, 01 Jan 1990 00:00:00 GMT', 'Vary': 'Origin, X-Origin, Referer', 'Content-Encoding': 'gzip', 'Server': 'ESF', 'X-XSS-Protection': '0', 'X-Frame-Options': 'SAMEORIGIN', 'X-Content-Type-Options': 'nosniff', 'Alt-Svc': 'h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"', 'Transfer-Encoding': 'chunked'} ... debug: b'<?xml version="1.0" encoding="UTF-8"?>\n\n ... ... \n /caldav/v2/calendar_name/events/WEBEX-MEETING+CENTER-6.0686402-108582152.ics</D:href>\n \n HTTP/1.1 200 OK</D:status>\n \n text/calendar; component=vevent</D:getcontenttype>\n "63518914751"</D:getetag>\n \n </D:prop>\n </D:propstat>\n </D:response>\n ... ...D:response xmlns:carddav="urn:ietf:params:xml:ns:carddav" xmlns:cm="http://cal.me.com/_namespace/" xmlns:md="urn:mobileme:davservices">\n /caldav/v2/calendar_name/events/WEBEX-MEETING+CENTER-6.0680272-245422177.ics</D:href>\n \n HTTP/1.1 200 OK</D:status>\n \n text/calendar; component=vevent</D:getcontenttype>\n "63531356273"</D:getetag>\n \n </D:prop>\n </D:propstat>\n </D:response>\n ... ... /D:multistatus>\n' ... debug: Normalized URL from '/caldav/v2/calendar_name/events/m6icp53ietetp7l1gqe15540lk%40google.com.ics' to '/caldav/v2/calendar_name/events/m6icp53ietetp7l1gqe15540lk@google.com.ics' debug: Normalized URL from '/caldav/v2/calendar_name/events/6pftiq1k4nm3dt9866thuo3br0%40google.com.ics' to '/caldav/v2/calendar_name/events/6pftiq1k4nm3dt9866thuo3br0@google.com.ics' debug: Normalized URL from '/caldav/v2/calendar_name/events/WEBEX-MEETING+CENTER-6.0686402-108582152.ics' to '/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0686402-108582152.ics' debug: Normalized URL from '/caldav/v2/calendar_name/events/248h42p0kesa2tvhsrj4c04n3k%40google.com.ics' to '/caldav/v2/calendar_name/events/248h42p0kesa2tvhsrj4c04n3k@google.com.ics' debug: Normalized URL from '/caldav/v2/calendar_name/events/6pj3gcj36dgm4b9n6phm2b9k74s3ebb275im6bb46hj68chh64s66p9ic8%40google.com.ics' to '/caldav/v2/calendar_name/events/6pj3gcj36dgm4b9n6phm2b9k74s3ebb275im6bb46hj68chh64s66p9ic8@google.com.ics' ... debug: Normalized URL from '/caldav/v2/calendar_name/events/0478200E00074C5B7101A82E008700E029A129CFACD0017700001706C698DFBD5A36141BDE96A1B1058C02B.ics' to '/caldav/v2/calendar_name/events/0478200E00074C5B7101A82E008700E029A129CFACD0017700001706C698DFBD5A36141BDE96A1B1058C02B.ics' debug: Normalized URL from '/caldav/v2/calendar_name/events/tooubg46aa4rfimr0up83370h8%40google.com.ics' to '/caldav/v2/calendar_name/events/tooubg46aa4rfimr0up83370h8@google.com.ics' debug: Normalized URL from '/caldav/v2/calendar_name/events/WEBEX-MEETING+CENTER-6.0680272-245422177.ics' to '/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0680272-245422177.ics' debug: Normalized URL from '/caldav/v2/calendar_name/events/0478200E00074C5B7101A82E008700E07E35C50B92D4017700001708697308B07EEDA4F95AE7E466CF804AD.ics' to '/caldav/v2/calendar_name/events/0478200E00074C5B7101A82E008700E07E35C50B92D4017700001708697308B07EEDA4F95AE7E466CF804AD.ics' debug: Normalized URL from '/caldav/v2/calendar_name/events/cdh3ae3464p3abb66hhj4b9k64p30b9pc9j3cb9h6so66d35c9i38p9kco%40google.com.ics' to '/caldav/v2/calendar_name/events/cdh3ae3464p3abb66hhj4b9k64p30b9pc9j3cb9h6so66d35c9i38p9kco@google.com.ics' ... debug: Already normalized: '/caldav/v2/calendar_name/events/0478200E00074C5B7101A82E0087001013269558E0D201770000170B92F3FD560EB8F48B07E8F4C3FCA120E.ics' debug: Already normalized: '/caldav/v2/calendar_name/events/apaqr9f9rpo4pppefe9h9cp1ks@google.com.ics' debug: Unsafe character: '%2B' debug: Already normalized: '/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0686402-108582152.ics' debug: Already normalized: '/caldav/v2/calendar_name/events/ik7bg09ub8cjs2p1i3nkdk3114@google.com.ics' ... debug: Already normalized: '/caldav/v2/calendar_name/events/9s46nk3pb4nho69g592b5fd65g@google.com.ics' debug: Already normalized: '/caldav/v2/calendar_name/events/3ffviaqu42ev1e2lj1proijedc@google.com.ics' debug: Unsafe character: '%2B' debug: Already normalized: '/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0680272-245422177.ics' debug: Already normalized: '/caldav/v2/calendar_name/events/5608r6i8nq19msdp1anr100s3v@google.com.ics' debug: b'<?xml version="1.0" encoding="utf-8" ?>\n <C:calendar-multiget xmlns:D="DAV:"\n xmlns:C="urn:ietf:params:xml:ns:caldav">\n \n \n \n </D:prop>\n D:href>/caldav/v2/calendar_name/events/or57be33sferu9juai0vn13u20@google.com.ics</D:href... ...\nD:href>/caldav/v2/calendar_name/events/9s46nk3pb4nho69g592b5fd65g@google.com.ics</D:href\nD:href>/caldav/v2/calendar_name/events/3ffviaqu42ev1e2lj1proijedc@google.com.ics</D:href\nD:href>/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0680272-245422177.ics</D:href>\n<D:href>/caldav/v2/calendar_name/events/5608r6i8nq19msdp1anr100s3v@google.com.ics</D:href\nD:href>/caldav/v2/calendar_name/events/0aln4lncn4oml6n60ubesepo24@google.com.ics</D:href\nD:href>/caldav/v2/calendar_name/events/b4i7bp6ialrv59cv1cga2mgqao@google.com.ics</D:href\nD:href>/caldav/v2/calendar_name/events/02nmc16u3nvrbcu7drajeut5a9@google.com.ics</D:href\n/caldav/v2/calendar_name/events/C38E9895-D6BA-4C30-9FFF-270FF248EA7A.ics</D:href> ... ... </C:calendar-multiget>' ... debug: Sending request... debug: 207 debug: {'Cache-Control': 'no-cache, no-store, max-age=0, must-revalidate', 'Content-Type': 'text/xml; charset=UTF-8', 'Date': 'Mon, 01 Feb 2021 23:06:00 GMT', 'Expires': 'Mon, 01 Jan 1990 00:00:00 GMT', 'Pragma': 'no-cache', 'Vary': 'Origin, X-Origin, Referer', 'Content-Encoding': 'gzip', 'Server': 'ESF', 'X-XSS-Protection': '0', 'X-Frame-Options': 'SAMEORIGIN', 'X-Content-Type-Options': 'nosniff', 'Alt-Svc': 'h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"', 'Transfer-Encoding': 'chunked'} ... debug: b'<?xml version="1.0" encoding="UTF-8"?>\n\n ... ...\n /caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0686402-108582152.ics</D:href>\n HTTP/1.1 404 Not Found</D:status>\n </D:response>\n \n /caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0680272-245422177.ics</D:href>\n HTTP/1.1 404 Not Found</D:status>\n </D:response>\n</D:multistatus>\n' ... debug: Already normalized: '/caldav/v2/calendar_name/events/uec002tktnj5if16b4lvb7tsa8@google.com.ics' debug: Already normalized: '/caldav/v2/calendar_name/events/tlp35ob9j4rl3fi8l2o4vjbp0g@google.com.ics' debug: Already normalized: '/caldav/v2/calendar_name/events/ts4p2oqivvt4i2abj7qhsanhuk@google.com.ics' debug: Unsafe character: '%2B' debug: Already normalized: '/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0686402-108582152.ics' debug: Skipping '/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0686402-108582152.ics', properties are missing. debug: Unsafe character: '%2B' debug: Already normalized: '/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0680272-245422177.ics' debug: Skipping '/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0680272-245422177.ics', properties are missing. error: Unknown error occurred for calendar_test/calendar_name: /caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0686402-108582152.ics error: Use -vdebug to see the full traceback. debug: File "/usr/local/lib/python3.7/site-packages/vdirsyncer/cli/tasks.py", line 67, in sync_collection debug: partial_sync=pair.partial_sync debug: File "/usr/local/lib/python3.7/site-packages/vdirsyncer/sync/init.py", line 135, in sync debug: b_nonempty = b_info.prepare_new_status() debug: File "/usr/local/lib/python3.7/site-packages/vdirsyncer/sync/init.py", line 58, in prepare_new_status debug: if prefetch else ()): debug: File "/usr/local/lib/python3.7/site-packages/vdirsyncer/storage/dav.py", line 503, in get_multi debug: raise exceptions.NotFoundError(href) error: 1 out of 12 tasks failed. ~ 


Reading the output, it seems the href cant be found. Is that because the href is using the encoded character? Any ideas on how to resolve the issue?
Accacio commented 3 years ago

I have exactly the same error with WEBEX meetings

jvasile commented 3 years ago

I have this exact error with WEBEX meetings as well.

In #491 an "unsafe character" error was fixed by removing it from the reserved char list. I tried that here, removing the + from line 29 of dav.py, but now I get a new error:

debug: Already normalized: '/caldav/v2/james@opentechstrategies.com/events/WEBEX-MEETING%2BCENTER-6.012920952-1184426117.ics'
debug: Skipping '/caldav/v2/james@opentechstrategies.com/events/WEBEX-MEETING%2BCENTER-6.012920952-1184426117.ics', properties are missing.
error: Unknown error occurred for personal_calendar/james@opentechstrategies.com: /caldav/v2/james@opentechstrategies.com/events/WEBEX-MEETING%2BCENTER-6.012920952-1184426117.ics
error: Use `-vdebug` to see the full traceback.
debug:   File "/home/james/src/3p/vdirsyncer/vdirsyncer/cli/tasks.py", line 68, in sync_collection
debug:     sync.sync(
debug:   File "/home/james/src/3p/vdirsyncer/vdirsyncer/sync/__init__.py", line 138, in sync
debug:     b_nonempty = b_info.prepare_new_status()
debug:   File "/home/james/src/3p/vdirsyncer/vdirsyncer/sync/__init__.py", line 60, in prepare_new_status
debug:     for href, item, etag in (self.storage.get_multi(prefetch)
debug:   File "/home/james/src/3p/vdirsyncer/vdirsyncer/storage/dav.py", line 507, in get_multi
debug:     raise exceptions.NotFoundError(href)
error: 1 out of 2 tasks failed.
dchersey commented 2 years ago

I have this too ... fortunately I don't have too many webex meetings.
I worked around it by finding the webex meeting invitation, deleting it from google calendar and re-creating without the invitation. Probably could have just edited it too.

It was pretty impossible to find the offending calendar entry by searching, but fortunately I noticed my calendars were not syncing within a few days of accepting a webex meeting, so it it was easy to find the entry.

It would be nice to see a fix that works since this is the only thing that derails my sync.

WhyNotHugo commented 2 years ago

Can you check if you have the same issue with the current master? I believe this is fixed via #929.