search

lfos / calcurse

A text-based calendar and scheduling application
https://calcurse.org
BSD 2-Clause "Simplified" License
993 stars 95 forks source link

calcurse-caldav does not create sync.db #137

Open marskar opened 6 years ago

marskar commented 6 years ago

First of all, thank you for calcurse, I really love it, though I do have a small problem with the calcurse-caldav script.

The script does not create sync.db, so I have to run calcurse-caldav --init keep-remote instead of just calcurse-caldav each time. To be clear, ~/.calcurse/caldav/ only contains config and oauth2_cred, not sync.db.

I am using MacOS with python 3.6.5.

I changed the shebang in the script to use my anaconda python interpreter:

#!/Users/username/anaconda3/bin/python

import argparse
...

The only error I get is after importing new objects: error: Missing ETag. Here is the abbreviated output of calcurse-caldav --init keep-remote --debug.

Removing all local calcurse objects...
> REPORT https://apidata.googleusercontent.com/caldav/v2/username@gmail.com/events/
> Headers: {'Content-Type': 'application/xml; charset=utf-8', 'Depth': '1'}
> <?xml version="1.0" encoding="utf-8" ?><C:calendar-query xmlns:D="DAV:"                   xmlns:C="urn:ietf:params:xml:ns:caldav"><D:prop><D:getetag /></D:prop><C:filter><C:comp-filter name="VCALENDAR" /></C:filter></C:calendar-query>

< Headers: {'vary': 'Origin, X-Origin', 'content-type': 'text/xml; charset=UTF-8', 'date': 'Wed, 08 Aug 2018 02:58:39 GMT', 'expires': 'Wed, 08 Aug 2018 02:58:39 GMT', 'cache-control': 'private, max-age=0', 'x-content-type-options': 'nosniff', 'x-frame-options': 'SAMEORIGIN', 'x-xss-protection': '1; mode=block', 'server': 'GSE', 'alt-svc': 'quic=":443"; ma=2592000; v="44,43,39,35"', 'transfer-encoding': 'chunked', 'status': '207', 'content-length': '9858', '-content-encoding': 'gzip'}
< <?xml version="1.0" encoding="UTF-8"?>
< <D:multistatus xmlns:D="DAV:" xmlns:caldav="urn:ietf:params:xml:ns:caldav" xmlns:cs="http://calendarserver.org/ns/" xmlns:ical="http://apple.com/ns/ical/">
<  <D:response xmlns:carddav="urn:ietf:params:xml:ns:carddav" xmlns:cm="http://cal.me.com/_namespace/" xmlns:md="urn:mobileme:davservices">
<   <D:href>/caldav/v2/username%40gmail.com/events/040000008200E00074C5B7101A82E008000000001032E219E727D401000000000000000010000000520C9754E7195348B8D8330B8095443D.ics</D:href>
<   <D:propstat>
<    <D:status>HTTP/1.1 200 OK</D:status>
<    <D:prop>
<     <D:getetag>"63669375554"</D:getetag>
<    </D:prop>
<   </D:propstat>
<  </D:response>
<  <D:response xmlns:carddav="urn:ietf:params:xml:ns:carddav" xmlns:cm="http://cal.me.com/_namespace/" xmlns:md="urn:mobileme:davservices">
...
New sync database entry: /caldav/v2/username%40gmail.com/events/040000008200E00074C5B7101A82E0080000000080A684B65D01D4010000000000000000100000004AD2AA5C136245499E905152F2B0D28F.ics 63669375575 9ceb385c95332e061e2c0e44fbff8d50f9cb92bd
Importing new object 63669373923.
New sync database entry: /caldav/v2/username%40gmail.com/events/0bmpjovrcdbgnn8vsu4gkre20h%40google.com.ics 63669373923 6b98e5881efcb36271c62ff4c9f5bd7ebf3c7f56
error: Missing ETag.
error: The error occurred while processing the following XML node:
error: <ns0:response xmlns:ns0="DAV:">
error:   <ns0:href>/caldav/v2/username%40gmail.com/events/poo9eff8smpd4q82t9nd7rd
error: 120_R20180213T171500%40google.com.ics</ns0:href>
error:   <ns0:status>HTTP/1.1 404 Not Found</ns0:status>
error:  </ns0:response>

I just realized that this issue is similar to (or the same as) #90 by @scndthe2nd.

I'm happy to send the full debug output. I saved it as a txt file.

lfos commented 6 years ago

To me, this looks like there is an issue with a pushed object which cannot be fetched from the server after it's been created. Could you share the full debug log, please? Make sure you strip any private data; if you're using the most recent development snapshot, at least authentication credentials are removed automatically but you might still want to look for any personal information that you do not want to share.

marskar commented 6 years ago

I agree with your diagnosis of the problem. I created the attached error log like this: calcurse-caldav --init two-way --debug > error-log.txt I also get one of the error messages below:

error: The server at apidata.googleusercontent.com replied with HTTP status code
error: 409 (Conflict) while trying to access https://apidata.googleusercontent.c
error: om/caldav/v2/calendar-ID@group.calendar.google.com/events/
error: f8aea5f5a6c35043b471a9df51803d60f04a90ab.ics.

error: The server at apidata.googleusercontent.com replied with HTTP status code
error: 503 (Service Unavailable) while trying to access https://apidata.googleus
error: ercontent.com/caldav/v2/calendar-ID@group.calendar.google.
error: com/events/5f5fd93f7d9f800a2d2e47446a0310a44b640a55.ics.

error: The server at apidata.googleusercontent.com replied with HTTP status code
error: 403 (Forbidden) while trying to access https://apidata.googleusercontent.
error: com/caldav/v2/calendar-ID@group.calendar.google.com/events
error: /c269aa14f34a0b9c3f1b5df9f6d4afaa99c4e098.ics.

error-log.txt

UPDATE: I started removing events one by one and then ran calcurse-caldav --init keep-local and I got the no errors:

Connecting to apidata.googleusercontent.com...
Removing all objects from the CalDAV server...
Pushing new object 5feb9910d8ea767fe16eedcfb2b7ba9249a2849d to the server.
Pushing new object 4932bac2110c65d45154f47a35f6736f619aa62c to the server.
Pushing new object 0769d704da06a036ab4c012b3086850c44414d38 to the server.
Pushing new object 025614a16727b999165e1a3dd9a94235c96cfe56 to the server.
Pushing new object 36d634673266947f775977067453811bdc3866a5 to the server.
Pushing new object 4d8462a423f804cef369e62c604a17499cefa3af to the server.
Pushing new object f14f55b55d36902bc29bd768129e202c8a37b432 to the server.
Pushing new object 99cfcb69d22f4b2fb1f12b4171d123760c357edc to the server.
Pushing new object a71d4a9e5dc49ca52728ac520cd571ef1e1eb189 to the server.
Saving synchronization database to /Users/skarzynskimw/.calcurse/caldav/sync.db...
0 items imported, 0 items removed locally.
9 items exported, 0 items removed from the server.

I imported some events: calcurse -i marskar_marskar@gmail.com.ics

Import process report: 0779 lines read
20 apps / 11 events / 0 todos / 0 skipped

I then tried a two-way init: calcurse-caldav --init two-way and got

Connecting to apidata.googleusercontent.com...
Importing new object 63670486764.
Importing new object 63670486762.
Importing new object 63670486763.
Importing new object 63670486765.
Importing new object 63670486769.
Importing new object 63670486761.
Importing new object 63670486767.
Importing new object 63670486766.
Importing new object 63670486761.
Pushing new object 57cf9bb2e1264eee142e75da2712937725e62d1d to the server.
Pushing new object 4932bac2110c65d45154f47a35f6736f619aa62c to the server.
error: The server at apidata.googleusercontent.com replied with HTTP status code
error: 409 (Conflict) while trying to access https://apidata.googleusercontent.c
error: om/caldav/v2/51ctfino68ppeq5g0kqo8sfg04@group.calendar.google.com/events/
error: 4932bac2110c65d45154f47a35f6736f619aa62c.ics.

I then ran calcurse-caldav and got

Connecting to apidata.googleusercontent.com...
Loading synchronization database from /Users/skarzynskimw/.calcurse/caldav/sync.db...
Importing new object 63670487055.
Pushing new object 7439858495d43afe7669bada263adf4b06667e27 to the server.
Pushing new object b2765c9c65db05de57fde021702a48b5ff747ec4 to the server.
Pushing new object f69a730d92697a9bff829395a93ba3c5f708bb50 to the server.
Pushing new object d09235bb28590f2e72860ed1108382284e1f335a to the server.
Pushing new object 33ae15ef93f72f852f2afc98e15e2720f9fc0ec4 to the server.
Pushing new object 90ffa6d5d697ca01fe4ac9c8424e7e0d54d82886 to the server.
Pushing new object 8bf8202118bbf4c4a7d0c659bbd7211942585d22 to the server.
Pushing new object f647e1a26fbf7454b557a1c0c217780de11b7c7c to the server.
Pushing new object 4b047ea1422cb60e1fd5c2d237882180bb29d119 to the server.
Pushing new object cf64bc0891223604bff5242e5ebab5836939198f to the server.
Pushing new object f3d5f0e8a194f6c515ee7f25376814fd03f2afac to the server.
Pushing new object 592757b044a31aa82227d601f112c18563c5294b to the server.
Pushing new object 4212742b07d94a6ab15c4de3739d5ab693a15cd4 to the server.
Pushing new object 341ffa5bf77bf9fa0353c61c12e769b5267b704a to the server.
Pushing new object 22ad2a26d9f2422ee32ca21913f7393a07ae7196 to the server.
Pushing new object 438629bfc253cdfb5fbc6fd651e89b770e933c70 to the server.
Pushing new object 74e98acc747f8e58014c954268e0c66ef710da5a to the server.
Pushing new object bc783e9595b64a764c7cc3fbc45a5fd1d19d7dbb to the server.
Pushing new object 2579cc211110288c136565e3f214354becfacbf2 to the server.
Pushing new object d29b226ae444b7ad5e3fc723268aab4d5d0a91e0 to the server.
Pushing new object 3cd4093f4f00756ba490ff1d25315ffd82941b7c to the server.
Pushing new object ae5abcbe76172369c4a39c4881df36f6f8f7f467 to the server.
Pushing new object e0356dbcb5c59d2bc8166d947f97c4aadf236129 to the server.
Pushing new object 042f870806286a85c1004d4f6512c2ee173946b7 to the server.
Pushing new object 8494199e3361235d8c3b4f78e3eac5c72796d633 to the server.
Pushing new object 473fff997943cfd4b1f7abf44386ab4b27da573a to the server.
Pushing new object f9b97ce3a2a3c783da69497ef444b471bc60450a to the server.
Pushing new object ddf5c5a2aebc517dda300074e2ba189e396fb30f to the server.
Pushing new object bfdf7f2f3ece3d9cd817726d06086b8084156ee7 to the server.
Pushing new object 39d2c9957a2db57fae1d9d3307627e08a2bb34f4 to the server.
Pushing new object 33064c886c32993ad03114504552381bcc772a1b to the server.
Pushing new object 5f5fd93f7d9f800a2d2e47446a0310a44b640a55 to the server.
Pushing new object 52b21288543b396ccb8d649ec385b9f3e2b55500 to the server.
Pushing new object 7aeadde9d48939f07e68aeaeb60512a2a86b202f to the server.
Pushing new object 8af764c015b1db928e07bd3dbb5cd6b9473445ca to the server.
Pushing new object dd39b5e06105ac1ebb3280e64b22caeece648dcb to the server.
Pushing new object 616f22881a3daac194d4b07bfe36ef446e6025f4 to the server.
Pushing new object 57cf9bb2e1264eee142e75da2712937725e62d1d to the server.
Pushing new object 4928a2cef3498593599384f23adc5e5084653459 to the server.
Pushing new object 1c8f21c54dd2356db2c273d765d7a685540a0f00 to the server.
Saving synchronization database to /Users/skarzynskimw/.calcurse/caldav/sync.db...
1 items imported, 0 items removed locally.
40 items exported, 0 items removed from the server.

I think I mashed keys until it started working, but I am not sure what is going on.

lfos commented 6 years ago

Which is the exact error you got with the log? It would have been better if you added 2>&1 to the logging command line such that errors are printed into the log file as well...

marskar commented 6 years ago

Ok, great idea. I ran calcurse-caldav --init two-way --debug > error-log.txt 2>&1 a couple of times for two different calendars and got the error logs below. calendar1: 404error-log.txt calendar2: 409error-log.txt calendar2: 503error-log.txt

lfos commented 6 years ago

Strange. It seems like the server claims to have a certain href but returns 404 Not Found for the same href later:

Connecting to apidata.googleusercontent.com...
> REPORT https://apidata.googleusercontent.com/caldav/v2/marskar@gmail.com/events/
> Headers: {'Content-Type': 'application/xml; charset=utf-8', 'Depth': '1'}
> <?xml version="1.0" encoding="utf-8" ?><C:calendar-query xmlns:D="DAV:"                   xmlns:C="urn:ietf:params:xml:ns:caldav"><D:prop><D:getetag /></D:prop><C:filter><C:comp-filter name="VCALENDAR" /></C:filter></C:calendar-query>

< Headers: {'vary': 'Origin, X-Origin', 'content-type': 'text/xml; charset=UTF-8', 'date': 'Tue, 21 Aug 2018 13:46:40 GMT', 'expires': 'Tue, 21 Aug 2018 13:46:40 GMT', 'cache-control': 'private, max-age=0', 'x-content-type-options': 'nosniff', 'x-frame-options': 'SAMEORIGIN', 'x-xss-protection': '1; mode=block', 'server': 'GSE', 'alt-svc': 'quic=":443"; ma=2592000; v="44,43,39,35"', 'transfer-encoding': 'chunked', 'status': '207', 'content-length': '12236', '-content-encoding': 'gzip'}
< <?xml version="1.0" encoding="UTF-8"?>
< <D:multistatus xmlns:D="DAV:" xmlns:caldav="urn:ietf:params:xml:ns:caldav" xmlns:cs="http://calendarserver.org/ns/" xmlns:ical="http://apple.com/ns/ical/">
< [...]
<  <D:response xmlns:carddav="urn:ietf:params:xml:ns:carddav" xmlns:cm="http://cal.me.com/_namespace/" xmlns:md="urn:mobileme:davservices">
<   <D:href>/caldav/v2/calendar-ID/events/poo9eff8smpd4q82t9nd7rd120_R20180213T171500%40google.com.ics</D:href>
<   <D:propstat>
<    <D:status>HTTP/1.1 200 OK</D:status>
<    <D:prop>
<     <D:getetag>"63669373380"</D:getetag>
<    </D:prop>
<   </D:propstat>
<  </D:response>
< [...]

Here, we're asking for a list of all hrefs and /caldav/v2/calendar-ID/events/poo9eff8smpd4q82t9nd7rd120_R20180213T171500%40google.com.ics is one of the paths returned.

Then, we're explicitly asking for the ETags of all hrefs again...

> REPORT https://apidata.googleusercontent.com/caldav/v2/marskar@gmail.com/events/
> Headers: {'Content-Type': 'application/xml; charset=utf-8'}
> <?xml version="1.0" encoding="utf-8" ?><C:calendar-multiget xmlns:D="DAV:"                      xmlns:C="urn:ietf:params:xml:ns:caldav"><D:prop><D:getetag /><C:calendar-data /></D:prop><D:href>/caldav/v2/calendar-ID/events/71678b2572c7be98b50088786c577193aeff4907.ics</D:href><D:href>/caldav/v2/calendar-ID/events/75h34d9ochj66bb46sqj8b9kcgo64b9pc5h34bb46kr38dphccp6acpmco%40google.com.ics</D:href><D:href>/caldav/v2/calendar-ID/events/7cvn71nu4msukkr87d1o89b7fr%40google.com.ics</D:href><D:href>/caldav/v2/calendar-ID/events/7kukuqrfedlm2f9tu8si7lepv0uccajj2f5ck1p25ucipleagljbrgdo29otit6e4ft0.ics</D:href><D:href>/caldav/v2/calendar-ID/events/7kukuqrfedlm2f9tleo5c1mavltresjfksr16o02u84ra670b822rueatvsap2sj7g80.ics</D:href><D:href>/caldav/v2/calendar-ID/events/131v7f543kl9qip84c4bfih3fq%40google.com.ics</D:href><D:href>/caldav/v2/calendar-ID/events/c269aa14f34a0b9c3f1b5df9f6d4afaa99c4e098.ics</D:href><D:href>/caldav/v2/calendar-ID/events/11ukd31obvld579u694iksl0ln%40google.com.ics</D:href><D:href>/caldav/v2/calendar-ID/events/3ghs8667ks0cmr2n3q025t9445%40google.com.ics</D:href><D:href>/caldav/v2/calendar-ID/events/73t0b64o0ch1dq1qukonpfbpru%40google.com.ics</D:href><D:href>/caldav/v2/calendar-ID/events/4g81gj7frc6099kemech8h2kq5%40google.com.ics</D:href><D:href>/caldav/v2/calendar-ID/events/353eeg1f4jegblg3g5sv1jpm3j%40google.com.ics</D:href><D:href>/caldav/v2/calendar-ID/events/ccsm2cr4coq32b9o71j34b9kcdgj2b9p6dj34b9o6ko3aob461im8cj468%40google.com.ics</D:href><D:href>/caldav/v2/calendar-ID/events/3s5ak1bfs5jc1a94gr3kra2oh0%40google.com.ics</D:href><D:href>/caldav/v2/calendar-ID/events/poo9eff8smpd4q82t9nd7rd120%40google.com.ics</D:href><D:href>/caldav/v2/calendar-ID/events/poo9eff8smpd4q82t9nd7rd120_R20180213T171500%40google.com.ics</D:href><D:href>/caldav/v2/calendar-ID/events/040000008200E00074C5B7101A82E00800000000703D98D58F38D4010000000000000000100000002F964A39430F51419A9C6C7490A4DD6F.ics</D:href><D:href>/caldav/v2/calendar-ID/events/4eb6d2e206dcce279d66ea5225c5a5637cc8db6d.ics</D:href><D:href>/caldav/v2/calendar-ID/events/7kukuqrfedlm2f9t5h18i9nb6vim45sogpaapp6n8cuj68ga0vhivdgonochmi73hnm0.ics</D:href><D:href>/caldav/v2/calendar-ID/events/7kukuqrfedlm2f9t76bek0p94f5a2u2j2g5oomv21hlonm9jlo8uo2ulc5osuf5blrv0.ics</D:href><D:href>/caldav/v2/calendar-ID/events/0pru3tdqbnnb51gdi5s45jda2r%40google.com.ics</D:href><D:href>/caldav/v2/calendar-ID/events/0qvaeu0p0p65iecjkj12649hur%40google.com.ics</D:href><D:href>/caldav/v2/calendar-ID/events/a8573204-a240-4b74-8fb1-6739280b2fcf.ics</D:href><D:href>/caldav/v2/calendar-ID/events/7kukuqrfedlm2f9teq4q7vvo11cb6j2b8vro67kron2l8nevssduup97mliedltnki20.ics</D:href><D:href>/caldav/v2/calendar-ID/events/f8aea5f5a6c35043b471a9df51803d60f04a90ab.ics</D:href><D:href>/caldav/v2/calendar-ID/events/1sdgtlojt3dvqcd4s87cj59mv0%40google.com.ics</D:href><D:href>/caldav/v2/calendar-ID/events/Icalf5aa50ca2e06d5fbbf4fd460b566614a.ics</D:href><D:href>/caldav/v2/calendar-ID/events/998D41D0-D838-4A5B-81CB-621616BDF57F.ics</D:href><D:href>/caldav/v2/calendar-ID/events/4j108mamki7uji53q3cujlh32o%40google.com.ics</D:href><D:href>/caldav/v2/calendar-ID/events/01ne42mt01og2tkjqkg32uts4k%40google.com.ics</D:href></C:calendar-multiget>

< Headers: {'vary': 'Origin, X-Origin', 'content-type': 'text/xml; charset=UTF-8', 'date': 'Tue, 21 Aug 2018 13:46:41 GMT', 'expires': 'Tue, 21 Aug 2018 13:46:41 GMT', 'cache-control': 'private, max-age=0', 'x-content-type-options': 'nosniff', 'x-frame-options': 'SAMEORIGIN', 'x-xss-protection': '1; mode=block', 'server': 'GSE', 'alt-svc': 'quic=":443"; ma=2592000; v="44,43,39,35"', 'transfer-encoding': 'chunked', 'status': '207', 'content-length': '51824', '-content-encoding': 'gzip'}
< <?xml version="1.0" encoding="UTF-8"?>
< <D:multistatus xmlns:D="DAV:" xmlns:caldav="urn:ietf:params:xml:ns:caldav" xmlns:cs="http://calendarserver.org/ns/" xmlns:ical="http://apple.com/ns/ical/">
< [...]
<  <D:response xmlns:carddav="urn:ietf:params:xml:ns:carddav" xmlns:cm="http://cal.me.com/_namespace/" xmlns:md="urn:mobileme:davservices">
<   <D:href>/caldav/v2/calendar-ID/events/poo9eff8smpd4q82t9nd7rd120_R20180213T171500%40google.com.ics</D:href>
<   <D:status>HTTP/1.1 404 Not Found</D:status>
<  </D:response>
< [...]

... and in the response, the server replies to /caldav/v2/calendar-ID/events/poo9eff8smpd4q82t9nd7rd120_R20180213T171500%40google.com.ics with 404 Not Found.

[...]
error: Missing ETag.
error: The error occurred while processing the following XML node:
error: <ns0:response xmlns:ns0="DAV:">
error:   <ns0:href>/caldav/v2/calendar-ID/events/poo9eff8smpd4q82t9nd7rd
error: 120_R20180213T171500%40google.com.ics</ns0:href>
error:   <ns0:status>HTTP/1.1 404 Not Found</ns0:status>
error:  </ns0:response>

This error message then "makes sense". Of course, it is a good idea to make the script more robust to safeguard against such inconsistencies. I will look into this later. However, it would also be interesting why this happens in the first place. Is there anything modifying items on the server while the synchronization script is running? If so, this might be a race condition. Otherwise, could you try to analyze the item /caldav/v2/calendar-ID/events/poo9eff8smpd4q82t9nd7rd120_R20180213T171500%40google.com.ics manually and see whether it actually exists? If it does indeed, is there anything special about this particular item?

marskar commented 6 years ago

I have no idea how to find the item /caldav/v2/calendar-ID/events/poo9eff8smpd4q82t9nd7rd. I can try to compare it specific events, if there is a quick way to find it. I'm afraid that the item does not actually exist in the my calendar.

lfos commented 6 years ago

For starters, you could try to visit https://apidata.googleusercontent.com/caldav/v2/calendar-ID/events/poo9eff8smpd4q82t9nd7rd120_R20180213T171500%40google.com.ics (after logging in) and see what happens.

marskar commented 6 years ago

I'm logged in to my google account, but the site still says Login Required without asking for my login info. I tried a couple of different event IDs. I got everything to work with a new calendar I set up, by running calcurse-caldav --init keep-remote. When I change the calendar-ID in my config file back to my main calendar, I still get the Missing ETag error (which means no sync.db file is created) when I run calcurse-caldav --init keep-remote. The very curious thing is that I can then import all of the events from my main calendar and my contacts into the new calendar in calcurse with calcurse -i marskar_marskar@gmail.com.ics and calcurse -i \#contacts@group.v.calendar.google.com.ics and then run calcurse-caldav without any problems. This makes me wonder... Is it possible that there is some kind of broken event that does not actually show up in my main calendar that causes this problem? Perhaps the problematic event is not something I entered, because earlier I even tried removing my main calendar (deleting everything) and entering the events again manually.

rslindee commented 5 years ago

Has there been any progress on this? I'm experiencing a similar issue ("Missing ETag" followed by a "404 Not Found" in the debug output when attempting to create a sync.db. If I attempt to view said .ics, I get a "Login Required" message).

Is there perhaps a way to override or ignore said problematic items?

edit: One interesting data point is that if I export my "main" calendar to an ics file, then import said file into a "new" calendar and set caldav to the "new" calendar, everything appears to work fine.

sjl-10x commented 5 years ago

I'm seeing the same thing as @rslindee (Missint ETag, 404 in debug output, trying to view in a browser says "Login Required).

lfos commented 5 years ago

Strange. Well, maybe we should just display a warning if any of the reported events returns a 404 and continue syncing.

vap0rtranz commented 5 years ago

UPDATE: nevermind; mine works now. My config was still doing a DryRun.

BTW: if a rerun is done on --init and the Google API details are changed (OAuth, project, etc.), the /.calcurse/caldav/oauth2_cred is not updated. I had to get it out of that way during troubleshooting:

mv ~/.calcurse/caldav/oauth2_cred{,.old}

It looks like I may have the same issue on ArchLinux, calcurse v4.4.0, python v3.7.2. The init run looks but:

$ calcurse-caldav --init=two-way --authcode <INSERT>
...
Importing new object 63643870296.
Importing new object 63632972821.
Importing new object 63632345075.
Saving synchronization database to /home/justin/.calcurse/caldav/sync.db...
0 items imported, 0 items removed locally.
0 items exported, 0 items removed from the server.

But the sync.db file actually isn't saved (or created):

$ ls -l ~/.calcurse/caldav/   
total 8
-rw-r--r-- 1 justin justin 2386 Mar  9 08:57 config
-rw------- 1 justin justin 1087 Mar  9 08:54 oauth2_cred
sjl-10x commented 5 years ago

Well, maybe we should just display a warning if any of the reported events returns a 404 and continue syncing.

This seems like a reasonable workaround to me. There's probably just one wonky event somewhere in my calendar that's breaking things.

sjl-10x commented 5 years ago

Strange. Well, maybe we should just display a warning if any of the reported events returns a 404 and continue syncing.

@lfos I could try to send a PR for this if you can point me at the relevant section of the code. I'd love to be able to use a curses interface to my calendar but right now I can't start because of this issue.

lfos commented 5 years ago

@sjl-10x I think you should have a look at the loop in pull_objects():

https://github.com/lfos/calcurse/blob/8741334d83aa5f77f1169af70493f394f860779f/contrib/caldav/calcurse-caldav.py#L439-L478

We should probably check the status field of the response before doing anything else and ignore the item (with a warning) when it's 404.

sjl-10x commented 5 years ago

I hacked in a quick check for this to see if it would work:

diff --git a/contrib/caldav/calcurse-caldav.py b/contrib/caldav/calcurse-caldav.py
index 7b9b56c..e8505eb 100755
--- a/contrib/caldav/calcurse-caldav.py
+++ b/contrib/caldav/calcurse-caldav.py
@@ -442,6 +442,16 @@ def pull_objects(hrefs_missing, hrefs_modified, conn, syncdb, etagdict):
             die_atnode('Missing href.', node)
         href = hrefnode.text

+        statusnode = node.find("./D:status", namespaces=nsmap)
+        if statusnode is not None:
+            status = re.match(r'HTTP.*(\d\d\d)', statusnode.text)
+            if status is None:
+                die_atnode('Could not parse status.', node)
+            statuscode = status.group(1)
+            if statuscode == '404':
+                print('Skipping missing item: {}'.format(href))
+                continue
+
         etagnode = node.find("./D:propstat/D:prop/D:getetag", namespaces=nsmap)
         if etagnode is None:
             die_atnode('Missing ETag.', node)

This did allow it to get past the problematic items and sync them down. Unfortunately now I can see there there are other problems that mean calcurse isn't going to work for me:

So, I don't think I'm going to spend any more time on this. If someone else wants to use the diff above as a starting point, feel free. Thanks anyway.

lfos commented 5 years ago

So, I don't think I'm going to spend any more time on this. If someone else wants to use the diff above as a starting point, feel free. Thanks anyway.

You could also try using vdirsyncer for CalDAV synchronization. You might have more luck with that. See [1] for details.

[1] https://github.com/lfos/calcurse/tree/pu/contrib/vdir

sjl-10x commented 5 years ago

Also, hilariously, your comment with the diff is appearing at the bottom of this page for me and is dated "5 hours from now":

time is hard

So somehow I managed to hit a time zone bug in GitHub when reporting a time zone bug on GitHub.

lfos commented 5 years ago

Also, hilariously, your comment with the diff is appearing at the bottom of this page for me and is dated "5 hours from now": [...] So somehow I managed to hit a time zone bug in GitHub when reporting a time zone bug on GitHub.

Interesting. Reported this to the GitHub support team.

fmv1992 commented 5 years ago

I had the same bug

The fix: https://github.com/lfos/calcurse/issues/137#issuecomment-490541349 worked for me.

Please incorporate it into the project.

Thanks.

holalluis commented 4 years ago

I had the same error, and the piece of code from @sjl-10x worked for me, thanks! I would like to delete from google calendar the events that give the 404 error, how I can do this?

$ calcurse-caldav
Connecting to apidata.googleusercontent.com...
Loading synchronization database from /Users/lluisma/.calcurse/caldav/sync.db...
Skipping missing item: /caldav/v2/mymail%40gmail.com/events/qevnuokta4de6k9n705emtub1c_R20190828T221200%40google.com.ics
Skipping missing item: /caldav/v2/mymail%40gmail.com/events/qevnuokta4de6k9n705emtub1c%40google.com.ics
Skipping missing item: /caldav/v2/mymail%40gmail.com/events/qevnuokta4de6k9n705emtub1c_R20190827T221200%40google.com.ics
Saving synchronization database to /Users/lluisma/.calcurse/caldav/sync.db...
0 items imported, 0 items removed locally.
0 items exported, 0 items removed from the server.

I looked inside the sync.db file but the codes are not there. Any clue? Thanks!

lfos commented 4 years ago

These items are not in the sync DB since they're never imported. You need to delete them from the server. Actually, there might be a chance that they're not even on the server (since the server reports a 404 when trying to fetch them). I don't know what to do in this case.