CalDav Bugfix Collection for Nathaniel (maker of tsdav)

PeerRich commented 2 years ago

thank you Nathaniel (@natelindev ) for offering your help.

for context, we use https://github.com/natelindev/tsDAV by @natelindev

some of these may be out of date / fixed already

[x] upgrade to latest tsDAV and test https://github.com/calendso/calendso/pull/993
[x] research a list of most common calendars that use CalDav
[x] CalDav integration with Fastmail doesn't work #626
[x] Events from CalDAV not taken into consideration when checking availability #624
[x] Getting started - input gets reset when switching tabs #622
[x] Issue creating bookings in Caldav #589
[ ] CalDAV integration with Baikal does not work #588
[x] Caldav integration is creating events in every calendars #537
[ ] CalDAV availability checking is slow (30+ seconds) #479
[x] CalDAV timezone seems to be incorrect (Nextcloud/Zimbra CalDAV server) #467
[ ] Require encryption key for CalDAV and Apple #645
[ ] CALDAV Integration with KDAV (Kopano) does not work #510
[ ] CalDav/Lark Suite can't connect to Cal - https://github.com/calcom/cal.com/issues/2989
[ ] Nextcloud - JSON error on booking but it the booking goes through - https://github.com/calcom/cal.com/issues/2775
[ ] CalDav/Zimbra - existing events (set to busy) don't reflect in the availability.

natelindev commented 2 years ago

I'll be working on these this Sunday.

PeerRich commented 2 years ago

let me know if you have any blockers 🙏

natelindev commented 2 years ago

let me know if you have any blockers 🙏

I think it's better to add me to project collaborators list, I can work with forking the repo but it's not very elegant.

natelindev commented 2 years ago

For task upgrade to latest tsDAV and test: since repo is currently filled with typescript errors, how do I know if I didn't break anything? also the e2e tests are failing too.

natelindev commented 2 years ago

list of calendar services that supports caldav:

1CRM A1.net all-inkl.com AOL Mail Baïkal Bitrix24 Calendar and Contacts Server cPanel Cyrus IMAP DAViCal DavMail Gateway Davis Daylite DingTalk / 钉钉 / Dīngdīng Dolibarr EDIS EGroupware Fastmail fruux GMX Google Group Office Hetzner KonsoleH Horde IceWarp iCloud Kerio Connect Kolab Now Kopano Liberta luckycloud macOS Server mail.de mail.ru mailbox.org Mailfence MDaemon Nextcloud openCRX Open-Xchange / OX App Suite Oracle Beehive Oracle Communications UCS ownCloud OwnCube Posteo Radicale Roundcube SmarterMail SOGo Synology DSM Teambox Tine 2.0 T-Online web.de Xandikos Yahoo Mail! Yandex Zaclys Zimbra Zoho

PeerRich commented 2 years ago

For task upgrade to latest tsDAV and test: since repo is currently filled with typescript errors, how do I know if I didn't break anything? also the e2e tests are failing too.

@KATT ?

natelindev commented 2 years ago

After some investigation of current implementation of caldav integration, I believe it need some major changes. Currently fastmail, nextcloud works fine with tsdav integration tests, while not working at all in calendso.

jdaviescoates commented 2 years ago

while not working at all in calendso.

Yeah, I think CalDAV ought to be unchecked over on #62 @baileypumfleet because integration that doesn't actually work isn't really integration at all. Really great product though, so can't wait to begin using it once CalDAV works!

PeerRich commented 2 years ago

After some investigation of current implementation of caldav integration, I believe it need some major changes. Currently fastmail, nextcloud works fine with tsdav integration tests, while not working at all in calendso.

hmm that's bad news. I dm'ed you!

PeerRich commented 2 years ago

Nextcloud & Fastmail has been fixed: https://github.com/calendso/calendso/pull/1300, thank you @llldar

sydwardrae commented 2 years ago

Hey @natelindev , i have a customer with the following issues : a customer saying that their Appointments from CalDAV don’t appear as busy slots - they’re using Woelkli.com, a Nextcloud server.

From what I can see, their integrations are correct so I think it's a bug. Please let me know if I can provide further information - thank you!

PeerRich commented 2 years ago

one user is reporting:

I successfully generated an app-specific password on Apple. BUT when I enter it on Cal, it says "could not add this caldav account".

saschafoerster commented 2 years ago

I am using Kerio Connect as a CalDAV source and I get the following error: "invalid ical body. component began but did not end" I am not shure, if this is a bug of Kerio, tsdav or cal.com, but I am ready to find out and help (just I don't know exactly how to find the right information). All I could find is this log-file:

2|calcom | @calcom/web:start: 21:38:39.651 timeZoneName WARN CalendarManager 
2|calcom | @calcom/web:start:   at new ParserError (/home/cal.com/node_modules/ical.js/build/ical.js:1756:13)
2|calcom | @calcom/web:start:   at Object.parser [as parse] (/home/cal.com/node_modules/ical.js/build/ical.js:1794:13)
2|calcom | @calcom/web:start:   at /home/cal.com/apps/web/.next/server/chunks/3372.js:6252:77
2|calcom | @calcom/web:start:   at Array.forEach (<anonymous>)
2|calcom | @calcom/web:start:   at CalDavCalendarService.getAvailability (/home/cal.com/apps/web/.next/server/chunks/3372.js:6250:17)
2|calcom | @calcom/web:start:   at runMicrotasks (<anonymous>)
2|calcom | @calcom/web:start:   at processTicksAndRejections (internal/process/task_queues.js:95:5)
2|calcom | @calcom/web:start:   at async Promise.all (index 0)
2|calcom | @calcom/web:start:   at async getBusyCalendarTimes (/home/cal.com/apps/web/.next/server/chunks/6477.js:105:19)
2|calcom | @calcom/web:start:   at async handler (/home/cal.com/apps/web/.next/server/pages/api/availability/[user].js:517:23)
2|calcom | @calcom/web:start:   at async Object.apiResolver (/home/cal.com/node_modules/next/dist/server/api-utils/node.js:184:9)
2|calcom | @calcom/web:start:   at async NextNodeServer.runApi (/home/cal.com/node_modules/next/dist/server/next-server.js:395:9)
2|calcom | @calcom/web:start:   at async Object.fn (/home/cal.com/node_modules/next/dist/server/base-server.js:477:37)
2|calcom | @calcom/web:start:   at async Router.execute (/home/cal.com/node_modules/next/dist/server/router.js:243:32)
2|calcom | @calcom/web:start:   at async NextNodeServer.run (/home/cal.com/node_modules/next/dist/server/base-server.js:594:29)
2|calcom | @calcom/web:start:   at async NextNodeServer.handleRequest (/home/cal.com/node_modules/next/dist/server/base-server.js:299:20) {
2|calcom | @calcom/web:start:  message: 'invalid ical body. component began but did not end'
2|calcom | @calcom/web:start: }

PeerRich commented 2 years ago

hey @natelindev can you check out Kerio Connect as a CalDAV source?

saschafoerster commented 2 years ago

Would love to help debugging, if I can do something let me know.

PeerRich commented 2 years ago

Would love to help debugging, if I can do something let me know.

that would be great!

saschafoerster commented 2 years ago

With version 1.6.2 I can still connect (and reconnect) caldav with Kerio as caldav-server. I use this URL https://kerio.domain.tld/caldav/users/DOMAIN/USER/ and I can see my connected caldav-account: https://kerio.domain.tld/full-calendars/DOMAIN/USER/INDIVIDUAL NUMBER/ The error message I get now has changed to:

TypeError: undefined is not an object (evaluating 'h.length') when I check the connection here: https://calcom.domain/availability/troubleshoot

PeerRich commented 2 years ago

we've assigned a new engineer to some of those bugs @alishaz-polymath

zomars commented 2 years ago

Found a bug that was appearing in Zoho calendars: https://github.com/mozilla-comm/ical.js/issues/514

zomars commented 2 years ago

Also made a temp fix that prevents blocking availability 92de793cb9091c0f5d4a4901aa375606c27c5630

saschafoerster commented 2 years ago

Hi there, with debug-mode on; i found another (similar) error message in browser console, saying that "operation was aborted and: "invalid ical body. component began but did not end"

1|calcom  | @calcom/web:start: 19:39:35.830 timeZoneName WARN CalendarManager 
1|calcom  | @calcom/web:start:   at new ParserError (/home/cal.com/node_modules/ical.js/build/ical.js:1756:13)
1|calcom  | @calcom/web:start:   at Object.parser [as parse] (/home/cal.com/node_modules/ical.js/build/ical.js:1794:13)
1|calcom  | @calcom/web:start:   at /home/cal.com/apps/web/.next/server/chunks/4270.js:224:77
1|calcom  | @calcom/web:start:   at Array.forEach (<anonymous>)
1|calcom  | @calcom/web:start:   at CalDavCalendarService.getAvailability (/home/cal.com/apps/web/.next/server/chunks/4270.js:222:17)
1|calcom  | @calcom/web:start:   at processTicksAndRejections (internal/process/task_queues.js:95:5)
1|calcom  | @calcom/web:start:   at async /home/cal.com/apps/web/.next/server/chunks/5041.js:4461:116
1|calcom  | @calcom/web:start:   at async Promise.all (index 0)
1|calcom  | @calcom/web:start:   at async getBusyCalendarTimes (/home/cal.com/apps/web/.next/server/chunks/5041.js:4470:19)
1|calcom  | @calcom/web:start:   at async getBusyTimes (/home/cal.com/apps/web/.next/server/chunks/5041.js:4947:35)
1|calcom  | @calcom/web:start:   at async getUserAvailability (/home/cal.com/apps/web/.next/server/chunks/5041.js:5068:23)
1|calcom  | @calcom/web:start:   at async /home/cal.com/apps/web/.next/server/chunks/6105.js:85:28
1|calcom  | @calcom/web:start:   at async Object.apiResolver (/home/cal.com/node_modules/next/dist/server/api-utils/node.js:185:9)
1|calcom  | @calcom/web:start:   at async NextNodeServer.runApi (/home/cal.com/node_modules/next/dist/server/next-server.js:395:9)
1|calcom  | @calcom/web:start:   at async Object.fn (/home/cal.com/node_modules/next/dist/server/base-server.js:496:37)
1|calcom  | @calcom/web:start:   at async Router.execute (/home/cal.com/node_modules/next/dist/server/router.js:226:36) {
1|calcom  | @calcom/web:start:  message: 'invalid ical body. component began but did not end'
1|calcom  | @calcom/web:start: }
1|calcom  | @calcom/web:start: PerformanceEntry {
1|calcom  | @calcom/web:start:  name: 'Measuring endpoint: /api/availability/saschafoerster?dateFrom=2022-06-27T22:00:00Z&dateTo=2022-06-28T21:59:59Z',
1|calcom  | @calcom/web:start:  entryType: 'measure',
1|calcom  | @calcom/web:start:  startTime: 1008360.841986,
1|calcom  | @calcom/web:start:  duration: 1681.123231
1|calcom  | @calcom/web:start: }

zomars commented 2 years ago

@alishaz-polymath would you kindly investigate?

alishaz-polymath commented 2 years ago

@zomars On it

saschafoerster commented 2 years ago

@zomars @alishaz-polymath With the newest version 1.7.7, I get an error message in the web-interface again:

Something went wrong.
TypeError: undefined is not an object (evaluating 'p.length')

Browser-console:

[Error]  << query #2 viewer.me 
{input: undefined, result: TRPCClientError: The operation was aborted., elapsedMs: 364, context: {}}

    (anonyme Funktion) (_app-4c5647c7057e4ddc.js:1:136386)
    (anonyme Funktion) (_app-4c5647c7057e4ddc.js:1:136854)
    p (_app-4c5647c7057e4ddc.js:1:10532)
    (anonyme Funktion) (_app-4c5647c7057e4ddc.js:1:10553)
    onDone (_app-4c5647c7057e4ddc.js:1:12283)
    done (_app-4c5647c7057e4ddc.js:1:11513)
    (anonyme Funktion) (_app-4c5647c7057e4ddc.js:1:13829)
    t (_app-4c5647c7057e4ddc.js:1:213143)
    (anonyme Funktion) (_app-4c5647c7057e4ddc.js:1:199966)
    (anonyme Funktion) (_app-4c5647c7057e4ddc.js:1:223010)
    (anonyme Funktion) (_app-4c5647c7057e4ddc.js:1:222692)
    (anonyme Funktion) (_app-4c5647c7057e4ddc.js:1:214125)
    (anonyme Funktion) (_app-4c5647c7057e4ddc.js:1:231789)
    Gu (framework-dbb4adbd2655d20c.js:1:92950)
    Ju (framework-dbb4adbd2655d20c.js:1:93147)
    ws (framework-dbb4adbd2655d20c.js:1:112985)
    (anonyme Funktion) (framework-dbb4adbd2655d20c.js:1:110375)
    S (framework-dbb4adbd2655d20c.js:1:131185)
    T (framework-dbb4adbd2655d20c.js:1:131719)
[Error] Failed to load resource: the server responded with a status of 500 (Internal Server Error) (saschafoerster, line 0)
[Error] TypeError: undefined is not an object (evaluating 'p.length') — troubleshoot-94184235e0a488c8.js:0
    uu (framework-dbb4adbd2655d20c.js:1:71390)
    (anonyme Funktion) (framework-dbb4adbd2655d20c.js:1:71916)
    sa (framework-dbb4adbd2655d20c.js:1:50763)
    gi (framework-dbb4adbd2655d20c.js:1:100832)
    vi (framework-dbb4adbd2655d20c.js:1:100400)
    hi (framework-dbb4adbd2655d20c.js:1:99936)
    (anonyme Funktion) (framework-dbb4adbd2655d20c.js:1:112071)
    ks (framework-dbb4adbd2655d20c.js:1:112533)
    rs (framework-dbb4adbd2655d20c.js:1:105048)
    rs
    S (framework-dbb4adbd2655d20c.js:1:131185)
    T (framework-dbb4adbd2655d20c.js:1:131719)

Logfiles:

1|calcom   | @calcom/web:start: [OK][0.0004s] query 'viewer.me'
1|calcom   | @calcom/web:start: [OK][0.0007s] query 'viewer.me'
1|calcom   | @calcom/web:start: [ERROR][0.0121s] GET '/api/availability/saschafoerster?dateFrom=2022-07-12T22:00:00Z&dateTo=2022-07-13T21:59:59Z'

alishaz-polymath commented 2 years ago

@saschafoerster Thanks for sharing. I'll check this out 🙏

saschafoerster commented 2 years ago

And the logs with debug-mode on:

1|calcom  | @calcom/web:start: [OK][0.0050s] query 'viewer.availability.schedule'
1|calcom  | @calcom/web:start: [OK][0.0052s] query 'viewer.availability.schedule'
1|calcom  | @calcom/web:start: prisma:query SELECT "public"."users"."id", "public"."users"."username", "public"."users"."name", "public"."users"."email", "public"."users"."emailVerified", "public"."users"."password", "public"."users"."bio", "public"."users"."avatar", "public"."users"."timeZone", "public"."users"."weekStart", "public"."users"."startTime", "public"."users"."endTime", "public"."users"."bufferTime", "public"."users"."hideBranding", "public"."users"."theme", "public"."users"."created", "public"."users"."trialEndsAt", "public"."users"."defaultScheduleId", "public"."users"."completedOnboarding", "public"."users"."locale", "public"."users"."timeFormat", "public"."users"."twoFactorSecret", "public"."users"."twoFactorEnabled", "public"."users"."identityProvider", "public"."users"."identityProviderId", "public"."users"."invitedTo", "public"."users"."plan", "public"."users"."brandColor", "public"."users"."darkBrandColor", "public"."users"."away", "public"."users"."allowDynamicBooking", "public"."users"."metadata", "public"."users"."verified", "public"."users"."role", "public"."users"."disableImpersonation" FROM "public"."users" WHERE "public"."users"."email" = $1 LIMIT $2 OFFSET $3
1|calcom  | @calcom/web:start: prisma:query SELECT "public"."users"."id", "public"."users"."timeZone", "public"."users"."bufferTime", "public"."users"."startTime", "public"."users"."endTime", "public"."users"."defaultScheduleId" FROM "public"."users" WHERE "public"."users"."username" = $1 LIMIT $2 OFFSET $3
1|calcom  | @calcom/web:start: prisma:query SELECT "public"."Credential"."id", "public"."Credential"."type", "public"."Credential"."key", "public"."Credential"."userId", "public"."Credential"."appId" FROM "public"."Credential" WHERE "public"."Credential"."userId" IN ($1) OFFSET $2
1|calcom  | @calcom/web:start: prisma:query SELECT "public"."Availability"."id", "public"."Availability"."userId", "public"."Availability"."eventTypeId", "public"."Availability"."days", "public"."Availability"."startTime", "public"."Availability"."endTime", "public"."Availability"."date", "public"."Availability"."scheduleId" FROM "public"."Availability" WHERE "public"."Availability"."userId" IN ($1) OFFSET $2
1|calcom  | @calcom/web:start: prisma:query SELECT "public"."SelectedCalendar"."userId", "public"."SelectedCalendar"."integration", "public"."SelectedCalendar"."externalId" FROM "public"."SelectedCalendar" WHERE "public"."SelectedCalendar"."userId" IN ($1) OFFSET $2
1|calcom  | @calcom/web:start: prisma:query SELECT "public"."Schedule"."id", "public"."Schedule"."timeZone", "public"."Schedule"."userId" FROM "public"."Schedule" WHERE "public"."Schedule"."userId" IN ($1) OFFSET $2
1|calcom  | @calcom/web:start: prisma:query SELECT "public"."Availability"."id", "public"."Availability"."userId", "public"."Availability"."eventTypeId", "public"."Availability"."days", "public"."Availability"."startTime", "public"."Availability"."endTime", "public"."Availability"."date", "public"."Availability"."scheduleId" FROM "public"."Availability" WHERE "public"."Availability"."scheduleId" IN ($1) OFFSET $2
1|calcom  | @calcom/web:start: [ERROR][0.0431s] GET '/api/availability/saschafoerster?dateFrom=2022-07-12T22:00:00Z&dateTo=2022-07-13T21:59:59Z'
1|calcom  | @calcom/web:start: prisma:query SELECT "public"."users"."id", "public"."users"."username", "public"."users"."name", "public"."users"."email", "public"."users"."bio", "public"."users"."timeZone", "public"."users"."weekStart", "public"."users"."startTime", "public"."users"."endTime", "public"."users"."defaultScheduleId", "public"."users"."bufferTime", "public"."users"."theme", "public"."users"."created", "public"."users"."hideBranding", "public"."users"."avatar", "public"."users"."twoFactorEnabled", "public"."users"."disableImpersonation", "public"."users"."identityProvider", "public"."users"."brandColor", "public"."users"."darkBrandColor", "public"."users"."plan", "public"."users"."away", "public"."users"."completedOnboarding", "public"."users"."locale", "public"."users"."timeFormat", "public"."users"."trialEndsAt" FROM "public"."users" WHERE "public"."users"."id" = $1 LIMIT $2 OFFSET $3
1|calcom  | @calcom/web:start: prisma:query SELECT "public"."Credential"."id", "public"."Credential"."type", "public"."Credential"."key", "public"."Credential"."userId", "public"."Credential"."appId" FROM "public"."Credential" WHERE "public"."Credential"."userId" IN ($1) ORDER BY "public"."Credential"."id" ASC OFFSET $2
1|calcom  | @calcom/web:start: prisma:query SELECT "public"."SelectedCalendar"."userId", "public"."SelectedCalendar"."integration", "public"."SelectedCalendar"."externalId" FROM "public"."SelectedCalendar" WHERE "public"."SelectedCalendar"."userId" IN ($1) OFFSET $2
1|calcom  | @calcom/web:start: prisma:query SELECT "public"."DestinationCalendar"."id", "public"."DestinationCalendar"."integration", "public"."DestinationCalendar"."externalId", "public"."DestinationCalendar"."userId", "public"."DestinationCalendar"."bookingId", "public"."DestinationCalendar"."eventTypeId", "public"."DestinationCalendar"."credentialId" FROM "public"."DestinationCalendar" WHERE "public"."DestinationCalendar"."userId" IN ($1) OFFSET $2
1|calcom  | @calcom/web:start: [OK][0.0001s] query 'viewer.me'
1|calcom  | @calcom/web:start: [OK][0.0003s] query 'viewer.me'
1|calcom  | @calcom/web:start: [OK][0.0004s] query 'viewer.me'

PeerRich commented 2 years ago

@alishaz-polymath feel free to close this issue and move all open issues into a new thread

alishaz-polymath commented 2 years ago

@PeerRich Will do. I'm currently exploring exactly how we handle CalDav and just studying it as a whole. Once I have a set of actions ready, I'll do this 🙏

alishaz-polymath commented 2 years ago

Hey everyone, thanks a lot for your help and feedback. For a cleaner hunting, closing this in favor of #3457 🙏

calcom / cal.com

CalDav Bugfix Collection for Nathaniel (maker of tsdav) #956