runbox / runbox7

Runbox 7 web app
Other
126 stars 24 forks source link

#Calendar: Empty calendar with recurring events #932

Open eoli3n opened 3 years ago

eoli3n commented 3 years ago

I notice that the calendar app show empty data this morning and keep running "Refreshing events" endlessly.

From console

Error: Same occurrence found twice, protecting you from death by recursion
    next ical.js:7229
    next ical.js:8540
    generateEvents calendar.service.ts:410
    generateEvents calendar.service.ts:365
    updateEventList calendar.service.ts:572
    beforeViewRender calendar-app.component.ts:190
    Kr calendar-app.component.html:242
    Angular 3
    RxJS 5
    Angular 31
    RxJS 5
    Angular 11
castaway commented 3 years ago

Hi, looks like you might have found a recurrence style that the generator (of recurring events) can't cope with, do you recall which event / type of recurrence you had just added? You can login and see them directly via the info on this page https://help.runbox.com/using-a-calendar-client-with-caldav/ or via a third party client if you need to review.

kotecky commented 3 years ago

I'm not sure whether the error i got is or is not the same, the symptoms are (Loading local cache and Refreshing events wheels spinning). Could it be because of the amount of events loaded at a time? (two 2Mb files: events_raw, mdi.svg)

Recently i also added a recurrent event. But can't say whether there is a causality or not.

10 minutes after page load:

image

Error: Uncaught (in promise): TypeError: t is undefined eeur/Ai</e.prototype.fixICALEvent@https://runbox.com/app/calendar-app-calendar-app-module.f458c39ce210ecdfc1a7.js:1:364673 eeur/Ai</e.prototype.generateEvents/<@https://runbox.com/app/calendar-app-calendar-app-module.f458c39ce210ecdfc1a7.js:1:366679 eeur/Ai</e.prototype.generateEvents@https://runbox.com/app/calendar-app-calendar-app-module.f458c39ce210ecdfc1a7.js:1:366490 e/</<@https://runbox.com/app/calendar-app-calendar-app-module.f458c39ce210ecdfc1a7.js:1:362109 e/<@https://runbox.com/app/calendar-app-calendar-app-module.f458c39ce210ecdfc1a7.js:1:362048 r/</l</e.prototype.invoke@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:7581 onInvoke@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:1202642 r/</l</e.prototype.invoke@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:7521 r/</c</t.prototype.run@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:2681 P/<@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:14602 r/</l</e.prototype.invokeTask@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:8265 onInvokeTask@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:1202452 r/</l</e.prototype.invokeTask@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:8186 r/</c</t.prototype.runTask@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:3344 _@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:10581 r/</f</t.invokeTask@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:9426 y@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:22446 g@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:22753 EventListener.handleEventm/D<@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:23598 r/</l</e.prototype.scheduleTask@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:8006 onScheduleTask@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:5082 r/</l</e.prototype.scheduleTask@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:7905 r/</c</t.prototype.scheduleTask@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:3915 r/</c</t.prototype.scheduleEventTask@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:4364 I/<@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:25240 2YQP/fe</e.prototype.wrapEventTarget/</<@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:167313 ge/</</</<@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:170148 value/<@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:1529752 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:498308 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:498092 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:1348592 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:497970 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:517414 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:497970 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:1375273 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:497970 h@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:2388014 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:255326 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:255211 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:255032 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:285651 i/<@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:1375617 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:498308 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:498092 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:254608 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:497970 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:1424296 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:497970 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:1348592 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:497970 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:1348592 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:497970 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:1348592 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:497970 e@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:1286993 jy/R/T</e.ɵfac@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:1300523 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:1139742 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:1137909 n/<@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:452660 s/</<@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:452384 s/<@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:452489 a@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:452883 r/</l</e.prototype.invoke@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:7581 onInvoke@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:1202642 r/</l</e.prototype.invoke@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:7521 r/</c</t.prototype.run@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:2681 P/<@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:14602 r/</l</e.prototype.invokeTask@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:8265 onInvokeTask@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:1202452 r/</l</e.prototype.invokeTask@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:8186 r/</c</t.prototype.runTask@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:3344 @https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:10581 r/</f</t.invokeTask@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:9426 y@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:22446 g@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:22753 EventListener.handleEventm/D<@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:23598 r/</l</e.prototype.scheduleTask@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:8006 onScheduleTask@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:5082 r/</l</e.prototype.scheduleTask@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:7905 r/</c</t.prototype.scheduleTask@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:3915 r/</c</t.prototype.scheduleEventTask@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:4364 I/<@https://runbox.com/app/polyfills.141665300f3c6566463f.js:1:25240 2YQP/fe</e.prototype._wrapEventTarget/</<@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:167313 ge/</</</<@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:170148 value/<@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:1529752 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:498308 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:498092 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:1348592 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:497970 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:517414 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:497970 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:1375273 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:497970 h@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:2388014 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:255326 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:255211 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:255032 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:285651 i/<@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:1375617 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:498308 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:498092 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:254608 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:497970 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:1424296 value@https://runbox.com/app/main.40c2a8172f6e28915785.js:1:497970 Angular 18 _wrapEventTarget trycatch.js:74 ge instrument.js:316 value Angular RxJS 26 e rbwebmail.ts:198 fac main.40c2a8172f6e28915785.js:1 Angular 2 n sentry-error-handler.ts:45 s main.40c2a8172f6e28915785.js:1 s main.40c2a8172f6e28915785.js:1 a main.40c2a8172f6e28915785.js:1 Angular 20 _wrapEventTarget trycatch.js:74 ge instrument.js:316 value Angular RxJS 20 instrument.js:110:45 ge instrument.js:110 handleError sentry-error-handler.ts:60 Angular 2 RxJS 5 Angular 21 _wrapEventTarget trycatch.js:74 ge instrument.js:316 value Angular RxJS 26 e rbwebmail.ts:198 fac main.40c2a8172f6e28915785.js:1 Angular 2 n sentry-error-handler.ts:45 s main.40c2a8172f6e28915785.js:1 s main.40c2a8172f6e28915785.js:1 a main.40c2a8172f6e28915785.js:1 Angular 20 _wrapEventTarget trycatch.js:74 ge instrument.js:316 value Angular RxJS 20 TypeError: t is undefined fixICALEvent calendar.service.ts:220 generateEvents calendar.service.ts:389 generateEvents calendar.service.ts:365 reloadEvents calendar.service.ts:503 RxJS 19 u Angular i helpers.js:71 Angular 14 _wrapEventTarget trycatch.js:74 ge instrument.js:316 value Angular RxJS 24 reloadEvents calendar.service.ts:492 e calendar.service.ts:118 RxJS 6 syncCaldav calendar.service.ts:539 RxJS 21 u Angular i helpers.js:71 Angular 14 _wrapEventTarget trycatch.js:74 ge instrument.js:316 value Angular RxJS 13 instrument.js:110:45

eoli3n commented 2 years ago

Still empty calendar, I had an issue about a misparsed isc file on the server. At each davx5 sync, I got an error. I was thinking that removing that wrong ics file will unblock this too, it didn't.

What can I do to debug ?

eoli3n commented 2 years ago

As I use vdirsyncer to get ics files from server, how can I check for that double occurrence ? which field is found multiple times and should appear only once ?

eoli3n commented 2 years ago
➜ cat **/*.ics | grep '^UID:' | sort | uniq -d
UID:6pgj8opn74q68b9j6sq68b9k6sp3ebb260s68b9g6dgmcopj64pjgob5ck@google.com
UID:c9i32phn6cp66b9j61hj2b9k6pj66bb2cphjib9k6kqm4pb36ksjgob2co@google.com

~/.vdirsyncer/khal 
➜ grep -ri 'UID:6pgj8opn74q68b9j6sq68b9k6sp3ebb260s68b9g6dgmcopj64pjgob5ck@google.com'
home/6e6434ad-a0a0-4489-baa0-911fbbad333e.ics:UID:6pgj8opn74q68b9j6sq68b9k6sp3ebb260s68b9g6dgmcopj64pjgob5ck@google.com
home/6e6434ad-a0a0-4489-baa0-911fbbad333e.ics:UID:6pgj8opn74q68b9j6sq68b9k6sp3ebb260s68b9g6dgmcopj64pjgob5ck@google.com
home/6e6434ad-a0a0-4489-baa0-911fbbad333e.ics:UID:6pgj8opn74q68b9j6sq68b9k6sp3ebb260s68b9g6dgmcopj64pjgob5ck@google.com
home/6e6434ad-a0a0-4489-baa0-911fbbad333e.ics:UID:6pgj8opn74q68b9j6sq68b9k6sp3ebb260s68b9g6dgmcopj64pjgob5ck@google.com
home/6e6434ad-a0a0-4489-baa0-911fbbad333e.ics:UID:6pgj8opn74q68b9j6sq68b9k6sp3ebb260s68b9g6dgmcopj64pjgob5ck@google.com
home/6e6434ad-a0a0-4489-baa0-911fbbad333e.ics:UID:6pgj8opn74q68b9j6sq68b9k6sp3ebb260s68b9g6dgmcopj64pjgob5ck@google.com
home/6e6434ad-a0a0-4489-baa0-911fbbad333e.ics:UID:6pgj8opn74q68b9j6sq68b9k6sp3ebb260s68b9g6dgmcopj64pjgob5ck@google.com
home/6e6434ad-a0a0-4489-baa0-911fbbad333e.ics:UID:6pgj8opn74q68b9j6sq68b9k6sp3ebb260s68b9g6dgmcopj64pjgob5ck@google.com
home/6e6434ad-a0a0-4489-baa0-911fbbad333e.ics:UID:6pgj8opn74q68b9j6sq68b9k6sp3ebb260s68b9g6dgmcopj64pjgob5ck@google.com
home/6e6434ad-a0a0-4489-baa0-911fbbad333e.ics:UID:6pgj8opn74q68b9j6sq68b9k6sp3ebb260s68b9g6dgmcopj64pjgob5ck@google.com
home/6e6434ad-a0a0-4489-baa0-911fbbad333e.ics:UID:6pgj8opn74q68b9j6sq68b9k6sp3ebb260s68b9g6dgmcopj64pjgob5ck@google.com
home/6e6434ad-a0a0-4489-baa0-911fbbad333e.ics:UID:6pgj8opn74q68b9j6sq68b9k6sp3ebb260s68b9g6dgmcopj64pjgob5ck@google.com

~/.vdirsyncer/khal 
➜ grep -ri 'c9i32phn6cp66b9j61hj2b9k6pj66bb2cphjib9k6kqm4pb36ksjgob2co@google.com'
home/b3bba4b1-1d79-4a7f-8136-97f4ec2bc089.ics:UID:c9i32phn6cp66b9j61hj2b9k6pj66bb2cphjib9k6kqm4pb36ksjgob2co@google.com
home/b3bba4b1-1d79-4a7f-8136-97f4ec2bc089.ics:UID:c9i32phn6cp66b9j61hj2b9k6pj66bb2cphjib9k6kqm4pb36ksjgob2co@google.com

~/.vdirsyncer/khal 
➜ rm home/6e6434ad-a0a0-4489-baa0-911fbbad333e.ics home/b3bba4b1-1d79-4a7f-8136-97f4ec2bc089.ics

~/.vdirsyncer/khal 
➜ vdirsyncer sync
Syncing my_contacts/contacts
Syncing my_calendars/home
Syncing my_calendars/tasks
Syncing my_calendars/anniversaires
Deleting item 6pgj8opn74q68b9j6sq68b9k6sp3ebb260s68b9g6dgmcopj64pjgob5ck@google.com from my_calendars_remote/home
Deleting item c9i32phn6cp66b9j61hj2b9k6pj66bb2cphjib9k6kqm4pb36ksjgob2co@google.com from my_calendars_remote/home

Testing those two files with icalendar.org gave

Errors

Lines not delimited by CRLF sequence near line # 1Reference: RFC 5545 3.1. Content Lines UID value is not unique, duplicate found near line # 32Reference: RFC 5545 3.8.4.7. Unique Identifier

Refreshing runbox web app : not solved... Maybe I will try to validate all my ics files with icalendar.org. Let's check if there is an api.

kotecky commented 2 years ago

At each davx5 sync

I do too but see no errors. @eoli3n, where do the errors appear?

eoli3n commented 2 years ago

you do too what ? Error appears as push notification. But it was about a misparsed file, that problem is now solved (removed the file on the server with a ticket to the runbox team) and I still have my calendar empty on runbox web app. So that was not the problem.

eoli3n commented 2 years ago

@kotecky are you using simple calendar pro from f-droid on your smartphone ?

Check this out : https://github.com/SimpleMobileTools/Simple-Calendar/issues/1532

eoli3n commented 2 years ago

Ok, I needed to clean duplicates by SUMMARY, I also removed really old entries which has a strange syntax. Here the script I applied

#!/bin/bash

DIR="/home/user/.vdirsyncer/khal/home"
IFS=$'\n' 

# Remove old strange entries which contains a strange UID field
for f in $(egrep -rli 'X-WR-ALARMUID' "$DIR")
do
  grep 'SUMMARY' "$f"
  #rm "$f"               
done

# Remove duplicates by SUMMARY, it removes both files, it doesn't keep anything (DIRTY)
for f in $(cat "$DIR"/* | grep SUMMARY | sort | uniq -d )
do
  for i in $(grep -rl "^$f$" "$DIR")
  do
    # show SUMMARY
    grep 'SUMMARY' "$i"
    #rm "$i"                      
  done
done

Now Runbox web calendar is working well again !

kotecky commented 2 years ago

@kotecky are you using simple calendar pro from f-droid on your smartphone ?

No, i do not.

Check this out : SimpleMobileTools/Simple-Calendar#1532

I do not see multiple entries on my mobile. I had the impression that this bug is about the calendar web app. That is where i (so as you?) see a blank page with no data. Besides the web app i use the Business Calendar app and Thunderbird with no visible issues. In my case this issue (no events in the calendar web app, "refreshing calendars" spinning forever) persists and it should remain open.

In cases where there is a malformed entry, imho the server-side application should reject the incorrect entry. Alternatively, the web app should either parse it if it can or reject it if not but it should tolerate some errors. So as email, the calendars are meant to be used with apps from other vendors and there is always a risk of a malformed entry.

kotecky commented 2 years ago

Now Runbox web calendar is working well again !

Cleaning solves the symptoms reported in your issue but both cases (your and mine) showed that the calendar is not resilient. So @gtandersen might want to reopen it.

kotecky commented 1 year ago

@kotecky are you using simple calendar pro from f-droid on your smartphone ?

@eoli3n No. The errors were (and are) present in a browser.

kotecky commented 1 year ago

@gtandersen @eoli3n , i wondered whether the JSON containing the https://runbox.com/rest/v1/calendar/events_raw (events_raw.json) data file was valid. Validation tool: Notepad++'s plugin JsonTools (command Run tests). No idea what the results tell but there is a number of tests that fail. And no idea whether (if true) these failures would yield the errors visible in the browser console:

image

Let me know if i can help to test - my runbox7 webapp calendar is still empty (Thunderbird and phone app clients work well).

A disconnected (?) idea: should only future events (some recent and all future) be delivered in the raw-events data file? Mine has now 3Mb (1000 events since 2019) which could impact the rendering speed?

kotecky commented 1 year ago

And the same from MS Edge, also under Win10:

image