Subscription not fetched by server on creation

Perflyst commented 4 years ago

Steps to reproduce

Add a new iCal calendar

Expected behaviour

Events in calendar should be displayed

Actual behaviour

Nothing is displayed

Calendar app

Calendar app version: 2.0.0

CalDAV-clients used: nan

Client configuration

Browser: Firefox 72.0.1

Operating system: Ubuntu 18.04

Server configuration

Docker 18.0-apache

Updated from an older installed version or fresh install: no

List of activated apps:

Enabled:

accessibility: 1.4.0
activity: 2.11.0
apporder: 0.9.0
bookmarks: 2.3.4
calendar: 2.0.0
cloud_federation_api: 1.1.0
comments: 1.8.0
contacts: 3.1.6
dav: 1.14.0
federatedfilesharing: 1.8.0
federation: 1.8.0
files: 1.13.1
files_pdfviewer: 1.7.0
files_rightclick: 0.15.2
files_sharing: 1.10.1
files_trashbin: 1.8.0
files_versions: 1.11.0
files_videoplayer: 1.7.0
firstrunwizard: 2.7.0
logreader: 2.3.0
lookup_server_connector: 1.6.0
nextcloud_announcements: 1.7.0
notes: 3.1.1
notifications: 2.6.0
oauth2: 1.6.0
password_policy: 1.8.0
photos: 1.0.0
provisioning_api: 1.8.0
settings: 1.0.0
sharebymail: 1.8.0
systemtags: 1.8.0
tasks: 0.11.3
text: 2.0.0
theming: 1.9.0
twofactor_backupcodes: 1.7.0
twofactor_totp: 4.1.2
twofactor_u2f: 5.0.2
updatenotification: 1.8.0
user_ldap: 1.8.0
viewer: 1.2.0
workflowengine: 2.0.0

Nextcloud configuration:

{
    "system": {
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "overwriteprotocol": "https",
        "htaccess.RewriteBase": "\/",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "***REMOVED SENSITIVE VALUE***"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "18.0.0.10",
        "overwrite.cli.url": "https:\/\/***REMOVED SENSITIVE VALUE***",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "ldapIgnoreNamingRules": false,
        "ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory",
        "ldapUserCleanupInterval": 0,
        "theme": "",
        "loglevel": 2,
        "maintenance": false
    }
}

Logs

Nothing logged in browser or webserver error log

georgehrke commented 4 years ago

We changed the way subscriptions work in version 2. Subscriptions are not fetched by the browser anymore, but they are fetched in a cron-job on the server and cached.

You should hence make sure to properly configure the cron-job in the Nextcloud admin settings. They should appear after some while. What is missing at the moment is that the server does not immediately cache it on creation, but only after the cron ran the first time. I think there is an issue in the server repository for that.

Perflyst commented 4 years ago

looks like i did not wait long enough ;) could not find the issue, we should link it here before closing this issue

gnilebein commented 4 years ago

My cronjob runs every 5 Minutes, but the calendar is still not available after > 12 hours..

georgehrke commented 4 years ago

@gnilebein Can you please post the contents of the oc_jobs table? Feel free to replace personal data in the arguments column.

gnilebein commented 4 years ago

https://pastebin.com/VNQHcVKQ

Another strange thing is that the calendar no longer uses the X-WR-CALNAME for the name. It uses the Domain name as calendar name.

georgehrke commented 4 years ago

In oc_calendarsubscriptions, fetch the row with uri = heusenstammmein-abfallkalenderde-1 and principaluri = principals/users/patrick@gnilebein.de. Take the id of that row and check in oc_calendarobjects for calendarid = row id and calendartype = 1. Are there any rows matching that calendarid and calendartype?

gnilebein commented 4 years ago

It got an empty set of data: https://pastebin.com/5zfPPWGU

gnilebein commented 4 years ago

I have a second calender that does not work:

https://dienstplan.ffhstm.de/kalender.php?abteilung=ff3,kf3

I have access to server logs of dienstplan.ffhstm.de. When i add the calendar i can see a request:

5.45.108.193 - - [19/Jan/2020:17:52:05 +0100] "GET /kalender.php?abteilung=ff3,kf3 HTTP/1.1" 200 25228 "-" "Nextcloud Webcal Crawler"

But when i run the cronjob manually, there is not further request to the server.

An with a reload of the nextcloud calendar page it got this error message in debugging console:

https://ibb.co/SVkHd8T

Perflyst commented 4 years ago

To your empty data set, it actually gets a 301 (which is not handled by the app), try http://www.heusenstamm.mein-abfallkalender.de/ical.ics?sid=24360&cd=attachment&ft=6&fp=next_1000&wids=626,624,625,627,623,629,628&uid=110061&pwid=552dad48ff&cid=91&fn=Mein-Abfallkalender_Stadt_Heusenstamm_Zuk%c3%bcnftige_Termine.ics

This link works fine and shows the events in the calendar

gnilebein commented 4 years ago

To your empty data set, it actually gets a 301 (which is not handled by the app), try http://www.heusenstamm.mein-abfallkalender.de/ical.ics?sid=24360&cd=attachment&ft=6&fp=next_1000&wids=626,624,625,627,623,629,628&uid=110061&pwid=552dad48ff&cid=91&fn=Mein-Abfallkalender_Stadt_Heusenstamm_Zuk%c3%bcnftige_Termine.ics

This link works fine and shows the events in the calendar

That was the solution. Thanks.

gnilebein commented 4 years ago

But is still have Problems with my other calender subscription:

oc_calendarsubscriptions: https://pastebin.com/9krGCMax oc_calendarobjects: https://pastebin.com/2CywG3PA Screenshot Developer Console: https://ibb.co/2YkhYyQ

manfredu commented 4 years ago

After updating the calendar app to 2.0.0 and 2.0.1 my subscriptions do not get updated any longer. NC 17.0.2.

georgehrke commented 4 years ago

@manfredu Same as above, please make sure cron is configured properly and running often enough.

manfredu commented 4 years ago

@manfredu Same as above, please make sure cron is configured properly and running often enough.

According to /settings/admin it is using cron and running every 5 minutes:

What's the best way to check that it is really doing what it should?

gnilebein commented 4 years ago

@manfredu Same as above, please make sure cron is configured properly and running often enough.

According to /settings/admin it is using cron and running every 5 minutes:

What's the best way to check that it is really doing what it should?

Check out the posts above. It describes which tables you have to check.

However, I still don't understand why I get a 404 for the calendar in the console.

Can I give more information to solve the problem?

georgehrke commented 4 years ago

@gnilebein Can you please open a separate issue in the server repository. Please mention me in that ticket, i will take a look. It seems to be separate from not caching on creation.

palegrand commented 4 years ago

since I upgraded calendar 2.00 --> 2.01 subscriptions are not displayed anymore.
Adding new subscription (to a known working .ics) does not show anything either, even after multiple cron runs. Nextclouds own calendars work fine. only calender got upgraded. Nextcloud 17.0.2

georgehrke commented 4 years ago

since I upgraded calendar 2.00 --> 2.01 subscriptions are not displayed anymore.

Subscriptions don't appear in the calendar-list anymore or the contents of subscription does no longer show in the calendar-grid?

Can you please check the browser / javascript log?

palegrand commented 4 years ago

Subscriptions don't appear in the calendar-list anymore or the contents of subscription does no longer show in the calendar-grid?

I can see the entry in the calendar-list on the left hand side as before, but the contents do not show anymore (neither old nor new ones). select/unselect the calendar does make a difference

--> is it safe to downgrade from 2.01 to 2.00? Displaying subscriptions at the same time is one of the most important features to me (coordinate jobs, leisure and family...)

Goose-Tech commented 4 years ago

I enabled cron using the linked instructions. One calendar updated immediately, the second calendar is stuck. It is simply spinning. This appears to be isolated to the nextcloud server as my DAVx5 Android app picked up the new calendar and synced all the information correctly. Something's still wrong with the local display of the calendar's entries.

palegrand commented 4 years ago

seems to be cron-related, indeed. I finally started from shell in the nextcloud folder a php -f cron.php it ran a long time - and all entries are visible again :-) it seems to work only with 'Cron'. AJAX and Webcron come back quickly - but did/do NOT work.

manfredu commented 4 years ago

Something seems to be wrong with subscriptions. The cron job runs regularly, but still some calendar events are not display or not changed when they should etc.

One example: A row from calenderobjects contains

BEGIN:VCALENDAR VERSION:2.0 PRODID:-//Sabre//Sabre VObject 4.1.6//EN CALSCALE:GREGORIAN BEGIN:VEVENT DTSTAMP:20191009T120827Z DTSTART;VALUE=DATE:20200115 SUMMARY:Grobsperrgut UID:91f27aa8-fa64-4981-b52d-51ba3f191d4c X-MICROSOFT-CDO-ALLDAYEVENT:TRUE LOCATION:Zone H DESCRIPTION:Grobsperrgut Zone H CATEGORIES:Zone H Grobsperrgut END:VEVENT END:VCALENDAR

Shouldn't that be displayed on the 15.01.? It is not....

georgehrke commented 4 years ago

@manfredu Can you please open a separate issue for that, using the entire issue template including all requested log files. Thx.

georgehrke commented 4 years ago

I'm moving this issue to the server, since not caching the subscription on first creation is a server bug.

Other subscription related issues, that you are not sure of whether they are server or calendar app related, should still go into the calendar repo, so we can triage them.

georgehrke commented 4 years ago

Needs a backport to 17

manfredu commented 4 years ago

@manfredu Can you please open a separate issue for that, using the entire issue template including all requested log files. Thx.

https://github.com/nextcloud/calendar/issues/1899

palegrand commented 4 years ago

I still have the issue in 17.0.3. New events are not displayed. Cron runs without problem. Adding a new calender subscription with the very same .ics-link give me (after the first cron-run) a complete calendar with all events. But events added after this moment are not shown (neither the old nor the new subscription) I understand this might be fixed in 18. is there anything I can do in 17.0.3?

georgehrke commented 4 years ago

@palegrand See https://github.com/nextcloud/server/pull/19229 The patch will be included in 17.0.4

palegrand commented 4 years ago

@georgehrke Thank you - I updated two hours ago to 18.0.1 and hoped it would work there right away. Unfortunately it doesn't (same as 17.0.3 above)

palegrand commented 4 years ago

create subscription: new calendar show with all entries after first cron run --> ok
entries never get updated, cron does call \UpdateCalendarResourcesRoomsBackgroundJob, executes without error but does NOT show updates in the calendar.
Verify behaviour: delete subscription, recreate it: calendar shows correct with all recent entries after first cron run - but never updates after creation (as above
other calendars work/update fine. It only affects subsriptions. subscritptions used to work properly in calendar <2.0.1 current: stable (nextcloud 18.0.2, calendar 2.0.2)

Mirko607 commented 2 years ago

I am facing the same issue as @palegrand did.

Has there been opened a seperate issue?

Where and how could I provide more information if needed?

nextcloud / server