Create a Google Calendar out of the Schedule and add an ical on the top right

[mariobehling]

@sogko made a nice calendar thingy for FOSSASIA here: https://github.com/sogko/fossasia-2016-google-calendar

This enables us to get an ical as well. Please include the ical here: https://calendar.google.com/calendar/ical/oqrj3a93g17r1pgckrr2sv4klc%40group.calendar.google.com/public/basic.ics

1. [ ] Please create a Google calendar for OpenTechSummit
2. [ ] Find a way to automate so the calendar gets updates regularly (what options are there?)
3. [ ] Get an ical and publish it on the website on the top right with under the label "ical"

[aviaryan]

It looks like sogko's code auto publishes the ics to google calendar. That's good, we can run the calendar generator on Travis but it looks like the google calendar url will change with every update. In other words using sogko's code we won't able to update a calendar, only create a new one. @mariobehling Is that acceptable or the need is to keep updating a single calendar ?

ADDED - If we need to keep updating a single calendar, then one will need to modify sogko's code (in Golang). I don't know much of golang so I am not so sure about it :disappointed: . Calendar package docs are at https://godoc.org/google.golang.org/api/calendar/v3

[aditya1702]

@aviaryan Agree with you Could you help me out with this one?

[championswimmer]

Basically we need to make an iCal (or make n iCals, where n = number of tracks; using @sogko 's track-separation logic, which is nice).

Google Calendar accepts creating a calendar with iCal, so just making an iCal should solve our purpose I am guessing.

[aviaryan]

@championswimmer How do you propose to update the google calendar in case schedule and so ical updates ?

@aditya1702 The challenge with sogko's code is keeping the calendar link same across schedule updates. One solution I see is downloading the *.ics generated by the golang script to the scraper repo by Travis. That way atleast we will have a constant ical link. But the google calendar link is still a problem.

@sogko It would be great if you could help us here a bit.

[sogko]

Warning: Massive wall of text. I'm a douche

Edit: Formatting and such to help with you reading it. Sorry that I basically spewed my thoughts all over my keyboard.

Edit 2: Added recommendation (to use a CalDAV server and publish a webcal directly. Instead of using the approach of generating GCals, iCals etc)

---

Hey guys!

Been meaning to get in here earlier and help where I can, but I got caught up with work.

A little background about the work that I did: (Kinda need to justify/explain why it was done the way it is lol, because I think it isn't clear to most people what it does exactly. Hopefully I can clear up certain things and demystify certain things)

---

1. In the beginning: GCals Started with me wanting to figure out which events to attend because there were too many talks taking place in parallel to each other.

So I started with creating one giant google cal (Master GCal hereafter) that includes all events. I would get a link that would allow user to add the GCal to their calendar.google.com, i.e. https://calendar.google.com/calendar/render?cid=oqrj3a93g17r1pgckrr2sv4klc@group.calendar.google.com

But FOSSASIA2016 had too many events that it was a PITA to see whats going on a particular day in calendar.google.com. That's when I separated it by tracks, so I can essentially filter events by tracks, later on by Location. Kinda manageable.

Soon after, I realised that FOSSASIA2016 schedule is extremely fragile and it gets changed/updated frequently.

But that worked out okay, since I cached the GCal IDs (in data.json), so that if there are any changes, I would have to run the program manually so that it can:

• Go to each calendar (the "Master" and the many Tracks / Location calendars), delete each entry one at a time and re-add them. (GCal API to delete all entries for a given calendar didn't seemed to work).

Those that added the GCals to their own calendar.google.com would see the changes without having to re-import.

2. Can I have it in .ics / iCalendar? Then someone asked if they can have it in .ics so that they can use with their own Calendar app (notably OS X Calendar app, I did not test with other applications)

To achieve that, what I did was simply to get the .ics link from calendar.google.com

• Import the Master GCal into calendar.google.com
• Select the imported calendar, navigate to Calendar Settings for that calendar and get the public .ics link for it.
  • i.e. https://calendar.google.com/calendar/ical/oqrj3a93g17r1pgckrr2sv4klc%40group.calendar.google.com/public/basic.ics

That worked out okay, but there is a little problem: You can import it to your Calendar app, but if the original Master GCal changes (which it did, oh so frequently), the only way to get the latest changes is to:

• re-download the .ics file from calender.google.com (The link was static so it wasn't a big deal)
• re-import to your Calendar app again manually.

No bueno.

So in order to have a calendar that can be updated without having to users to re-import it: it can be done by using webcals.

3. Enter webcal:// Unfortunately, that requires a WebDAV/CalDav server that serves the calendar, so that your fav Calendar app can poll from, say every 1 hour (configurable), and see if there's any changes. I was cheap, with zero budget and zero backing, and I am not affiliated to FOSSASIA, so I had to find a solution.

At that time, there isn't a way to directly get a Webcal from GCal. So my solution is this: import the .ics file into my personal iCloud account, and publish the webcal it as a public calendar.

The steps are as follow: (prepare for the hack/laugh of your life)

Setup

• Generate Master GCal from FOSSASIA2016's sessions.json
• Import Master GCal to calendar.google.com
• Get .ics file for Master GCal from calender.google.com
• Create an iCloud calendar (I named it FOSSASIA 2016) using my personal iCloud account.
• Import the ics into the said created iCloud calendar.
• Share the calendar as a public calendar, and Calendar will give you a webcal:// link, for e.g. webcal://p42-calendars.icloud.com/published/2/QmWuTst1nDopR6oQVVIkV4co0ONQVq4lD2s7XeBGTdDZmn8J4lBQvKzgNWwSpa0w-QQJEdIT3ywlU105GBNXW3zfaDBSPNaBrl-krjqFbz8

Now everyone can add the calendar using the webcal link to their own Calendar app, and if there are changes to source iCloud calendar (i.e. on my personal iCloud account), everyone else will see the changes. Yay.

So essentially if there any changes to the FOSSASIA2016 schedule, I had to do the following:

• Run the program to update the GCals. (This would update the GCals for anyone who imported it to their calendar.google.com)
• Re-download the .ics file for the Master GCal.
• Re-import the .ics to the iCloud calendar that I published previously. (This would update for those that added the calendar to their application thru the webcal link.)

Phew.

Lessons learnt / Notes / Thoughts:

• Each calendar entry must have an unique identifier so that GCal and .ics can eliminate duplicates. Learnt that the hard way.
• For filtering by Location: Had to "scrub" location values manually, since some of the events took place in the same location, but for some reason, the name of the location was different.
• Had to cache calendar ids and calendar_entry ids, to avoid creating duplicate calendar entries when re-importing the ics to the Calendar app.
• Notice that I didn't create separate .ics or webcal links by Tracks or Locations. Why? Cos I already had too much on my plate.
• FOSSASIA2016 had a mobile app for their schedule which allows users to filter / search events. That had better user experience etc. But it doesn't get updated when schedule changes. Not sure why (probably they had to publish a new version every time there are changes? Why not let the app fetch from sessions.json as I did? Anyway, way off tangent)
• service.json is the JSON file that you get when you create a Service Account key for Google Calendar product. So someone needs to create one and generate it. It goes without saying why I didn't include it in the repo. Using different Service Account will create different calendars. Service Account A can't edit/modify calendars for Service Account B.

If I can do it all over again...

• If I can do it all over again, I might skip GCal all together and create an .ics directly. So that I can publish it as a webcal link, which can be imported by most Calendar apps, including calendar.google.com
• But caveat with calendar.google.com + webcal: unlike most Calendar apps, user can't set the refresh interval explicitly. It could take 20 mins or hours, or days before calendar.google.com might refresh the webcal feed. Boo.
• Downside of this approach: can't quite make it fully automated, I believe. One person has to be appointed to take care of the source iCalendar that publishes the webcal link, and he/she has to re-import the updated .ics for the webcal subscribers to get the latest changes.
• Possible workaround this issue (recommended): run a dedicated CalDAV server that publishes the calendar directly. For e.g.: Radicale http://radicale.org (Python-based). Im sure there are other open-sourced CalDAV server implementation or solutions.. You avoid the whole issue of creating GCals / iCals etc. Go straight to webcal.
• Downside to this approach: cost of running and maintaining a CalDAV server. and possible learning curve for those not familiar with running a CalDAV server.

I hope that cleared some things about what I did. I'll look through at the previous comments and address them individually.

Cheers!

[sogko]

@aviaryan: It looks like sogko's code auto publishes the ics to google calendar. That's good, we can run the calendar generator on Travis but it looks like the google calendar url will change with every update.

The google calendar (GCal) URL does not change. I store the calendar IDs in data.json. So the program will determine if data.json has a calendar ID. If it exists, use it. If not, create a new one (after which it will write to data.json)

@championswimmer: Google Calendar accepts creating a calendar with iCal, so just making an iCal should solve our purpose I am guessing.

That's one of the things that I took away from doing all this (see my notes above). Majority of Calendar applications accepts ical. But you would want webcal anyway since you want users to get the latest updates. But again, take note that Google Calendar does not allow user to set refresh interval for webcal feed. Annoying.

@aviaryan: @championswimmer How do you propose to update the google calendar in case schedule and so ical updates ?

Publish the ical as webcal as I've eventually done. More than one way to do this, but you can read about how I did it above.

Cheers! 🍻

[sogko]

Guys, I've updated my initial notes to include a recommendation on how you guys can proceed.

• Skip the whole generating GCals and .ics files. (My personal experience from the work that I put into: it was a pain in the ass. Don't do it.)
• Run a public CalDAV server that publishes a webcal.
• One CalDAV server implementation that I found: Radicale (Python-based).
• I'm sure there's other open-source implementations or solutions to address this very problem. (Using iCloud to publish a webcal is a laughable hack, I have to admit lol)

Cheers!

[aviaryan]

@sogko Thanks for your help. Your essay :smile_cat: was an entertaining and informational read. Now getting to the point, I don't know much about CalDAV server so it would be tough to get it working. BTW, I think that going till GCals + GCal-ICS will fulfill our needs (the 3 points mentioned by @mariobehling in OP)

1. As the Google Cal link will stay constant, we would have a Google Calendar for OpenTech which users can follow with their accounts and stay updated.
2. The calendar update can be done through Travis. ( BTW, @mariobehling If the need is to update the calendar with every change in the spreadsheet, then we can use a Google Apps Script to do so. https://github.com/fossasia/open-event-scraper/issues/13#issuecomment-193101352 )
3. We will also have an ical link

Please correct me if I am wrong.

I will start tinkering with your code and try to get this issue done. And I won't hesitate to @mention you if I need help. :stuck_out_tongue:

@aditya1702 If you don't mind, please let me do this.

[aditya1702]

@aviaryan Ok :)

[aviaryan]

@mariobehling Here is the OpenTechSummit calendar https://calendar.google.com/calendar/render?cid=t3sacg6lr2s85f51ujggg0usk4@group.calendar.google.com The corresponding ICS would be https://calendar.google.com/calendar/ical/t3sacg6lr2s85f51ujggg0usk4%40group.calendar.google.com/public/basic.ics

I will set the automated calendar updates soon.

[aviaryan]

The Travis build is functional now. https://travis-ci.org/aviaryan/opentechsummit-2016-calendar The repo is in my account right now. https://github.com/aviaryan/opentechsummit-2016-calendar Should I transfer it to OpenTec organisation ?

Since the calendar script generates calendar from the json generated in the scraper repo, so we will need to modify the travis.yml of scraper repo to be able to trigger build on this repo. I have created a script which can trigger the calendar build. It can be called in the scraper's bash script.

wget https://raw.githubusercontent.com/aviaryan/opentechsummit-2016-calendar/master/trigger-build.sh
bash trigger-build.sh

@mariobehling Is this OK ? Should I send a PR ?

[mariobehling]

@sogko and @/all Thanks a lot for the input on this issue. We will implement the ics generation in the orga server in future. The part of how GCal is staying relevant but it looks like it is mostly out of our influence to update GCals in a timely manner. Closing the issue as we are moving on. Thank you!