Write create/update calendar script

charlesreid1 commented 6 years ago

Now that we have working API tests, we need to convert them to a full-fledged calendar creation script.

This will work as follows:

One file, one program, using argparse
User can either create or update a calendar
User passes the name of the calendar on the command line
Structure will largely mirror that of gcal_update_calendar.py

charlesreid1 commented 6 years ago

[x] argparse arguments
- [x] create
- [x] update
- [x] name of calendar
- [x] ical list file
- [x] parameter validation
[x] create calendar - similar to gcal_update_calendar script
[x] update calendar - similar to gcal_update_calendar script
[x] main method logic - decide whether to call create or update
[x] revisit and review utils_gcal to remove cruft
- [x] old methods were only dealing with one ical file, need to deal with a list
- [x] solution: deal with file containing ical URLs list in the dcppc_calendars script, pass ical URLs to functions defined in utils_gcal (since there is already a method defined that expects ical URLs)
[x] revisit and review utils_ical to remove cruft

charlesreid1 commented 6 years ago

Ready for testing.

charlesreid1 commented 6 years ago

Tests have uncovered some bugs, but we're on the home stretch now.

(More corner cases... poorly formatted dates. There are no times - these may be all-day events?)

charlesreid1 commented 6 years ago

----------------------------------------
Processing event calendar11449279691groupsio
XXX Failed to convert event to JSON
----------------------------------------
Processing event calendar11449278958groupsio
XXX Failed to convert event to JSON
----------------------------------------

charlesreid1 commented 6 years ago

create is ok. update still reports failures:

Updating Google Calendar with events from components_map...
Calendar id: nihdatacommons.com_7emes1bsk61rh71a3enjjlkpec@group.calendar.google.com
----------------------------------------
Summary:
  ADD:    16 Google Calendar events to add
  REMOVE: 0 Google Calendar events to remove
  SYNC:   116 Google Calendar events to sync
Adding 16 events...
----------------------------------------
Adding event calendar8949292154groupsio
Title: KC7 Working Group
ERROR: Could not create event with event id: calendar8949292154groupsio
There may be a problem with the calendar/event id.
Calendar id: nihdatacommons.com_7emes1bsk61rh71a3enjjlkpec@group.calendar.google.com
Event id: calendar8949292154groupsio
Title: KC7 Working Group

Continuing...

----------------------------------------
Adding event calendar9283316654groupsio
Title: NIH Data Commons KC6 Meeting
ERROR: Could not create event with event id: calendar9283316654groupsio
There may be a problem with the calendar/event id.
Calendar id: nihdatacommons.com_7emes1bsk61rh71a3enjjlkpec@group.calendar.google.com
Event id: calendar9283316654groupsio
Title: NIH Data Commons KC6 Meeting

Continuing...

----------------------------------------
Adding event calendar8949292155groupsio
Title: KC7 Working Group
ERROR: Could not create event with event id: calendar8949292155groupsio
There may be a problem with the calendar/event id.
Calendar id: nihdatacommons.com_7emes1bsk61rh71a3enjjlkpec@group.calendar.google.com
Event id: calendar8949292155groupsio
Title: KC7 Working Group

Continuing...

----------------------------------------
Adding event calendar8949292159groupsio
Title: KC7 Working Group
ERROR: Could not create event with event id: calendar8949292159groupsio
There may be a problem with the calendar/event id.
Calendar id: nihdatacommons.com_7emes1bsk61rh71a3enjjlkpec@group.calendar.google.com
Event id: calendar8949292159groupsio
Title: KC7 Working Group

Continuing...

XXX Failed to convert ics to google calendar event
Traceback (most recent call last):
  File "/Users/charles/davis/calendars/scripts/util_gcal.py", line 222, in update_gcal_from_components_map
    ical_event = ics2gcal_event(ical_events[eid])
  File "/Users/charles/davis/calendars/scripts/util_gcal.py", line 633, in ics2gcal_event
    startdt = utc.localize(datetime.datetime.strptime(vevent_decode(vevent['DTSTART']), "%Y%m%dT%H%M%SZ"))
  File "/Users/charles/.pyenv/versions/miniconda3-4.3.30/lib/python3.6/_strptime.py", line 565, in _strptime_datetime
    tt, fraction = _strptime(data_string, format)
  File "/Users/charles/.pyenv/versions/miniconda3-4.3.30/lib/python3.6/_strptime.py", line 362, in _strptime
    (data_string, format))
ValueError: time data '20180530' does not match format '%Y%m%dT%H%M%SZ'
----------------------------------------
Adding event calendar11449278958groupsio
Title: KC7 Working Group
ERROR: Could not create event with event id: calendar8949292159groupsio
There may be a problem with the calendar/event id.
Calendar id: nihdatacommons.com_7emes1bsk61rh71a3enjjlkpec@group.calendar.google.com
Event id: calendar8949292159groupsio
Title: KC7 Working Group

Continuing...

----------------------------------------
Adding event calendar9513311710groupsio
Title: KC2: Weekly Meeting
ERROR: Could not create event with event id: calendar9513311710groupsio
There may be a problem with the calendar/event id.
Calendar id: nihdatacommons.com_7emes1bsk61rh71a3enjjlkpec@group.calendar.google.com
Event id: calendar9513311710groupsio
Title: KC2: Weekly Meeting

Continuing...

----------------------------------------
Adding event calendar8949292160groupsio
Title: KC7 Working Group
ERROR: Could not create event with event id: calendar8949292160groupsio
There may be a problem with the calendar/event id.
Calendar id: nihdatacommons.com_7emes1bsk61rh71a3enjjlkpec@group.calendar.google.com
Event id: calendar8949292160groupsio
Title: KC7 Working Group

Continuing...

----------------------------------------
Adding event calendar9513311709groupsio
Title: KC2: Weekly Meeting
ERROR: Could not create event with event id: calendar9513311709groupsio
There may be a problem with the calendar/event id.
Calendar id: nihdatacommons.com_7emes1bsk61rh71a3enjjlkpec@group.calendar.google.com
Event id: calendar9513311709groupsio
Title: KC2: Weekly Meeting

Continuing...

----------------------------------------
Adding event calendar9513311708groupsio
Title: KC2: Weekly Meeting
ERROR: Could not create event with event id: calendar9513311708groupsio
There may be a problem with the calendar/event id.
Calendar id: nihdatacommons.com_7emes1bsk61rh71a3enjjlkpec@group.calendar.google.com
Event id: calendar9513311708groupsio
Title: KC2: Weekly Meeting

Continuing...

----------------------------------------
Adding event calendar9513311711groupsio
Title: KC2: Weekly Meeting
ERROR: Could not create event with event id: calendar9513311711groupsio
There may be a problem with the calendar/event id.
Calendar id: nihdatacommons.com_7emes1bsk61rh71a3enjjlkpec@group.calendar.google.com
Event id: calendar9513311711groupsio
Title: KC2: Weekly Meeting

Continuing...

----------------------------------------
Adding event calendar8949292158groupsio
Title: KC7 Working Group
ERROR: Could not create event with event id: calendar8949292158groupsio
There may be a problem with the calendar/event id.
Calendar id: nihdatacommons.com_7emes1bsk61rh71a3enjjlkpec@group.calendar.google.com
Event id: calendar8949292158groupsio
Title: KC7 Working Group

Continuing...

----------------------------------------
Adding event calendar8949292157groupsio
Title: KC7 Working Group
ERROR: Could not create event with event id: calendar8949292157groupsio
There may be a problem with the calendar/event id.
Calendar id: nihdatacommons.com_7emes1bsk61rh71a3enjjlkpec@group.calendar.google.com
Event id: calendar8949292157groupsio
Title: KC7 Working Group

Continuing...

----------------------------------------
Adding event calendar9513311712groupsio
Title: KC2: Weekly Meeting
ERROR: Could not create event with event id: calendar9513311712groupsio
There may be a problem with the calendar/event id.
Calendar id: nihdatacommons.com_7emes1bsk61rh71a3enjjlkpec@group.calendar.google.com
Event id: calendar9513311712groupsio
Title: KC2: Weekly Meeting

Continuing...

----------------------------------------
Adding event calendar8949292156groupsio
Title: KC7 Working Group
ERROR: Could not create event with event id: calendar8949292156groupsio
There may be a problem with the calendar/event id.
Calendar id: nihdatacommons.com_7emes1bsk61rh71a3enjjlkpec@group.calendar.google.com
Event id: calendar8949292156groupsio
Title: KC7 Working Group

Continuing...

XXX Failed to convert ics to google calendar event
Traceback (most recent call last):
  File "/Users/charles/davis/calendars/scripts/util_gcal.py", line 222, in update_gcal_from_components_map
    ical_event = ics2gcal_event(ical_events[eid])
  File "/Users/charles/davis/calendars/scripts/util_gcal.py", line 633, in ics2gcal_event
    startdt = utc.localize(datetime.datetime.strptime(vevent_decode(vevent['DTSTART']), "%Y%m%dT%H%M%SZ"))
  File "/Users/charles/.pyenv/versions/miniconda3-4.3.30/lib/python3.6/_strptime.py", line 565, in _strptime_datetime
    tt, fraction = _strptime(data_string, format)
  File "/Users/charles/.pyenv/versions/miniconda3-4.3.30/lib/python3.6/_strptime.py", line 362, in _strptime
    (data_string, format))
ValueError: time data '20180629' does not match format '%Y%m%dT%H%M%SZ'
----------------------------------------
Adding event calendar11449279691groupsio
Title: KC7 Working Group
ERROR: Could not create event with event id: calendar8949292156groupsio
There may be a problem with the calendar/event id.
Calendar id: nihdatacommons.com_7emes1bsk61rh71a3enjjlkpec@group.calendar.google.com
Event id: calendar8949292156groupsio
Title: KC7 Working Group

Continuing...

----------------------------------------
Adding event calendar9513311707groupsio
Title: KC2: Weekly Meeting
ERROR: Could not create event with event id: calendar9513311707groupsio
There may be a problem with the calendar/event id.
Calendar id: nihdatacommons.com_7emes1bsk61rh71a3enjjlkpec@group.calendar.google.com
Event id: calendar9513311707groupsio
Title: KC2: Weekly Meeting

Continuing...

Done adding 16 events.

charlesreid1 commented 6 years ago

Added some exception-handling. We need to track down these 14 events that are problems and the 2 events with datetime format issues and repair the script...

charlesreid1 commented 6 years ago

Calling this done, since the script Just Works and the calendar is now being populated successfully.

dcppc / calendars

Write create/update calendar script #4