Canceled Event Shows on the Calendar and Events Pages

[allella]

@zach2825 I'm not sure when this event was canceled, but I suspect this could be a case where the local cache/DB for HG is holding on to an event it previously saw in the API, which was later canceled.

https://hackgreenville.com/events https://hackgreenville.com/calendar

The API doesn't have it listed because I think we're dropping canceled events. Though, are certainly use cases where showing canceled events in the API would allow consumers to show the event as canceled on their end, or choose to discard it.

Pinging @ramonaspence @pamelawoodbrowne @jeremywight for general thoughts on if the events API should retain canceled and then HG should show them in red, or some such. Or, do we want to keep dumping canceled events and expect the consuming apps to refresh and drop them off?

Here's the Synergy Mill example from this evening. I though the event was happening because I saw it earlier today on the calendar, but they canceled it at some point.

https://www.meetup.com/synergymill/events/gjnkqrybchbqb/

[ramonaspence]

I do think it would be helpful to users. Perhaps the cancelled events could be marked as such and then taken off of HG's api after an amount of time.

I don't see where cancelled events are getting dropped though, where is that happening?

[allella]

@ramonaspence Meetup may be ignoring cancelled events by default, so I'm not sure they are in /gtc/api/

The Meetup docs for events says,

A comma-delimited list of event statuses. Valid values are: "cancelled", "draft", "past", "proposed", "suggested", or "upcoming". Events with "draft" status may be accessed by the group's organizers; guest event hosts may access only the drafts of events they host. The "draft" status may not be requested with other status values. The status value defaults to "upcoming" unless a scroll parameter is provided. If more than one status is requested, events with "proposed" status are returned last.

I think it would be good for /gtc/api to return cancelled events too. We'll just have to make sure our consuming applications know this and handle cancelled events according to their preference.

Can you test to see about pulling upcoming,cancelled for Meetup and see if we can get the cancelled Synergy Mill reference in this issue to return with a status of cancelled?

Related, we have to normalize the statuses between event services and "canceled" was used before. If we prefer "cancelled" (two L's), then now would be the time to make that decision.

[ramonaspence]

tl;dr: I'm hacking away at it, but getting some unexpected results with the API. So, still hacking at it! (although I have some meetings in the afternoon, so I won't be able to work on this after lunch today). @allella

What I'm doing:

Alright, I'm trying to get the Tinker Tuesday from 05-12. I just did a second simple .py file to put the request in.

My url = https://api.meetup.com/{}/events?&sign=true&photo-host=public&no_earlier_than=2020-04-01T00:00:00.000&page=20&status=cancelled,upcoming'.format('synergymill')

I'm changing around the no_earlier_than parameter and I am unable to get events past 03-31. However I am getting the Tinker Tuesdays on 03-31, 03-24, 03-17 which are all cancelled.

I'm not sure why it won't bring back anything after 03-31 yet.

Results

I've already said it won't bring back events after 03-31, but to add to that I'm not getting any events with a status of 'upcoming'. Those two issues are likely the same however, since no event in the past would have a status of 'upcoming'. see note below

With the no_earlier_than parameter set to 04-01 OR the status parameter set to upcoming, I get an empty list back from the API.

Note

Synergy Mill does not have any upcoming events on Meetup.com https://www.meetup.com/synergymill/events/

However, DefCon864 events are showing at https://events.openupstate.org/api/gtc

If you look at their meetup calendar https://www.meetup.com/synergymill/events/calendar/2020-04/ There are events in April and one on May 5 that I expect to come in from the API.

[allella]

You're volunteering, so we'll take what we get, when we get it. Thanks for continuing to hack on it.

There's also a "past" status, maybe they change it to "past" after so many months? You might try requesting all of the status types and see if it's something like that. Also, there are other query parameters like &sign=true&photo-host=public so it could be that.

Or, try a different Meetup and see if the issue is isolated to Synergy Mill or if it's universal. Code For Greenville could be another group to test.

[ramonaspence]

@allella Okay, those are good things for me to check and tinker with. I was also looking at the parameters Meetup shows here: https://www.meetup.com/meetup_api/docs/:urlname/events/#list

So I definitely have plenty of things to try :)

[ramonaspence]

Okay, so I'm kind of just jumping back into this so pardon me in advance if I'm off track of our logic from last week.

I went through Code For Greenville's api results, but I don't see anything cancelled. All of it is past and upcoming. However, I did notice something worth noting. This event: https://www.meetup.com/synergymill/events/gjnkqrybcfbpc/ Which was cancelled, is coming back from the API and has a status of 'cancelled'.

On the other hand, the event you had a few days ago is no longer on meetup.com. https://www.meetup.com/synergymill/events/gjnkqrybchbqb/

The first link specifies the event is cancelled, while the second link 404s.

I would imagine that Synergy Mill may have removed the event after cancelling it.

Deleting cancelled events after an amount of time doesn't seem like a feature Meetup would have, but that doesn't make it impossible.

conclusion for now

We can get cancelled meetups with no problem, but the caveat is when the organization has deleted the cancelled event.

if it helps:

here is the request URL I'm using for codeforgreenville: r = requests.get('https://api.meetup.com/{}/events?&sign=true&no_earlier_than=2020-04-30T00:00:00.000&no_later_than=2021-03-31T00:00:00.000&status=cancelled,past,upcoming&photo-host=public&page=50'.format('code-for-greenville'))

and then the same url but with .format('synergymill')

[allella]

@zach2825

@ramonaspence updated the events API to allow for past events and cancelled.

The API is currently allowing for up to 30 days of past events and you can pull a date range using https://events.openupstate.org/api/gtc?start_date=2020-05-01&end_date=2020-06-01

We're figuring the HackGreenville calendar would be good to show the cancelled events with some sort of visual indicator, like an icon and/or color.

[ramonaspence]

@allella I'm noticing two of Synergy Mill's Tinker Tuesday events on the HG calendar are no longer on meetup.com (as opposed to being cancelled). The links provided by HG 404s on meetup's website.

I wonder if there is a simple test that could be implemented to validate the events' link and confirm that it doesn't get a 404 status.

OR

Are these showing because the script that aggregates (is that the right word?) data has not run recently enough to update the HG calendar with that info?

[allella]

@ramonaspence the HG calendar is caching results, so we'll probably want Zach to chime in before we expect the API changes to resolve on the HG calendar.

[zach2825]

Hey guys I'm sorry I dropped off for a bit. That should not happen again. The issue here appears to be caused by the code here.

https://github.com/codeforgreenville/hackgreenville-com/blob/develop/app/Console/Commands/PullEventsCommand.php

I'll stage new code that will remove events that drop off the API sorry I didn't account for that when I put that in place.

[zach2825]

Plan of attack:

I'm going to import the information that I can and when a future event does not exist in the API response any more I'll remove it from the database.

[zach2825]

I'm making some good progress on this. I'm trying to have something to resolve this issue by end of day.

If anybody would like to talk this through let me know we can zoom or something.

[ramonaspence]

@zach2825 I don't know PHP. But if you want any help pseudo-coding for this, I'm more than happy to get with you about any of it. Just ping me on here or on slack (@ramonaspence)!

[allella]

Zach,

Thanks for the update and work.

I trust you two can come up with sometime solid, but holler if there are specific questions for me or CFG.

[zach2825]

@ramonaspence came up with a good idea to add a new command in laravel that will update the event in the database and set it canceled. We think the API will return the event but with a canceled flag. @allella can you confirm that what is returned for canceled events?

A benefit of creating the new laravel command I can run the cancel event API calls more often to update canceled events more often

This work will leave canceled events in the calendar but we can make the event placeholder red or something to show that it was canceled.

[allella]

@zach2825

Yes, our events API will normalize Meetup and Eventbrite status to "cancelled" with two L's. They return different spellings and words for statuses and it normalizes them.

Here's an example of the Women Who Code event with a cancelled status

  {
    "created_at": "2020-01-10T22:11:14Z", 
    "data_as_of": "2020-05-20T02:40:10Z", 
    "description": "Women Who Code Greenville is very excited to announce that we are now accepting talk proposals for our upcoming conference, Sudo Conf, coming up on May 30th, 2020. Sudo Conf is our chance to get together in-person to explore and celebrate the craft of technology in a positive, inclusive environment. WWCode Greenville invites people from all across our community to come up on stage to share their insights and experience. Whether you\u2019ve worked in technology for decades or you\u2019ve just started this year, we want to hear from you! Learn more at <a href=\"https://sudoconf.com/\" class=\"linkified\">https://sudoconf.com/</a> Ticket sales available soon!! ", 
    "event_name": "Sudo Conf", 
    "group_name": "Women Who Code Greenville", 
    "nid": "40", 
    "rsvp_count": 10, 
    "status": "cancelled", 
    "tags": "1", 
    "time": "2020-05-30T13:00:00Z", 
    "url": "https://www.meetup.com/Women-Who-Code-Greenville/events/267817088/", 
    "uuid": "788992ce-51f8-44e2-b300-d495303e0025", 
    "venue": {
      "address": "411 University Ridge", 
      "city": "Greenville", 
      "country": "us", 
      "lat": 34.84038162231445, 
      "lon": -82.39857482910156, 
      "name": "NEXT Innovation Center", 
      "state": "SC", 
      "zip": "29601"
    }
  }

[zach2825]

I'm utilizing that status = cancelled now I'll have a PR soon.

I remember that now. We talked about canceled vs cancelled before. I'll make sure that is persistent in the project.

even Grammarly wants to correct it lol

[allella]

@zach2825 no pressure, but I was reminded by the OpenWorks dashboard about this issue with the cancelled events are still in the API for 30 days.

I've updated Openworks' dashboard to visually show the events at cancelled as an example of how consumers of the API might deal with it since we didn't want to delete cancelled events from the results. https://joinopenworks.com/dashboard/meetups.php

@JSn1nj4 is interested in helping out, so this may be something he's interested in if you haven't started with it.

[zach2825]

Thank you for the update. I've been slammed at work. I'm going to have a few in the morning. I should be able to get it resolved finally.

On Tue, Sep 8, 2020, 7:03 PM Jim Ciallella notifications@github.com wrote:

@zach2825 https://github.com/zach2825 no pressure, but I was reminded by the OpenWorks dashboard about this issue with the cancelled events are still in the API for 30 days.

I've updated Openworks' dashboard to visually show the events at cancelled as an example of how consumers of the API might deal with it since we didn't want to delete cancelled events from the results. https://joinopenworks.com/dashboard/meetups.php

@JSn1nj4 https://github.com/JSn1nj4 is interested in helping out, so this may be something he's interested in if you haven't started with it.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/codeforgreenville/hackgreenville-com/issues/63#issuecomment-689181033, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA64TKMEPTFFMXNXKXYPQXTSE2Z5JANCNFSM4M7G2QYQ .

[allella]

@zach2825 this is pulled to the stage site and I ran the three commands

The calendar isn't loading https://stage.hackgreenville.com/calendar and the there's a 500 error https://stage.hackgreenville.com/api/calendar?start=2020-08-30T00%3A00%3A00-04%3A00&end=2020-10-11T00%3A00%3A00-04%3A00

but I haven't seen anything in the logs yet.

Did I forgot to run a command?

I run composer install as well.

[allella]

I see the Larvel log caught the error

[previous exception] [object] (PDOException(code: 42S22): SQLSTATE[42S22]: Column not found: 1054 Unknown column 'cancelled_at' in 'where clause' at /home/stagehg/public_html/hack/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnection.php:61)

I ran the migrate command and it says there's nothing to migrate.