automatically create (recurring) events from pretix

[pr130]

our 'source of truth' for events is our pretix shop: https://pretix.eu/correlaid/ . currently we don't always remember to create the corresponding events on our website so visitors on our website don't get the full view of all our activities (or at least those represented on pretix).

we could use its API to automatically create the corresponding events for the website for the next month or so. this is especially interesting for recurring events such as the open onboarding call. the service should not commit to main directly but instead open a branch and make a PR.

technical implementation ideas:

• github action that runs monthly. problem: what if events are deleted on pretix? manually remove on website?
• set up a little service (e.g. Azure Function, aws lambda, something else lightweight that runs on our VM) that responds to a webhook : https://docs.pretix.eu/en/latest/api/webhooks.html . could be part of the correlbot or a separate service. advantage: webhooks can be setup to send events both for creation and deletion of events. disadvantage: more work / more complex. lives outside this repository.

[jandix]

@friep sounds like a really nice feature!

Maybe we could use the GitHub Actions schedule function to trigger the synchronization once per day?

Maybe we could also mark the cancelled events as cancelled so that events don't mystically disappear?

[Peter554]

@friep sounds a nice idea.

we could use its API to automatically create the corresponding events for the website for the next month or so.

Let's take https://raw.githubusercontent.com/CorrelAid/hugo-website/main/content/en/events/2021-08/open-onboarding-call.md as an example event. I am wondering how we can get all the required data from the pretix API response https://docs.pretix.eu/en/latest/api/resources/events.html#resource-description e.g. fields like correlaidx, eventRegistration, languages, event description... Looking at the https://docs.pretix.eu/en/latest/api/resources/events.html#resource-description I see a field meta_data ("Values set for organizer-specific meta data parameters."), maybe this information is somehow lurking there? Else I am not sure where we could get that data. Maybe you can send an example response from the API that contains this event (I would look myself but I think I would need an API key)? Then it might be clearer how to reconcile API response with the structure you store events in the hugo site.

[pr130]

@Peter554 thanks for your response!

i definitely did not think this through 100% before posting the issue so you definetely have a point re. potential "missing" data. i'd definitely be willing to drop some customization on the website end (e.g. correlaidx controls the background of the header image and the color scheme (e.g. here) - i'm not married to varying that). more tricky would probably be the speaker part. on the other hand, the process doesn't need to be fully automated. if there's some bits to do manually after a PR has been created, that'd be totally fine. from a process perspective, the actual opening of the project in vscode, creating the new files and filling in the yml header, committing and pushing etc are the parts that make it so annoying and "time consuming" (it doesn't take long at all.. it's just another thing to do when organizing an event).

i'll create a test event later and post the API response data here.

@jandix re cancelled events: would you keep a list of events then in the repo and do comparisons? or how would you go about it?

[pr130]

sorry for the late response! below is an example event.

major drawback: the actual description is not returned, i.e. that'd need to be done with web scraping (looks doable). also to extract a good slug suggestion from the title would be a challenge.

{
  "name": {
    "en": "CorrelAidX Stuttgart talk: „Modeling the Covid-19 pandemic – How we estimate undetected cases for the Dunkelzifferradar dashboard\""
  },
  "slug": "3gw3v",
  "live": true,
  "testmode": false,
  "currency": "EUR",
  "date_from": "2021-03-29T19:00:00+02:00",
  "date_to": "2021-03-29T20:30:00+02:00",
  "date_admission": {},
  "is_public": true,
  "presale_start": "2021-03-23T00:00:00+01:00",
  "presale_end": "2021-03-30T00:00:00+02:00",
  "location": {
    "en": "Online / Zoom"
  },
  "geo_lat": 43.5905,
  "geo_lon": 3.8595,
  "has_subevents": false,
  "meta_data": {},
  "seating_plan": {},
  "plugins": [
    "pretix.plugins.banktransfer",
    "pretix.plugins.paypal",
    "pretix.plugins.reports",
    "pretix.plugins.sendmail",
    "pretix.plugins.statistics",
    "pretix.plugins.stripe",
    "pretix.plugins.ticketoutputpdf",
    "pretix_digital",
    "pretix_facebook",
    "pretix_mollie",
    "pretix_passbook",
    "pretix_sepadebit"
  ],
  "seat_category_mapping": {},
  "timezone": "Europe/Berlin",
  "item_meta_properties": {},
  "valid_keys": {
    "pretix_sig1": []
  },
  "sales_channels": [
    "web"
  ]
}

[Peter554]

@friep thanks for posting that response

major drawback: the actual description is not returned, i.e. that'd need to be done with web scraping (looks doable).

hmm that is odd, and annoying. I didn't use pretix before so am stabbing in the dark a bit, but would you have any idea what the pretix "Item" resource is https://docs.pretix.eu/en/latest/api/resources/items.html#get--api-v1-organizers-(organizer)-events-(event)-items- ? It seems to me that "Events" have "Items", and "Items" have descriptions. So maybe we should actually be looking there for the description.

also to extract a good slug suggestion from the title would be a challenge.

I'm not quite sure what that means. In the response you posted the Event contains the slug, so I don't really understand what you mean by "extract a good slug"?

[jandix]

@jandix re cancelled events: would you keep a list of events then in the repo and do comparisons? or how would you go about it?

If we use Python we could try to parse the meta fields in the files and add specific meta fields that are linked to the data from petrix. Maybe it is even possible to introduce an additional meta field that represents a unique identifier that can be linked to the petrix API (e.g.: slug). We could also introduce an additional meta field (e.g. is_cancelled) that is automatically set to True when the event does not exist anymore. I think that my help people that the event did not just disappear but it was actually cancelled.

I would like to avoid adding an additional "database" (CSV, JSON, SQLite, etc.) if we can use the filesystem.

[pr130]

@Peter554 i think the items are just analogous for ticket types or similar things. it's a very complex tool with lots of options and options. we only use it to approx 30% of capacity. re the slug: that's true, we could just use the slug of pretix for the event on the website. however, the slug on hugo websites are just the file names and i'd not be a huge fan of having files names 23erw.md or similar.

@jandix yes re your points.

i'll be in contact with you over slack, i think we should have a call (if possible from your side) so that we can come up with a more detailed plan maybe and answer any other open questions..

[Peter554]

FYI I created a draft PR. Already opened a few threads I am aware will need discussing. Feel free to reply in the PR, or else we can discuss in a call some time.

https://github.com/CorrelAid/hugo-website/pull/200