4. Automate fetching the new results

[lachiemurray]

Everything up until this point still relies on someone remembering to upload the results file each week but we could automate this too. I think there are two obvious possibilities:

a) Ask Jonathan Moscrop to fix the OneDrive upload and pull the new file down from there (I've sent him an email) b) Pull the results directly from PowerOf10 - using the library you mentioned or another method

Given the unreliability we've seen with Jonathan's spreadsheet the preferred option is now b) - pulling the results directly from PowerOf10.

Unfortunately, as far as I can tell, there is no page on PowerOf10 that shows all the recent results for members of a particular club so we will either have to:

1) Scrape all recent results and filter for Phoenix athletes

Recent results list:

Detailed results:

2) Scrape the profile page of every known Phoenix athlete

Profile view:

(1) has the advantage that we don't need to know the names of all the Phoenix athletes in advance but is potentially more complex in that we would have to:

• Scrape both the recent results list and the detailed results
• Detect and remember whether results are fully or partially uploaded
• Potentially navigate multiple pages of results for larger events

(2) requires us to know the names of all Phoenix athletes and scrape each athletes profile page at least once per week (about 600 in total) which could result in more requests overall but simpler scraping logic

Assuming we were to take the second approach we would need to maintain an up-to-date list of all Phoenix members. We can export a list of members from Clubtrac, which would give us a starting to import into the website database:

And either manually refresh this list on a regular basis or set up a Zapier integration to be notified when a new member joins or existing member leaves (waiting for details on exactly what's supported here from Clubtrac).

Proposed Data Model

Event

• Name (string)
• Location (string)
• Date (string)

Result

• AthleteID (AthleteFk)
• EventID (EventFk)
• Distance (string)
• Time (time)
• Overall Position (number) (optional)
• Age Position (number) (optional)
• Gender Position (number) (optional)

Athlete

• Name (string)
• DOB (string)
• Sex (string)
• EA number (string)
• PowerOf10Id (string)
• ClubtracId ? (string)

Open questions

• What athlete data can we get from Clubtrac?
• How to distinguish between athletes with the same name when scraping (do we have any?)
• What events does the Clubtrac Zapier integration notify about?

[lachiemurray]

Have asked Jonathan if he would be willing to share more details about what exactly it is that he does