search

datactive / bigbang

Scientific analysis of collaborative communities
http://datactive.github.io/bigbang/
MIT License
149 stars 52 forks source link

IETF attendance updates for online-only meetings #424

Closed npdoty closed 3 years ago

npdoty commented 3 years ago

IETF 108, 109, 110 (and maybe some to come) have been all virtual and use a different web page format for showing attendance, e.g. https://registration.ietf.org/108/participants/remote/

We should update ietfattendance.py to parse this new format (and to know which meetings to use which URL path and scraper).

lumisota commented 3 years ago

The Datatracker has an endpoint that gives meeting registration data for IETF meetings (from around IETF 72 onwards): https://datatracker.ietf.org/api/v1/stats/meetingregistration/

This is exposed by the ietfdata library: https://github.com/glasgow-ipl/ietfdata/blob/7d4f4207fe35afa6233aeb787990639b44f6d1ee/ietfdata/datatracker.py#L4036

npdoty commented 3 years ago

thank you @lumisota ! I did specifically look at the datatracker's page to try to find this, but didn't see it. Is the recommended way to just look at their Django Python models rather than at the datatracker web page or documentation?

lumisota commented 3 years ago

Yes, would do that (links to the models at https://datatracker.ietf.org/api/). Otherwise, the ietfdata library provides fairly complete coverage of the API (with a few exceptions).

sbenthall commented 3 years ago

oh, that's much better than web scraping the attendance records!

On Mon, Mar 1, 2021 at 2:25 PM Stephen McQuistin notifications@github.com wrote:

The Datatracker has an endpoint that gives meeting registration data for IETF meetings (from around IETF 72 onwards): https://datatracker.ietf.org/api/v1/stats/meetingregistration/

This is exposed by the ietfdata library:

https://github.com/glasgow-ipl/ietfdata/blob/7d4f4207fe35afa6233aeb787990639b44f6d1ee/ietfdata/datatracker.py#L4036

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/datactive/bigbang/issues/424#issuecomment-788206694, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAQZEGRJTC36PXWZZYO6XTTBPS2DANCNFSM4YNDK4QA .

npdoty commented 3 years ago

I'm working on an updated notebook that would do some of the graphing/data analysis but will probably use ietfdata only.

It might be that we can remove a whole .py file, which I would count as a success. :)

sbenthall commented 3 years ago

I agree. Better use of the ietfdata package is a real win.

On Tue, Mar 2, 2021 at 11:38 AM Nick Doty notifications@github.com wrote:

I'm working on an updated notebook that would do some of the graphing/data analysis but will probably use ietfdata only.

It might be that we can remove a whole .py file, which I would count as a success. :)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/datactive/bigbang/issues/424#issuecomment-789042010, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAQZEEDGBHO5OCLOP2S24LTBUIADANCNFSM4YNDK4QA .