SReview integration

[johanvdw]

SReview, het video review systeem dat ik geschreven heb, verwacht een video stream die het dan, op basis van het talk schedule, opsplitst in verschillende individuele video's. Vervolgens worden de video's beschikbaar gemaakt en gaat er een notificatie naar de sprekers en naar de devroom organisatoren met een vraag om te reviewen.

Om die reden heb ik de volgende dingen nodig:

• het schedule van het event. Belangrijk daarin is dat ik moet weten welke talk we aan het bekijken zijn, waarvoor ik een stabiele identifier nodig heb. Bij Penta gebruikte ik daarvoor het Penta-id, wat ik uit de schedule XML file kon parsen.
• minder vanzelfsprekend misschien is dat we ook een stabiele id nodig hebben voor elke spreker. Ten eerste omdat sprekers soms details over hun identiteit aanpassen, ten tweede omdat we dan makkelijk de sprekers die meerdere talks doen kunnen matchen, als laatste ook om ze dan te kunnen matchen met de into over hun e-mailadres.
• voor de notificatie heb ik contactinformatie nodig: e-mailadressen voor alle sprekers, en e-mailadressen voor de devroom organisatoren.

Ik neem aan dat we wel gewoon terug een Penta XML-bestand zullen publiceren, en dat mijn parser van dat bestand met hooguit een paar tweaks wel zal werken, ten minste als jij er voor kunt zorgen dat de stabiele identifiers die ik nodig heb ergens in dat bestand aanwezig zijn.

Wat ik nog niet weet is hoe we dat laatste zullen doen. We maakten die info niet publiekelijk beschikbaar met Penta, omdat we het voor spammers niet té makkelijk willen maken, maar dat betekent wel dat de er op een andere manier aan moeten kunnen komen. Bij voorkeur heb ik hiervoor niet gewoon een dump, maar wel iets wat me toelaat om lookups te doen, anders moet ik dit heel erg last minute gaan regelen om er zeker van te zijn dat ik iedereen heb, wat niet ideaal is. Bij Penta opende ik hiervoor een rechtstreekse verbinding met de database waarin ik dan een query deed naar een bepaalde view; als dat het makkelijkste is voor jou kunnen we dat blijven doen, maar als je één of andere ReST-API hebt dan is dat misschien wel iets makkelijker.

Idealiter zou ik daar de komende maand al naar kunnen kijken, zodat ik eventuele problemen met de schedule parser etc kan aanpakken.

[johanvdw]

Here is an example xml. I think you should be able to use this. schedule.xml.zip

For persons, the person 'id' seems suitable. For events, i should check if we use guid vs id

[johanvdw]

For querying the email addresses, you can use this api endpoint: https://pretalx.fosdem.org/api/events/xxx/speakers/?

where xxx is the event see eg: https://pretalx-test.fosdem.org/api/events/democon/speakers/? you can browse here after logging in or use an api key that can be found at: https://pretalx.fosdem.org/orga/me (we should probably create an sreview account).

It looks like that endpoint contains all email adresses, including from submitted but not accepted talks - you should merge with the talk 'code'

I do however think that this is only part of the necessary changes. We should also look at where and how submitters and devroom managers can see a list of videos that should be reviewed.

[yoe]

Here is an example xml. I think you should be able to use this. schedule.xml.zip

For persons, the person 'id' seems suitable. For events, i should check if we use guid vs id

This seems to parse correctly with the parser that we used previously. I'm currently using the id attribute on <event> elements, not the guid one, because that was the penta id. Are those not stable? If not, I'll switch over, but if they are, then a short number is easier to deal with than a long guid.

The person id does indeed seem suitable, but it does not show up in the API result:

For querying the email addresses, you can use this api endpoint: https://pretalx.fosdem.org/api/events/xxx/speakers/?

where xxx is the event see eg: https://pretalx-test.fosdem.org/api/events/democon/speakers/? you can browse here after logging in or use an api key that can be found at: https://pretalx.fosdem.org/orga/me (we should probably create an sreview account).

This API call returns a paged array of all the speakers:

{
    "count": 31,
    "next": "https://pretalx-test.fosdem.org/api/events/democon/speakers/?limit=25&offset=25",
    "previous": null,
    "results": [
        {
            "code": "BUBTQ9",
            "name": "Robert Richardson",
            "biography": null,
            "submissions": [
                "2"
            ],
            "avatar": ""
        },
        # (...)
    ]
}

However, that does not contain the email address (presumably because I don't have the rights to this yet -- please flip the necessary bits for the "wouter@f.o" user I just created, both in the test and in the prod instance of pretalx) nor the speaker id. The "code" seems like it might be relevant, but it's not in schedule.xml. There are URLs in the <url> attribute in schedule.xml (that use "localhost:8080" in them, yuck) which seem to have similar codes in the final element of the path, but none of those codes match up with any of the speakers' codes.

Again, perhaps I just need an account and then everything will be resolved? If not, I'm not sure how this will work.

It looks like that endpoint contains all email adresses, including from submitted but not accepted talks - you should merge with the talk 'code'

The way I use it, is I first parse the schedule and put the speakers in the database, and then in a separate job, look for speakers for whom I do not (yet?) have an email address and do a lookup of them. This way we ignore speakers

I do however think that this is only part of the necessary changes. We should also look at where and how submitters and devroom managers can see a list of videos that should be reviewed.

There is no such list :) instead, devroom managers and submitters get an email for each of the talks that they need to review when it is ready for review, and then that email contains a link to the review URL.

But perhaps I'm misunderstanding what you are talking about? What exactly do you mean?

[johanvdw]

There is no such list :) instead, devroom managers and submitters get an email for each of the talks that they need to review when it is ready for review, and then that email contains a link to the review URL.

There used to be a link in pentabarf iirc: https://fosdem.org/2024/manuals/program/speaker/#uploading-videos

[yoe]

There used to be a link in pentabarf iirc: https://fosdem.org/2024/manuals/program/speaker/#uploading-videos

That part is for uploading videos for the online-only component of an online-only or hybrid event. Since we're not doing online-only stuff this year, that is not relevant.

If and when that ever changes, we can worry about it again, but in the mean time you can ignore that.

[johanvdw]

but at least, I think you will need an api endpoint for uploading the link to the final video right?

[yoe]

On Wed, Nov 08, 2023 at 01:32:15AM -0800, Johan Van de Wauw wrote:

but at least, I think you will need an api endpoint for uploading the link to the final video right?

Yes, that would be useful, indeed. The endpoint would also need to either filter out duplicate links, or allow me to check which links exist so I can do the filtering on my side.

-- @.***{be,co.za} wouter@{grep.be,fosdem.org,debian.org}

I will have a Tin-Actinium-Potassium mixture, thanks.

[johanvdw]

Here is an example xml. I think you should be able to use this. schedule.xml.zip For persons, the person 'id' seems suitable. For events, i should check if we use guid vs id

This seems to parse correctly with the parser that we used previously. I'm currently using the id attribute on elements, not the guid one, because that was the penta id. Are those not stable? If not, I'll switch over, but if they are, then a short number is easier to deal with than a long guid.

Actually the website still provides the xml in the old format. Perhaps you should keep using that: https://fosdem.org/2024/schedule/xml

[yoe]

Here is an example xml. I think you should be able to use this. schedule.xml.zip For persons, the person 'id' seems suitable. For events, i should check if we use guid vs id

This seems to parse correctly with the parser that we used previously. I'm currently using the id attribute on elements, not the guid one, because that was the penta id. Are those not stable? If not, I'll switch over, but if they are, then a short number is easier to deal with than a long guid.

Actually the website still provides the xml in the old format. Perhaps you should keep using that: https://fosdem.org/2024/schedule/xml

... and I do, for main schedule information, but that wasn't what this was about :)

[yoe]

For querying the email addresses, you can use this api endpoint: https://pretalx.fosdem.org/api/events/xxx/speakers/?

where xxx is the event see eg: https://pretalx-test.fosdem.org/api/events/democon/speakers/? you can browse here after logging in or use an api key that can be found at: https://pretalx.fosdem.org/orga/me (we should probably create an sreview account).

It looks like that endpoint contains all email adresses, including from submitted but not accepted talks - you should merge with the talk 'code'

I do however think that this is only part of the necessary changes. We should also look at where and how submitters and devroom managers can see a list of videos that should be reviewed.

I haven't had time to look into this until now.

The test instance shows me a list of speakers, but no email addresses, not even after logging in.

The production instance does not show me a list of speakers -- as in, the API call returns this:

{
    "count": 0,
    "next": null,
    "previous": null,
    "results": []
}

[johanvdw]

So we now have: To fetch speaker details: https://pretalx.fosdem.org/fosdem-2024/p/matrix/ I recommend using this one instead of the speakers one from pretalx because it only shows relevant speakers and it is our own endpoint so we can make it stable. Let me know if you need extra fields

For adding the video links after they are ready we have: https://pretalx.fosdem.org/fosdem-2024/p/videolink/ this gives a list of all talks with their respective videolinks. You can post a new link to https://pretalx.fosdem.org/fosdem-2024/p/videolink// eg curl -H "Authorization: Token $PRETALX_TOKEN" -X POST -H "Content-Type: application/json" -d '[{"link":"http://localhost/", "description": "Video Recording xxx"}]' https://pretalx-test.fosdem.org/democon/p/videolink/3/ and clear any existing video recordings with: curl -H "Authorization: Token $PRETALX_TOKEN" -X POST -H "Content-Type: application/json" -d '[]' https://pretalx-test.fosdem.org/fosdem-2024/p/videolink/3/

Also again here, if you would like to store additional things, let me know. The description must start with Video Recording to work.

[yoe]

the /p/videolink URL (on a GET) seems to work (haven't attempted the POST part yet).

The /p/matrix URL does not work. I get a 404. Can you please look at that?

Thanks,

[johanvdw]

Did you try with your token?

Pretalx tends to send 404 if you are not logged in (which I dislike a bit).

If logged in you should also be able to see https://pretalx.fosdem.org/fosdem-2024/p/matrix/ with your user.

[yoe]

I only tried it from the browser for now, and I was logged in when I tried it. Makes no difference.

[yoe]

On the assumption that perhaps I never accepted the invite which you sent me to join the FOSDEM team (I'm not 100% sure but this might actually have been forgotten), I went to my mailbox and followed the link again, just to make sure. It did not resolve the issue.

I also tried the videolink POST. After downloading my token and setting it (correctly) in the Authentication header , on /fosdem-2024/p/videolink/3023 I get a 301 to /fosdem-2024/p/videolink/3023/, and there I get a 404.

[yoe]

So the command I tried for the second one is:

curl -i -X POST -q -H "Authorization: Token (...token goes here...)" -H "Content-Type: Application/json" -d '[{"link":"https://video.fosdem.org/invalid", "description": "Video recording (mp4)"}]' https://pretalx.fosdem.org/fosdem-2024/p/videolink/3023/

[johanvdw]

Ah, now I understand the issue, I changed the rights of the fosdem staff team to make them reviewer only, and made program team the only who could edit. I sent you a new invite which should make sure you can edit proposals (which is the field I check for).

[yoe]

You probably guessed as much already, but: after accepting the invite, I am able to access the APIs now.

I'd like to keep this issue open in case I find some problems during testing, but I'm 99% sure that won't be the case. I'll let you know in a few days if there's anything else.

[johanvdw]

This was implemented.