voxxrin / voxxrin3

GNU Affero General Public License v3.0
9 stars 2 forks source link

Devoxx crawler should use eventId #12

Closed stephanj closed 1 year ago

stephanj commented 1 year ago

The Devoxx crawler should use an cfpId request parameter so each CFP.DEV instance can use it's own event slug to trigger a refresh (dvbe23, dvma23, dvua23, etc.). As a result we can automatic crawling refreshes with in the CFP.DEV web app.

See cloud/functions/src/crawlers/devoxx/crawler.ts

If cfpId is not constant and varies based on different conditions, you might want to turn DEVOXX_CRAWLER into a function that takes cfpId as an argument:

typescript
Copy code
export const getDevoxxCrawler = (cfpId: string) => {
    return {
        // existing DEVOXX_CRAWLER code,
        // use cfpId where needed
    };
}
You'd then call this function with the appropriate cfpId whenever you're setting up a new crawler.
fcamblor commented 1 year ago

You can now pass an eventId query parameter to the crawler in order to only crawl some specific events granted to your crawlingToken :

Example : https://us-central1-voxxrin-v3-demo.cloudfunctions.net/crawl?eventId=dvbe23&crawlingToken=