TeMU-BSC / iberifier

2 stars 0 forks source link

Rewrite API Call to loop through the days rather than collecting the information with from and to limits #42

Closed Oliph closed 1 year ago

Oliph commented 1 year ago

Currently on Twitter API call and MyNews API call, we use the from and the to to bound the query between two dates. The issues is that if the limit is hit before reaching the end of the period we only get the first day in the order returned by the API. One way to do could be to adapt the following to the MyNews API and Twitter API

def get_call_per_day(query, period, overall_limit):
    remaining_days = len(period)
    remaining_limit = overall_limit
    limit_per_day = remaining_limit/remaining_days
    for day in period:
        call, nbr_results = api(query, day, limit_per_day)
        remaining_days = remaining_days - 1
        remaining_limit = remaining_limit - nbr_results
        limit_per_day = remaining_limit/remaining_days