iamrony777 / javinfo-api

An API to scrape Japanese adult video data from various sites
57 stars 17 forks source link

Add Priority List for provider #33

Closed lucius100 closed 3 months ago

lucius100 commented 4 months ago

Can we add priority list to the provider ? I mean the order of scraping, instead of get data from the fastest provider, it will search the priority provider for example It will go first from r18-javdb-javdatabase-javlibrary in the way we arrange / setting the priority, because each site has different category / tags, make the data seems complicated

it will search first from r18, if no result it goes to javdb, then the next, not just random scraping the fastest.

iamrony777 commented 4 months ago

it will search first from r18, if no result it goes to javdb, then the next, not just random scraping the fastest.

This approach will increase response time, but instead of prioritizing requests, I think I can prioritize responses. For example, fetch all responses at once, then return the result according to priority.