search

nikulpatel3141 / ETF-Scraper

Scrape public ETF and Mutual Fund holdings information
MIT License
16 stars 1 forks source link

Non US end points #8

Open jaacobl opened 9 months ago

jaacobl commented 9 months ago

It seems like iShares has different endpoints for different regions... Any chance you could get the other regions to work as well? I'm trying to scrape holdings for iShares etfs that are not listed under the product set for the US region (i.e. ISF, IAEX, etc).

Example etf listed under 'ch' region but not 'us' :

https://www.ishares.com/ch/professionals/en/products/etf-investments#/?productView=etf&pageNumber=1&sortColumn=totalFundSizeInMillions&sortDirection=desc&dataView=keyFacts

provider = Provider.IShares.value

host = "https://www.ishares.com"

listing_endpoint = (
    "/us/product-screener/product-screener-v3.1.jsn?dcrPath=/templatedata/config/product-screener-v3/"
    "data/en/us-ishares/ishares-product-screener-backend-config&siteEntryPassthrough=true"
)

I tried replacing /us/ with some other regions (i.e. /ch/) but it doesnt look like the urls are the same.

nikulpatel3141 commented 9 months ago

Hi, you're right I haven't included other iShares regions. Those are slightly more difficult for various reasons (different share classes, file formats, URLs etc).

If it's just a few funds you're after you can manually look at the download URLs and alter the holdings dates to scrape historical holdings.