search

Crinibus / scraper

Web scraper for scraping, tracking and visualizing prices of products on various websites.
MIT License
157 stars 30 forks source link

Error with adding Elgiganten links after update and new feature request #232

Closed Trixarn closed 10 months ago

Trixarn commented 10 months ago

Thank you so much for the new update! But now I'm having a problem with adding Elgiganten link from .dk and .se but just commented out the .dk link in domains.py so it works for me now :)

    def _get_json_api_data(self) -> dict:
        id_number = self._get_product_id()

        # API link to get price and currency
#        if "elgiganten.dk" in self.url:
 #           api_link = f"https://www.elgiganten.dk/cxorchestrator/dk/api?appMode=b2c&user=anonymous&operationName=getProductWithDynamicDetails&variables=%7B%22articleNumber%22%3A%22{id_number}%22%2C%22withCustomerSpecificPrices%22%3Afalse%7D&extensions=%7B%22persistedQuery%22%3A%7B%22version%22%3A1%2C%22sha256Hash%22%3A%229bfbc062032a2a6b924883b81508af5c77bbfc5f66cc41c7ffd7d519885ac5e4%22%7D%7D"  # noqa E501
  #      elif "elgiganten.se" in self.url:
        api_link = f"https://www.elgiganten.se/cxorchestrator/se/api?getProductWithDynamicDetails&appMode=b2c&user=anonymous&operationName=getProductWithDynamicDetails&variables=%7B%22articleNumber%22%3A%22{id_number}%22%2C%22withCustomerSpecificPrices%22%3Afalse%7D&extensions=%7B%22persistedQuery%22%3A%7B%22version%22%3A1%2C%22sha256Hash%22%3A%22229bbb14ee6f93449967eb326f5bfb87619a37e7ee6c4555b94496313c139ee1%22%7D%7D"  # noqa E501
#        else:
#            raise WebsiteVersionNotSupported(get_website_name(self.url, keep_tld=True))
        response = request_url(api_link)
        return response.json()

Also changed short_url to .se instead of the .dk

    def get_short_url(self) -> str:
        id = self._get_product_id()
        return f"https://www.elgiganten.se/product/{id}"

One more question, is it possible to fetch the outlet prices from like https://www.elgiganten.dk/product/mobil-tablet-smartwatch/mobiltelefon/google-pixel-7-pro-smartphone-12128-gb-obsidian/525075 or add a parameter to the database from like https://www.elgiganten.dk/product/outlet/google-pixel-7-pro-smartphone-12128-gb-obsidian/582585 to see when it is back in the outlet stock? Maybe hard to code it.. :)

Crinibus commented 10 months ago

I can see both problems. Both problems should be fixed in #233 and #234. I don't think I will prioritize getting the outlet price for now. If I get the time and find a way to implement a way to look when a product is back in stock, I might add the functionality to do so.