Clueless-Community / scrape-up

A web-scraping-based python package that enables you to scrape data from various platforms like GitHub, Twitter, Instagram, or any useful website.
https://pypi.org/project/scrape-up/
MIT License
251 stars 241 forks source link

walmart_scraper #1091

Closed DivyaVijay1234 closed 3 months ago

DivyaVijay1234 commented 5 months ago

Description

This pull request creates the WalmartProduct class by adding detailed logging for debugging, improving error handling with specific messages for each method, verifying and matching Walmart's current structure, and ensuring robust configuration management for request headers. These changes improve the reliability and maintainability of the web scraping process, making it easier to identify and resolve issues when fetching product details, images, and customer reviews from Walmart's website.

Resolves: #969

Checklist

Before submitting this pull request, kindly verify that the ensuing checkpoints have been reached.

Screenshots

Screenshot 2024-06-01 005018

I certify that I have carried out the relevant checks and provided the requisite screenshot for validation by submitting this pull request.

DivyaVijay1234 commented 5 months ago

Hello @nikhil25803, I have made the changes requested by you. Do let me know if any further changes are required. Thank you.

nikhil25803 commented 5 months ago

I tested the endpoints, some of them are returning error and even no result at all

DivyaVijay1234 commented 5 months ago

@nikhil25803 Would you be able to give me a few examples of cases for which the code fails to yield accurate results? I will look into it and make necessary changes. I appreciate your time and patience in this regard. Thank you.