Giglium / vinted_scraper

A very simple Python package that scrapes the Vinted website to retrieve information about its items.
MIT License
15 stars 3 forks source link

Retrieving the online date of an article does not work #37

Closed FlyZ1z closed 7 months ago

FlyZ1z commented 8 months ago

I'm currently trying to retrieve the date but it keeps getting None Could someone please explain to me ?

Giglium commented 8 months ago

Hi can you give me a code sample? Normally, If some attributes are None means that it wasn't found in the response, maybe because they are returned from other APIs

FlyZ1z commented 8 months ago

I'm trying to get the date using created_at but it's not working with this code, this may be my way of using it too

`def detailurl(url, limit=10): parsed_url = urlparse(url) query_params = parse_qs(parsed_url.query)

catalog_ids = query_params.get('catalog[]', [])
size_ids = query_params.get('size_ids[]', [])
brand_ids = query_params.get('brand_ids[]', [])
price_to = query_params.get('price_to', [])[0]
currency = query_params.get('currency', [])[0]
status_ids = query_params.get('status_ids[]', [])
created_at = query_params.get('create_at[]', [])

vinted_wrapper = VintedWrapper("https://www.vinted.fr")

search_params = {
    "order": "newest_first",
    "created_at": created_at
}

search_results = vinted_wrapper.search(params=search_params)
items_details = []

if search_results.get("items"):
    items = [VintedItem(json_data=item) for item in search_results["items"][:limit]]

    for item in items:
        created_at = item.created_at if hasattr(item, "created_at") else "Non spécifié"
        items_details.append({
            "id": item.id,
            "created_at": created_at

 print(items_details)

 if __name__ == "__main__":
url = "https://www.vinted.fr/catalog?catalog[]=79&order=newest_first&brand_ids[]=304&size_ids[]=208&price_to=30&currency=EUR"
detailurl(url, limit=10)`
Giglium commented 7 months ago

The search API doesn't return the created_at attribute that is returned from the item API. If you print search_results you will not find it in the response.

If you want the created_at attribute you should call the vinted_wrapper.item() function for each item. I modify your code:

vinted_wrapper = VintedWrapper("https://www.vinted.fr")

search_params = {
    "order": "newest_first",
}

search_results = vinted_wrapper.search(params=search_params)
items_details = []

if search_results.get("items"):
    items = [VintedItem(json_data=vinted_wrapper.item(item["id"])["item"]) for item in search_results["items"][:10]]

    for item in items:
        created_at = item.created_at if hasattr(item, "created_at") else "Non spécifié"
        items_details.append({
            "id": item.id,
            "created_at": created_at})

    print(items_details)

Basically, I've changed: items = [VintedItem(json_data=item) for item in search_results["items"][:limit]] with: items = [VintedItem(json_data=vinted_wrapper.item(item["id"])["item"]) for item in search_results["items"][:limit]]

Or a version with the VintedScraper obj that handles all the conversion from JSON to VintedItem:

vinted_wrapper = VintedScraper("https://www.vinted.fr")

search_params = {
    "order": "newest_first",
}

search_results = vinted_wrapper.search(params=search_params)
items_details = []

items = [vinted_wrapper.item(item.id) for item in search_results[:10]]

for item in items:
    created_at = item.created_at if hasattr(item, "created_at") else "Non spécifié"
    items_details.append({
        "id": item.id,
        "created_at": created_at})

print(items_details)

Let me know if it works for you otherwise, I will close this issue since it isn't a bug.

FlyZ1z commented 7 months ago

Yes it works thank you very much !