Giglium / vinted_scraper

A very simple Python package that scrapes the Vinted website to retrieve information about its items.
MIT License
15 stars 3 forks source link

Dynamically set user agent on _curl function #3

Closed Giglium closed 1 year ago

Giglium commented 1 year ago

What is the current behavior? And why it does suit you?

Right now, I have hardcoded the user agent: https://github.com/Giglium/vinted_scraper/blob/591fec59802bf599af0656bd61e0030952bd978f/src/vinted_scraper/__init__.py#L73-L77

It should be more useful if we randomly change the user agent also to keep up to date with the most used.

Describe the solution you'd like

It should be nice to be able to load the user agent randomly from a file list and maybe also add a Makefile command that performs a wget to update the list.