search

alirezamika / autoscraper

A Smart, Automatic, Fast and Lightweight Web Scraper for Python
MIT License
6.24k stars 654 forks source link

[refactor] optimize unique function #4

Closed Lulzx closed 4 years ago

Lulzx commented 4 years ago

using regular dict to extract unique values is significantly faster if compared to other methods.

alirezamika commented 4 years ago

Does it preserve the order of the elements in the list? Also, items may not be hashable.

Lulzx commented 4 years ago

Yes, It preserves the order. But, I'm unsure about the latter.

alirezamika commented 4 years ago

The list contains unhashable items (like lists) so it doesn't work. Thank you.