search

scrapy / scurl

Performance-focused replacement for Python urllib
Apache License 2.0
21 stars 6 forks source link

Support function urlparse #20

Closed malloxpb closed 6 years ago

malloxpb commented 6 years ago

Hey @lopuhin , as I am working on adding more funcs to the project as mentioned in #18, I realized to support urlparse, we will need to add some options to the current urlsplit function, particularly option scheme and allow_fragments.

Currently, I know that if the input does not have scheme and the scheme option is specified, then the output will contain the input scheme. That's why I created this PR. Do you think that is correct? Please let me know 😄

malloxpb commented 6 years ago

Hey @lopuhin , we do have urlsplit test with scheme passed https://github.com/nctl144/urlparse4/blob/master/tests/test_urlparse.py#L764

malloxpb commented 6 years ago

Hey Konstantin, after checking the performance test, I got the performance of urlparse is 0.19 sec (if it does not have to decode the result) and 0.22sec (if it has to decode the result), compared to 0.13 and 0.15 sec of urlsplit. There might still be a way to further optimize this function 😄

malloxpb commented 6 years ago

I will work on the failed tests now 😄 particularly creating a tuple class for this function