python-validators / validators

Python Data Validation for Humans™.
MIT License
977 stars 155 forks source link

not working for some urls #213

Closed DeepInEvil closed 1 year ago

DeepInEvil commented 2 years ago

Hey guys, thanks a lot for the work! really appreciate it.

However it is failing for some urls like 'www.intesasanpaolo.com' Can you guys please check?

Wombatpm commented 2 years ago

I hit the same issue and am working on a pull request. The problem is that the validator is expecting a protocol such as http:// or https:// or ftp://. Now I was taught long ago that a url required a protocol, but that was back in the day when browsers handled ftp, gopher, telnet, and others. Now a url is just assumed to be a web address and browser will put http or https automatically. Even the MDN documentation says it is optional.

You don't need to include the protocol (the browser uses HTTP by default) or the port (which is only required when the targeted Web server is using some unusual port), but all the other parts of the URL are necessary.May 15, 2022

[[What is a URL? - Learn web development | ](https://developer.mozilla.org/en-US/docs/Learn/Common_questions/What_is_a_URL#:~:text=You%20don't%20need%20to%20include%20the%20protocol%20(the%20browser,of%20the%20URL%20are%20necessary.)

In validators/url.py. line 10 needs to be changed to capture 0 or 1 of the protocol identifier

    # protocol identifier
    r"(?:(?:https?|ftp)://)?"
Wombatpm commented 2 years ago

I ended up creating a new validator called website. It is designed to work with inputs that may be coming from a marketing type organization where the concepts of URL, WEBSITE, INTERNETADDRESS get all bumbled togther. Its basically the same as the url validator, but the protocols have been limited to http and https. and the whole need for a protocol (http://) portion is option. http://home.netscape.com and home.netscape.com are now both valid websites, but the later will fail the url validator.

You can see my changes and pull request here https://github.com/kvesteri/validators/pull/214