mystic-ai / pipeline

Pipeline is an open source python SDK for building AI/ML workflows
https://www.mystic.ai
Apache License 2.0
117 stars 21 forks source link

container file tweaks #373

Closed plutopulp closed 6 months ago

plutopulp commented 6 months ago

Allow for file.urls and fixes a bug with file url parsing.

plutopulp commented 6 months ago

What was the bug this was trying to fix out of interest?

Also, if we no longer need the validators package could you remove it from poetry and also from pipeline/container/docker_templates/dockerfile_template.txt?

Yeah good shout, have removed the validators package as doesn't seemed to be used anymore. As discussed, originally we had:

 if validators.url(path_or_url):
            url = str(urllib.parse.urlparse(path_or_url).geturl())

But it doesn't return a bool when it fails, but an error object, e.g.:

>>> validators.url("abc@email.com")
ValidationError(func=url, args={'value': 'abc@email.com'})