search

cernopendata / cernopendata-client

CERN Open Data command-line client
http://cernopendata-client.readthedocs.io/
GNU General Public License v3.0
10 stars 9 forks source link

cernopendata-client download-files --recid 3005 --verify --protocol https #92

Closed tiborsimko closed 3 years ago

tiborsimko commented 3 years ago 
$ cernopendata-client download-files --recid 3005 --verify --protocol https
==> Downloading file 1 of 1
  -> File: ./3005/0d0714743f0204ed3c0144941e6ce248.configFile.py
Traceback (most recent call last):
  File "/home/simko/.virtualenvs/cernopendata-client/bin/cernopendata-client", line 8, in <module>
    sys.exit(cernopendata_client())
  File "/home/simko/.virtualenvs/cernopendata-client/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/simko/.virtualenvs/cernopendata-client/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/simko/.virtualenvs/cernopendata-client/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/simko/.virtualenvs/cernopendata-client/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/simko/.virtualenvs/cernopendata-client/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/simko/.virtualenvs/cernopendata-client/lib/python3.8/site-packages/cernopendata_client/cli.py", line 291, in download_files
    download_single_file(path=path, file_location=file_location, protocol=protocol)
  File "/home/simko/.virtualenvs/cernopendata-client/lib/python3.8/site-packages/cernopendata_client/downloader.py", line 68, in download_single_file
    c.perform()
pycurl.error: (1, 'Protocol "root" not supported or disabled in libcurl')
tiborsimko commented 3 years ago

Another trouble with HTTPS protocol:

$ cernopendata-client download-files --recid 3005 --verify --protocol https --server https://opendata-dev.cern.ch
==> ERROR: Invalid value for --server: https://opendata-dev.cern.ch - Server should be a valid HTTP URI
ParthS007 commented 3 years ago

I fixed it in #89 itself.

tiborsimko commented 3 years ago

Thanks. One thing that remains is what to do when users specify --protocol https but --server http://.... The current behaviour is:

$ cernopendata-client get-file-locations --recid 3005 --protocol https --server http://opendata-dev.cern.ch
https://opendata.cern.ch/eos/opendata/cms/configuration-files/2011/0d0714743f0204ed3c0144941e6ce248.configFile.py
$ cernopendata-client get-file-locations --recid 3005 --protocol http --server http://opendata-dev.cern.ch
http://opendata-dev.cern.ch/eos/opendata/cms/configuration-files/2011/0d0714743f0204ed3c0144941e6ce248.configFile.py
$ cernopendata-client get-file-locations --recid 3005 --protocol http --server https://opendata-dev.cern.ch
https://opendata-dev.cern.ch/eos/opendata/cms/configuration-files/2011/0d0714743f0204ed3c0144941e6ce248.configFile.py

That is, even if users force HTTP it is HTTPS which is used.

Two solutions:

  1. Let's verify strictly that HTTP protocol means HTTP server, and HTTPS protocol means HTTPS server, and always raise errors for HTTP/HTTPS mixtures.

  2. Let's remove https protocol from options, keeping only http or xrootd, and let's accept --server http://... or --server https://... as valid possibilities, and let's start either HTTP or HTTPS transler based on that.

I think the second option would be more user friendly. We don't really need two separate options to specify this. If users specify the protocol already as part of the server, then we can take that. (Kind of like people write firefox http://example.org or firefox https://example.org without worrying about any other command line options.)