Improve `image upload` command documentation

[mrndev]

Description

ionosctl image upload not working. Im using the same command as in the example from help

~$ ionosctl image upload --location fra --image OPNsense-23.1-OpenSSL-dvd-amd64.iso Error: ftps: connection setup failed: cmd failed read expected code 331 with message "": failed to read code, got code 500 and message USER: command requires a parameter: 500 USER: command requires a parameter

Am I missing some parameters? Its hard to interpret what the error means...

Expected behavior

image gets uploaded

Environment

~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.1 LTS
Release:        22.04
Codename:       jammy
~$ ionosctl --version
ionosctl version v6.6.3

How to Reproduce

see command above

[avirtopeanu-ionos]

hi, are you logged in? Try exporting IONOS_USERNAME and IONOS_PASSWORD environment variables, or ionosctl login to generate a credentials file. Since ionosctl image upload uses FTP, it doesn't support use of tokens (IONOS_TOKEN).

Note: We are currently working on an improvement #295 which will improve various functionalities of the config file, including ionosctl login.

[mrndev]

I'm using the token so that probably is the problem. Thanks. (The error message could be more helpful though)

Is ionosctl using FTP+TLS by default? Is there any risk that my credentials are sent in clear text using plain FTP?

[avirtopeanu-ionos]

Hi, yes. I agree, the error message should be improved. As for FTP+TLS - No risk of plain FTP. A security risk is if you are setting the --skip-verify flag (by default this is not set), which skips TLS cert check and you can be man-in-the-middle attacked

dialOptions := ftps.DialOptions{
    Host:        p.Url,
    Port:        p.Port,
    Username:    s.client.GetConfig().Username,
    Passowrd:    s.client.GetConfig().Password,
    ExplicitTLS: true, // means that FTPS (FTP Secure, FTP over TLS) is being used
    TLSConfig:   &tlsConfig,
}

tlsConfig := tls.Config{
    InsecureSkipVerify: p.SkipVerify,
    ServerName:         p.Url,
    RootCAs:            p.ServerCertificate,
    MaxVersion:         tls.VersionTLS12,
}

[mrndev]

I exported IONOS_USERNAME and IONOS_PASSWORD and now it works - thanks.

during upload i bumped to this:

1. I started an image upload
2. I interrupted it with ctrl-c
3. image upload could not be started again anymore because the image is already on the server, which is ok'ish, if all i need to do is to delete the partly uploaded image
4. I was able to see the image with ionosctl list. I then removed the image with ionosctl delete. The image was not anymore listed with ionosctl list

5. If I now try to upload the image, the ionosctl still gives the error that the image is already on the server

   ~$ ionosctl image list | grep OPN WARNING: The following flags are deprecated: --type TYPE --licence-type LICENCE_TYPE --location LOCATION --image-alias IMAGE_ALIAS --latest LATEST. Use --filters --order-by --max-results options instead! ~$ ionosctl image upload -v --location fra --image OPNsense-23.1-OpenSSL-dvd-amd64.iso [INFO] Uploading OPNsense-23.1-OpenSSL-dvd-amd64.iso to ftp-fra.ionos.com Error: OPNsense-23.1-OpenSSL-dvd-amd64.iso already exists at ftp-fra.ionos.com

as a workaround i can of course rename my image file, after which it works, but just for the record - maybe this can be improved.

[avirtopeanu-ionos]

Hi, I agree. I reproduced, and so I got in touch with the API team. TL;DR: since the FTP server is independent of the API, you have to sadly make a support request at support@cloud.ionos.com to delete the 0 byte files from the FTP server (which you can't manually do, sadly).

That's the expected behaviour. Files uploaded via FTP cannot be deleted on the FTP server itself. FTP is uploading the image to VDC/DCD - and does not know if the image still exists there, or if it has been deleted via DCD/API at a later time. Customers can open a Support Ticket asking to delete the placeholder (file with 0 byte) on the FTP server: support@cloud.ionos.com

I will however add this to the documentation, I wasn't aware of this behaviour myself.

[mrndev]

ok - thanks for checking this

[avirtopeanu-ionos]

I will close this issue when:

a) the quirk described in https://github.com/ionos-cloud/ionosctl/issues/302#issuecomment-1592798628 is documented, and point the users in the direction of making a support request at support@cloud.ionos.com b) the error message for invalid credentials is improved c) the documentation for logging in is improved (e.g. you can only use username & password, token is not supported) d) we let the user know that we are indeed using FTP over TLS. so, no plaintext passwords

[mrndev]

great product care!

[hegerdes]

Hi, I agree. I reproduced, and so I got in touch with the API team. TL;DR: since the FTP server is independent of the API, you have to sadly make a support request at support@cloud.ionos.com to delete the 0 byte files from the FTP server (which you can't manually do, sadly).

That's the expected behaviour. Files uploaded via FTP cannot be deleted on the FTP server itself. FTP is uploading the image to VDC/DCD - and does not know if the image still exists there, or if it has been deleted via DCD/API at a later time. Customers can open a Support Ticket asking to delete the placeholder (file with 0 byte) on the FTP server: support@cloud.ionos.com

I will however add this to the documentation, I wasn't aware of this behaviour myself.

This is a major drawback!

Aborted uploads (which happens in unstable networks) can not be retired. You have to open a support ticket to delete something you uploaded yourself?

Even if it is not aborted it regularly errors because the image service is too slow to recognize a new upload. See the following logs:

Logs

``` $ wget -q https://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-amd64.vmdk -O vm-images/ubuntu-focal-amd64-${IMG_POSTFIX}.vmdk $ ionosctl image upload -i vm-images/ubuntu-focal-amd64-${IMG_POSTFIX}.vmdk --location txl,fra --name CI_DEMO-${IMG_POSTFIX} --cloud-init V1 --description DEMO --licence-type LINUX --verbose [INFO] Uploading vm-images/ubuntu-focal-amd64-2023-09-05.vmdk to ftp-txl.ionos.com [INFO] Uploading vm-images/ubuntu-focal-amd64-2023-09-05.vmdk to ftp-fra.ionos.com [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] [INFO] Got images by listing: [] [INFO] Total images: [] Error: failed updating image with given properties, but uploading to FTP sucessful: ran out of time: context deadline exceeded ```

You then have to wait x minutes until the service decides to pick up the images and then set the attributes which completely obsoletes the use of the ionosctl.

Please provide an integrated image upload system that is HTTP based and not on FTP and does not need the account credentials. My recommendation would be a S3 based system

[mrndev]

Hi Henrik, thanks for your comment. The feature/improvement you are requesting is independent of the ionosctl CLI, but generic suggestions to improve the products always helpful. I will add a feature request for this.

[avirtopeanu-ionos]

@hegerdes you could attempt uploading the images with a longer context deadline by using --timeout <seconds>, which might result in the images being picked up by ionosctl.

However, I can't comment on the FTP nature of the system. I would like working with an S3 based one, but as @mrndev said this is independent of this tool sadly.

[avirtopeanu-ionos]

Hi! #411 has finally been released in v6.7.3. Sorry for the delay.