retry for ftp/tftp

[bcg62]

Retry functionality exists for http/https, but not ftp/tftp

https://coreos.com/ignition/docs/latest/operator-notes.html#http-backoff-and-retry

This can create flaky results if boot ordering, or delays in network initialization throw off ignition on its first attempt to grab configuration.

https://github.com/coreos/ignition-dracut/issues/41

[bgilbert]

Ignition doesn't support FTP, but you're right about TFTP.

We should probably abstract the retry logic into generic code.

[bcg62]

is there a specific reason ftp is excluded? Aside from the config.url cmdline property i see it also cannot be used for remote files.

IE:

    - path: /usr/local/bin/test.sh
      filesystem: root
      contents:
        remote:
          url: ftp://.....
      mode: 755

$ ct -in-file  ./test
error at line 264, column 14
invalid url scheme

[bgilbert]

@bcg62 Is there a particular reason FTP should be supported? We generally don't add features to Ignition unless there's a compelling need, and HTTP infrastructure is much more broadly deployed.

(TFTP is supported because the machine may have been booted via PXE + TFTP, and it can be convenient to keep all bootstrap data in the same place.)

[bcg62]

@bgilbert right, i am in that camp with tftp as my infrastructure is pxe booted. I also happen to have some things hosted via ftp.

http://man7.org/linux/man-pages/man7/dracut.cmdline.7.html Its slightly confusing that ftp is supported in an initramfs generated via dracut (root=live:ftp://..) which i am also using.

To see this functionality work in the raw initramfs, and fail in ignition leaves one wondering why it wasn't implemented.

It also adds additional burden on a user trying to convert from cloud-init to ignition like my self as i now have to revamp extraneous things outside of ignition.

[ajeddeloh]

I think ftp was never implemented because no one asked for it/implemented it.

With regards to timeouts, it looks like each protocol handles them differently and we should fix this. Specifically there are two main parts: 1) Retries and backoff: This looks like it's currently only implemented for HTTP(S). When Ignition fails to open a connection (and get HTTP headers) it retries a number of times with an exponential backoff. This should be abstracted and applied to all supported url schemes. 2) Timeouts within a protocol: We have different timeouts for different aspects of a protocol (again only implemented for HTTP(S)). These are things like the timeout to get headers, timeouts to complete the transaction, etc. These by their nature are tied to their protocol.