Invalid use of discovery.sendtargets parameters

[Patricol]

Summary: csi-driver-iscsi tries to update values that don't exist; and just keeps retrying - never successfully mounting anything.

What happened:

iscsiadm.go:85] Setting CHAP Discovery...
iscsiadm.go:129] Begin createCHAPEntries (discovery=true)...
multipath.go:33] Executing command 'iscsiadm' with args: '[-m node -T REDACTED -p REDACTED -o update -n discovery.sendtargets.auth.authmethod -v CHAP -n discovery.sendtargets.auth.username -v REDACTED -n discovery.sendtargets.auth.password -v REDACTED -n discovery.sendtargets.auth.username_in -v REDACTED -n discovery.sendtargets.auth.password_in -v REDACTED]'.
multipath.go:61] Finished executing command.
iscsiadm.go:44] Run iscsiadm command: iscsiadm -m node -T REDACTED -p REDACTED -o update -n discovery.sendtargets.auth.authmethod -v CHAP -n discovery.sendtargets.auth.username -v REDACTED -n discovery.sendtargets.auth.password -v REDACTED -n discovery.sendtargets.auth.username_in -v REDACTED -n discovery.sendtargets.auth.password_in -v REDACTED
iscsiadm.go:52] Output of iscsiadm command: {output: }
iscsiadm.go:54] Error message returned from iscsiadm command: exit status 7
iscsi.go:388] Error creating db entry: failed to update discoverydb with CHAP, err: exit status 7

exit status 7 is ISCSI_ERR_INVAL - invalid argument., and sure enough; when running directly I get

iscsiadm: Cannot modify discovery.sendtargets.auth.authmethod. Invalid param name.
iscsiadm: Could not execute operation on all records: invalid parameter

(It'll mention whichever arg is first in the list; all 5 are invalid.) Relevant function is here: https://github.com/kubernetes-csi/csi-driver-iscsi/blob/2214d7113eb76501a189c7948937fe2cc19ec8b8/pkg/iscsilib/iscsiadm.go#L127 Replacing all the discovery.sendtargets prefixes with node.session works; so it's likely an issue of my iscsiadm (which is v2.1.10) not populating the discovery.sendtargets fields; so they're not there to be updated.

What you expected to happen: Not actually sure if or where the discovery.sendtargets should be added; the iscsiadm docs show rather limited uses of -o update with -n and -v. (Updating name/value pairs.) So that does imply that it might just need to be skipped; or the initial population of args should be adjusted to ensure they are set.

How to reproduce it:

Used both discoveryCHAPAuth=true and sessionCHAPAuth=true; specified all 8 values in the secret json. Ran on a fresh node; so the rescan etc. failed (which isn't an issue.) Used the example nginx yaml from this repo.

Anything else we need to know?:

Environment:

• CSI Driver version: current master (2214d7113eb76501a189c7948937fe2cc19ec8b8)
• Kubernetes version (use kubectl version): v1.30.3; kubectl is v1.31.0.
• OS (e.g. from /etc/os-release): nixos 24.11.20240824.d0e1602
• Kernel (e.g. uname -a): 6.10.6
• Install tools: Unsure what this means. Using a terraform-proxmox-nixos-rke2 stack.
• Others: iscsid and iscsiadm are both v2.1.10.

[humblec]

@Patricol definitely its on iscsiadm binary and version. We generally test the iscsiadm from open-iscsi https://github.com/open-iscsi/open-iscsi . The iscsiadm here looks like coming from nixos .. do you have any other setup where you can test this ? or can you try the iscsiadm based on open-iscsi and revert ?

[Patricol]

Perhaps I'm misunderstanding, but NixOS's iscsiadm is from open-iscsi.

The package is built here, it basically just clones https://github.com/open-iscsi/open-iscsi and runs its regular build; only setting a few directory locations.