aws configure import --csv Prompt error, Expected header "User Name" not found (when missing `file://` prefix)

qian-jiahong commented 1 year ago

Describe the bug

When executing: aws configure import --csv, the wrong file path is entered, and the "error prompt" received is wrong.

Expected Behavior

$ aws configure import --csv aws_user_credentials.csv $ aws configure import --csv aws_user_credentials-noexist.csv

It should prompt that the file does not exist. Instead of: Expected header "User Name" not found

Current Behavior

$ aws configure import --csv aws_user_credentials.csv Expected header "User Name" not found

$ aws configure import --csv aws_user_credentials-noexist.csv Expected header "User Name" not found

$ aws configure import --csv file://./aws_user_credentials.csv Successfully imported 1 profile(s)

$ aws configure import --csv file://./aws_user_credentials-noexist.csv Error parsing parameter '--csv': Unable to load paramfile file://./aws_user_credentials-noexist.csv: [Errno 2] No such file or directory: './aws_user_credentials-noexist.csv'

Reproduction Steps

Download the certificate file (.csv) from the aws website to the current directory. Named as: aws_user_credentials.csv $ aws configure import --csv aws_user_credentials.csv $ aws configure import --csv aws_user_credentials-noexist.csv

Possible Solution

No response

Additional Information/Context

No response

CLI version used

aws-cli/2.11.3 Python/3.11.2 Linux/5.15.0-56-generic exe/x86_64.ubuntu.20 prompt/off

Environment details (OS name and version, etc.)

Linux ubuntu20-s1 5.15.0-56-generic #62~20.04.1-Ubuntu SMP Tue Nov 22 21:24:20 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

tim-finnigan commented 1 year ago

Hi @qian-jiahong thanks for reaching out. This is an error that I can reproduce. The documentation for aws configure import does use file:// as a prefix, but regardless the Expected header "User Name" not found is not a helpful error message.

We received another issue recently (https://github.com/aws/aws-cli/issues/7721) involving this command, but that issue specifically had to do with a Byte Order Mark that wasn't getting parsed properly, and that also resulted in the Expected header "User Name" not found error. That issue should now be fixed in version 2.11.4. I will slightly modify the title of this issue to distinguish the two issues.

vitaly-pavluk commented 1 year ago

Thank you for that issue. It drives me crazy for 2 hours when I tried to figure out why on my win laptop I cannot import csv in the terminal using just the command aws configure import --csv rootkey.csv cli told me that Expected header "User Name" not found but it was in the csv file. Finally when I changed the command on aws configure import --csv file://rootkey.csv everything was imported and no errors appeared in the terminal.
AWS SDK guys please just read "Principle of least astonishment" so it will dramatically reduce number of issues in AWS SDK GH.

qian-jiahong commented 1 year ago

Thank you for that issue. It drives me crazy for 2 hours when I tried to figure out why on my win laptop I cannot import csv in the terminal using just the command aws configure import --csv rootkey.csv cli told me that Expected header "User Name" not found but it was in the csv file. Finally when I changed the command on aws configure import --csv file://rootkey.csv everything was imported and no errors appeared in the terminal. AWS SDK guys please just read "Principle of least astonishment" so it will dramatically reduce number of issues in AWS SDK GH.

Me too.

tarsir commented 1 year ago

A couple weeks ago I gave this issue a try with my PR #7763, but I'm not sure if there's process I'm missing to get it reviewed by a maintainer 😅 My PR is probably a bit out of date by now, which I'm happy to fix if I could get a bit of guidance on the review process :)

ravinarayansingh commented 1 year ago

I was facing same issue and it working if you provide file path like file:///
aws configure import --csv file:///home/xxx/credential.csv

stride314 commented 11 months ago

I had the same result as @ravinarayansingh, adding the extra / resolved the problem. The error message "Expected header "User Name" not found" is unhelpful.

aws configure import --csv file:/users/xxxx/aws/s3-accessKeys.csv Expected header "User Name" not found

aws Michael$ aws configure import --csv file:///users/xxxx/aws/s3-accessKeys.csv Successfully imported 1 profile(s)

jcsahnwaldt commented 9 months ago

@tim-finnigan I think the real problem here is that the --csv parameter of the aws configure import command accepts either literal CSV or a file:// URI. Apparently aws configure import interprets anything that doesn't start with file:// as literal CSV.

Examples:

> aws configure import --csv 'Access Key ID,Secret Access key'
Expected header "User Name" not found

> aws configure import --csv 'User Name,Access Key ID,Secret Access key'
Successfully imported 0 profile(s)

> aws configure import --csv $'User Name,Access Key ID,Secret Access key\nFoo,Bar,Baz'
Successfully imported 1 profile(s)

> echo $'User Name,Access Key ID,Secret Access key\nFoo,Bar,Baz' > users.csv

> aws configure import --csv users.csv
Expected header "User Name" not found

> aws configure import --csv file:users.csv
Expected header "User Name" not found

> aws configure import --csv file:/users.csv
Expected header "User Name" not found

> aws configure import --csv file://users.csv
Successfully imported 1 profile(s)

> aws configure import --csv file://no-such-file.csv
Error parsing parameter '--csv': Unable to load paramfile file://no-such-file.csv: [Errno 2] No such file or directory: 'no-such-file.csv'

I'd say this behavior of the --csv parameter is a rather unusual (and thus confusing) choice for a CLI parameter. Some of these choices would probably be better:

Just interpret the --csv argument as a file name, don't accept literal CSV at all. Is that feature even necessary / useful? Does anyone use literal CSV on the command line?
Have separate parameters for literal CSV and CSV file, e.g. --csv-text and --csv-file.
Use a smarter heuristic to detect whether the argument contains literal CSV or a file name. For example: If the argument value starts with User Name and/or contains commas and/or contains line breaks, assume it's literal CSV; otherwise assume it's a file name. (That's close to what @tarsir's PR #7763 does.)

At the very least, the documentation should be improved. Currently, aws configure import help says:

--csv (string) The credentials in CSV format generated by the AWS web console. The CSV file must contain the "User name", "Access key ID", and "Secret access key" headers.

The "CSV file" part sounds like the argument should be a file name. And the part "in CSV format generated by the AWS web console" is at best half true, as the AWS web console omits the "User Name" header.

tim-finnigan commented 3 weeks ago

Thanks for sharing more context here. The --csv command does work inline like this:

aws configure import --csv "User Name,Access Key ID,Secret Access Key
username,AKIAIOSFODNN7EXAMPLE,wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"

But the documentation does reference a "CSV file" for that command, and doesn't specify that the file:// prefix is required. I agree that the docs, input validation, and/or error message could be improved here.

aws / aws-cli