Private CA Certificates in wrong format (extra tab)

tedivm commented 6 years ago

I have a PrivateCA which has a "RootCA" that is in my control and the "Intermediary CA" that is run on the AWS hardware (a pretty standard setup).

When running this command the output is (as expected) a certificate chain file-

aws acm-pca get-certificate  
  --certificate-authority-arn ""   
  --certificate-arn ""
   --output text > example.crt

The actual certificates themselves are fine, however there's an extra "tab" between the end of the first certificate and the start of the second one.

This tab is consistently there with all of the certificates I sign and retrieve, and it breaks the certificate chain formatting. While this bug persists it's impossible to get a working chain file directly from the AWS CLI- instead the output has to be filtered to remove the random tab or applications like nginx will choke on the certificate chain.

justnance commented 6 years ago

@tedivm - Thanks for reaching out and for your patience. I am not able to reproduce this behavior under CLI version 1.16.21 through 1.16.24. Is this still an issue and what version is or was in use at the time. (aws --version). If you use the --debug option, there is a line starting with MainThread - botocore.parsers - DEBUG - Response body:.

Do you see any white space or an extra tab displayed after the Response body:, if so that is an indicator it is coming from the service and not the CLI.

justnance commented 5 years ago

@tedivm - I worked with our Premium Support Security team and we were able to reproduce the behavior you are seeing here. This behavior is expected when using --output text. As per our online documentation, the text format organizes the AWS CLI's output into tab-delimited lines which results in the tab when making a single call.

Another example that shows tabs is the following command:

aws kms list-keys --output text

Basically when text output is used and there are more than one Key Value pair in the JSON object, the results return a tab between the values. The CLI does not have any control over this behavior as it is only echoing the results from the API.

To eliminate the tab and still use text output, you may need to pull one value at a time as using JSON format would return invalid certificate format.

It might also be possible to use --query. similar to the following:

aws acm-pca get-certificate --certificate-authority-arn ARNHere --certificate-arn ARNHere --query 'Certificate' --output text

aws acm-pca get-certificate --certificate-authority-arn ARNHere --certificate-arn ARNHere --query 'CertificateChain' --output text

tedivm commented 5 years ago

Are you sure this is expected behavior? Literally all the documentation on this says that you're outputting a properly formatted PEM certificate chain-

This command outputs the base64 encoded PEM format certificate and the certificate chain.

https://docs.aws.amazon.com/acm-pca/latest/userguide/PcaGetCert.html

The example output on that page even shows a properly formatted (ie, no tabs) certificate chain.

This really seems like a bug. If it isn't then you should You should update the documentation to reflect that you are not actually outputting the proper PEM chain like you're advertising and displaying.

tedivm commented 5 years ago

Even with that "text output uses tab delimiters" this seems like a bug, as the tab it at the start of the line (not separating multiple fields), and there isn't a separate field available anyways.

justnance commented 5 years ago

@tedivm - Thank you for your feedback. The tab-delimited formatting is expected output for the CLI when using --output text. I have collaborated with our documentation writers to update and clarify the expected output and have escalated this issue to our ACM service team for review and improvement. Because this issue is not controlled by the CLI, follow up with the ACM service team can be done via the their forums or via a case with AWS Support.

tedivm commented 5 years ago

When is the documentation going to be updated? It's still showing it with no tab, and it's still claiming that it's outputting in PEM formatting despite you saying that it isn't the case.

tedivm commented 5 years ago

Here's a screenshot showing the claim that it follows PEM format, combined with the example showing no tab.

tedivm commented 5 years ago

I've gone ahead and opened a ticket on the forums- hopefully someone there will take this issue seriously.

bisdavid commented 5 years ago

@tedivm, I'm going to work with the writer for that service to get this addressed. We do take it seriously, it's just that there are many moving parts and it takes a while. The content for this guide is distributed across many writers and we have to coordinate the efforts. Working on the forum is the best way to get the developer team's attention to the "functionality" issue, so thanks for doing that. We will, in the interim, work on getting the docs updated to warn that the TAB will be in there and some work-arounds that will help until the dev team can update the code (if they choose to do so). Thanks for your patience.

justnance commented 5 years ago

@tedivm - I agree with your assessment that the documentation is incorrect and the service needs improvement. This issue was escalated internally and previously collaborated with @bisdavid regarding this issue. Thanks again for reporting this issue and your patience while we find the best resolution.

I will leave this issue open pending an update to our documentation.

justnance commented 5 years ago

@tedivm - Thanks again for reporting this issue. Our online documentation has been updated and the page with the incorrect information has been removed. Closing this issue.

Thank you for your patience.

tedivm commented 5 years ago

Your documentation is still wrong.

If you go to the get-certificate documentation that is up right now it says-

The following get-certificate command retrieves a base64 encoded PEM format certificate.

But according to you that is no longer- and has never been- the case.

justnance commented 5 years ago

@tedivm - Thanks for the update. I've collaborated with our CLI reference documentation writers and we are investigating on how to improve this documentation. Thanks for your patience.

kdaily commented 4 years ago

This looks like it has been addressed:

https://docs.aws.amazon.com/cli/latest/reference/acm-pca/get-certificate.html#examples

tedivm commented 4 years ago

I still can't believe you took a bug report, said "no that's a feature", and then updated the documentation instead of the actual bug- but at least the documentation is consistent now.

kdaily commented 4 years ago

Hi @tedivm,

I appreciate the feedback. I think the initial issue has been addressed by fixing the documentation. However, I can see how this is a frustrating experience. While the behavior of text output is as described and the documentation has been reflected, the example using text output may not be the best. The related command aws acm export-certificate has a similar structure, but slightly different behavior with respect to newlines after the certificate text. Straight away the guidance here uses JSON output piped to jq.

The export-certificate example is:

aws acm export-certificate \
--certificate-arn arn:aws:acm:region:account:certificate/12345678-1234-1234-1234-123456789012 \
--passphrase fileb://path-to-passphrase-file  \
| jq -r '"\(.Certificate)\(.CertificateChain)\(.PrivateKey)"'

I confirmed that I can do something similar with get-certificate (note I had to force a new line in between):

aws acm-pca get-certificate \
--certificate-authority-arn arn:aws:acm-pca:us-west-2:ZZZZZZZZZZZZ:certificate-authority/..... \
--certificate-arn arn:aws:acm-pca:us-west-2:ZZZZZZZZZZZZZ:certificate-authority/../certificate/........ \
| jq -r '"\(.Certificate)\n\(.CertificateChain)"'

I will follow up with the team that maintains this API and see what can be done to have consistency in the output and the examples, but cannot provide an ETA on getting it addressed.

thetoolsmith commented 2 years ago

@kdaily That get-certificate cli command piping to jq does not work on it's own for our case. We are using the file for importing using "import-certificate-authority-certificate", it still throws the same invalid format error when done as a single file. What we had to do based on AWS support recommendation, was to make use of the --certificate and --certificate-chain params and import the two separated files in the same call.

aws / aws-cli

Private CA Certificates in wrong format (extra tab) #3465