XRAY 3.x Custom Mounts Are Improperly Coded

evannorstrand commented 4 years ago

Background: I had a need many months ago for custom CA certs to be added to the xray deployment, made the proper changes, and PR'd them into master.

We just made the upgrade to XRAY 3.x and the fixes we put in place are no long working as intended.

server:

Set up your volume to the CA config map

customVolumes: |

name:ca configMap: name: ca
Mount the volume to the certs directory using the subPath method

customVolumeMounts: |
name: ca mountPath: /etc/ssl/certs/ca.crt subPath: ca.crt

indexer:

Set up your volume to the CA config map

customVolumes: |

name:ca configMap: name: ca
Mount the volume to the certs directory using the subPath method

customVolumeMounts: |
name: ca mountPath: /etc/ssl/certs/ca.crt subPath: ca.crt

The above code originally mounted the custom CA into both the indexer and the server. The above logic has been replaced with just

server:

Set up your volume to the CA config map

customVolumes: |

name:ca configMap: name: ca
Mount the volume to the certs directory using the subPath method

customVolumeMounts: |
name: ca mountPath: /etc/ssl/certs/ca.crt subPath: ca.crt

This is not sufficient with the addition of the router pod.

As this code was working, I am leaving this as a bug report, with the expectation of a fix being implemented in a future release. I am also submitting a ticket to associate this request with my enterprise license.

evannorstrand commented 4 years ago

@chukka @eldada

https://github.com/jfrog/charts/commit/0d1d87c729b286d25749df1eddc10df233691425

This commit broke the custom volume mounts as a whole.

My recommendation is to remove the logic from indexer and server completely, and move it to the router.

evannorstrand commented 4 years ago

Another comment, to avoid breaking deployments that are using the server custom mounts, you'll need to leave the value as server.CustomVolumes

evannorstrand commented 4 years ago

My Manual fix was to edit the deployment after it was released via helm, and mount the ca into the router directly. A similar fix in the yaml should resolve this ticket.

chukka commented 4 years ago

@evannorstrand Sorry for delayed response .

Can you please share the following things to investigate further

chart version of xray (3.x) you upgraded (is it 2.x to 3.x upgrade of xray ? or fresh install)
If it is 3.x to 3.x Xray upgrade (please share chart versions)

evannorstrand commented 4 years ago

Fresh install

chukka commented 4 years ago

@evannorstrand We understand this is a breaking change. We thought about it and decided to move in this direction .

For 3.x to 3.x (breaking ) upgrades :

Please run kubectl delete statefulsets <old_statefulset_xray_name> and run helm upgrade

I'm marking this issue as closed. Feel free to reopen if it doesn't work

evannorstrand commented 4 years ago

Chukka, this is an issue that MULTIPLE of your clients are having. Deleting and reinstalling won't fix the problem.

It's an issue with the logic of the helm charts.

chukka commented 4 years ago

@evannorstrand Fix for broken customvolumeMounts has been released in latest chart version xray-3.4.1

jfrog / charts