search

Azure-Samples / iot-edge-opc-plc

Sample OPC UA server with nodes that generate random and increasing data, anomalies and much more ...
MIT License
218 stars 94 forks source link

Use cert-manager issued certificate and run in Kubernetes #360

Closed jlian closed 3 months ago

jlian commented 3 months ago

Please provide us with the following information:

This issue is for a: (mark with an x)

- [ ] bug report -> please search issues before submitting
- [ ] feature request
- [x] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Install cert-manager and trust-manager in a Kubernetes cluster, then create a self-signed CA Issuer and trust bundle, then deploy this YAML in attempt to use a cert-manager issued certificate for the OPC PLC 

apiVersion: apps/v1
kind: Deployment
metadata:
  name: opc-plc-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: opc-plc
  template:
    metadata:
      labels:
        app: opc-plc
    spec:
      containers:
      - name: opc-plc
        image: mcr.microsoft.com/iotedge/opc-plc:latest
        args: ["--at=FlatDirectory", "--drurs", "--ll-debug"]
        ports:
        - containerPort: 50000
        volumeMounts:
          - name: opc-plc-tls
            mountPath: /app/pki/own
          - name: trust-bundle
            mountPath: /app/pki/trusted
      volumes:
        - name: opc-plc-tls
          secret:
            secretName: opc-plc-tls
        - name: trust-bundle
          configMap:
            name: aio-bundle
---
apiVersion: v1
kind: Service
metadata:
  name: opc-plc-service
spec:
  selector:
    app: opc-plc
  ports:
    - protocol: TCP
      port: 80
      targetPort: 50000
  type: ClusterIP
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: opc-plc-certificate
spec:
  secretName: opc-plc-tls
  duration: 2160h # 90d
  renewBefore: 360h # 15d
  issuerRef:
    name: aio-ca-issuer
    kind: ClusterIssuer
  commonName: OpcPlc
  dnsNames:
  - opcplc
  - opc-plc-service.default.svc.cluster.local
  - opc-plc-service
  - opc-plc-service.default
  usages:
  - digital signature
  - key encipherment
  - server auth
  - client auth
  privateKey:
    algorithm: RSA
    size: 2048
  encodeUsagesInRequest: true
  isCA: false

Any log messages given by the failure

<6>2024-05-31T03:38:26.063Z - Application Certificate store type is: FlatDirectory                                                       
<6>2024-05-31T03:38:26.063Z - Application Certificate store path is: FlatDirectory:pki/own                                               
<6>2024-05-31T03:38:26.063Z - Rejection of SHA1 signed certificates is disabled                                                          
<6>2024-05-31T03:38:26.063Z - Minimum certificate key size set to 1024                                                                   
<6>2024-05-31T03:38:26.063Z - Trusted Issuer store type is: FlatDirectory                                                                
<6>2024-05-31T03:38:26.063Z - Trusted Issuer Certificate store path is: FlatDirectory:pki/issuer                                         
<6>2024-05-31T03:38:26.063Z - Trusted Peer Certificate store type is: FlatDirectory                                                      
<6>2024-05-31T03:38:26.063Z - Trusted Peer Certificate store path is: FlatDirectory:pki/trusted                                          
<6>2024-05-31T03:38:26.063Z - Rejected certificate store type is: FlatDirectory                                                          
<6>2024-05-31T03:38:26.063Z - Rejected Certificate store path is: FlatDirectory:pki/rejected                                             
<6>2024-05-31T03:38:26.064Z - Added security policy http://opcfoundation.org/UA/SecurityPolicy#Basic256Sha256 with mode SignAndEncrypt   
<6>2024-05-31T03:38:26.064Z - Added security policy http://opcfoundation.org/UA/SecurityPolicy#Aes128_Sha256_RsaOaep with mode SignAndEnc
rypt                                                                                                                                     
<6>2024-05-31T03:38:26.064Z - Added security policy http://opcfoundation.org/UA/SecurityPolicy#Aes256_Sha256_RsaPss with mode SignAndEncr
ypt                                                                                                                                      
<6>2024-05-31T03:38:26.064Z - Added security policy http://opcfoundation.org/UA/SecurityPolicy#Basic256Sha256 with mode Sign             
<6>2024-05-31T03:38:26.064Z - Added security policy http://opcfoundation.org/UA/SecurityPolicy#Aes128_Sha256_RsaOaep with mode Sign      
<6>2024-05-31T03:38:26.064Z - Added security policy http://opcfoundation.org/UA/SecurityPolicy#Aes256_Sha256_RsaPss with mode Sign       
<6>2024-05-31T03:38:26.064Z - LDS(-ME) registration interval set to 0 ms (0 means no registration)                                       
<6>2024-05-31T03:38:26.064Z - Application certificate with thumbprint B452A4C12322C13E0D97256FE4554D293789F729 found in the application c
ertificate store                                                                                                                         
<6>2024-05-31T03:38:26.066Z - Checking application instance certificate.                                                                 
<6>2024-05-31T03:38:26.069Z - Loading private key succeeded for B452A4C12322C13E0D97256FE4554D293789F729 - CN=OpcPlc                     
<6>2024-05-31T03:38:26.070Z - Check certificate: [CN=OpcPlc] [B452A4C12322C13E0D97256FE4554D293789F729]                                  
<6>2024-05-31T03:38:26.072Z - Check application instance certificate. [CN=OpcPlc] [B452A4C12322C13E0D97256FE4554D293789F729]             
<6>2024-05-31T03:38:26.084Z - Enumerate certificates - certificate added EC55F6C50ECBDF04E3801D7C9640A3647197A1A6                        
<3>2024-05-31T03:38:26.213Z - Certificate validation failed with suppressible errors but was rejected. Reason=BadCertificateUseNotAllowed
. [CN=OpcPlc] [B452A4C12322C13E0D97256FE4554D293789F729]                                                                                 
<3>2024-05-31T03:38:26.216Z - Error validating certificate. Exception: Usage of certificate is not allowed.. Use certificate anyway?     
<3>2024-05-31T03:38:26.217Z - OPC UA server failed unexpectedly                                                                          
Opc.Ua.ServiceResultException: The certificate with subject CN=OpcPlc in the configuration is invalid.                                   
 Please update or delete the certificate from this location:                                                                             
 FlatDirectory:pki/own                                                                                                                   

   at Opc.Ua.Configuration.ApplicationInstance.CheckApplicationInstanceCertificate(Boolean silent, UInt16 minimumKeySize, UInt16 lifeTime
InMonths, CancellationToken ct)                                                                                                          
   at OpcPlc.Configuration.OpcUaAppConfigFactory.ConfigureAsync() in D:\a\1\s\src\Configuration\OpcUaAppConfigFactory.cs:line 159        
   at OpcPlc.OpcPlcServer.StartPlcServerAndSimulationAsync() in D:\a\1\s\src\OpcPlcServer.cs:line 279                                    
   at OpcPlc.OpcPlcServer.StartPlcServerAsync(CancellationToken cancellationToken) in D:\a\1\s\src\OpcPlcServer.cs:line 237              
   at OpcPlc.OpcPlcServer.StartAsync(String[] args, CancellationToken cancellationToken) in D:\a\1\s\src\OpcPlcServer.cs:line 126        
Unhandled exception. System.AggregateException: One or more errors occurred. (The certificate with subject CN=OpcPlc in the configuration
 is invalid.                                                                                                                             
 Please update or delete the certificate from this location:                                                                             
 FlatDirectory:pki/own                                                                                                                   
)                                                                                                                                        
 ---> Opc.Ua.ServiceResultException: The certificate with subject CN=OpcPlc in the configuration is invalid.                             
 Please update or delete the certificate from this location:                                                                             
 FlatDirectory:pki/own                                                                                                                   

   at Opc.Ua.Configuration.ApplicationInstance.CheckApplicationInstanceCertificate(Boolean silent, UInt16 minimumKeySize, UInt16 lifeTime
InMonths, CancellationToken ct)                                                                                                          
   at OpcPlc.Configuration.OpcUaAppConfigFactory.ConfigureAsync() in D:\a\1\s\src\Configuration\OpcUaAppConfigFactory.cs:line 159        
   at OpcPlc.OpcPlcServer.StartPlcServerAndSimulationAsync() in D:\a\1\s\src\OpcPlcServer.cs:line 279                                    
   at OpcPlc.OpcPlcServer.StartPlcServerAsync(CancellationToken cancellationToken) in D:\a\1\s\src\OpcPlcServer.cs:line 237              
   at OpcPlc.OpcPlcServer.StartAsync(String[] args, CancellationToken cancellationToken) in D:\a\1\s\src\OpcPlcServer.cs:line 126        
   --- End of inner exception stack trace ---                                                                                            
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)                                              
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)                                   
   at System.Threading.Tasks.Task.Wait()                                                                                                 
   at OpcPlc.Program.Main(String[] args) in D:\a\1\s\src\Program.cs:line 15                                                              
Stream closed EOF for default/opc-plc-deployment-cd7cfd9f6-hgvpc (opc-plc)

Expected/desired behavior

Should be able to use a cert-manager issued cert for the application certificate here.

OS and Version?

Kubernetes v1.28.8+k3s1 in k3d in a Debian machine

Versions

OpcPlc v2.12.11

Mention any other details that might be useful

I followed guidance from https://github.com/Azure-Samples/iot-edge-opc-plc/issues/348 as much as I could in the Certificate configuration.

Prior to this there were clear errors like "issuer isn't trusted" (hence the trust bundle) and "cert revocation list not found" (hence --drurs). Now the error is no longer specific, so I don't know what exactly is wrong with the certificate anymore.

Thanks! We'll be in touch soon.

jlian commented 3 months ago

Actually, I got it working! I went back to https://github.com/Azure-Samples/iot-edge-opc-plc/issues/348 and noticed that I missed some details, notably

  1. In the dnsName I should've used the name of the service, which is opc-plc-service
  2. In the args, also put the name of the service "--ph=opc-plc-service", "--cdn=opc-plc-service"

  3. Needed to have an URI subject alternative name

    uris:
 - urn:OpcPlc:opc-plc-service
  4. Needed the data encipherment key usage.

Final YAML that worked for me 

apiVersion: apps/v1
kind: Deployment
metadata:
  name: opc-plc-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: opc-plc
  template:
    metadata:
      labels:
        app: opc-plc
    spec:
      containers:
      - name: opc-plc
        image: mcr.microsoft.com/iotedge/opc-plc:latest
        args: ["--at=FlatDirectory", "--drurs", "--ll-debug", "--ph=opc-plc-service", "--cdn=opc-plc-service", "--ses"]
        ports:
        - containerPort: 50000
        volumeMounts:
          - name: opc-plc-tls
            mountPath: /app/pki/own
          - name: trust-bundle
            mountPath: /app/pki/trusted
      volumes:
        - name: opc-plc-tls
          secret:
            secretName: opc-plc-tls
        - name: trust-bundle
          configMap:
            name: aio-bundle
---
apiVersion: v1
kind: Service
metadata:
  name: opc-plc-service
spec:
  selector:
    app: opc-plc
  ports:
    - protocol: TCP
      port: 80
      targetPort: 50000
  type: ClusterIP
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: opc-plc-certificate
spec:
  secretName: opc-plc-tls
  duration: 2160h # 90d
  renewBefore: 360h # 15d
  issuerRef:
    name: aio-ca-issuer
    kind: ClusterIssuer
  commonName: OpcPlc
  dnsNames:
    - opc-plc-service
    - opc-plc-service.default.svc.cluster.local
    - opc-plc-service.default
  uris:
    - urn:OpcPlc:opc-plc-service
  usages:
    - digital signature
    - key encipherment
    - data encipherment
    - server auth
    - client auth
  privateKey:
    algorithm: RSA
    size: 2048
  encodeUsagesInRequest: true
  isCA: false