deislabs / mystikos

Tools and runtime for launching unmodified container images in Trusted Execution Environments
142 stars 49 forks source link

Confidential ML python app fails but returns 0 #1486

Open salsal97 opened 1 year ago

salsal97 commented 1 year ago

Images in the confml sample were corrupted and hence it was failing, but it was not propagating an error code and failing silently. Eg: https://openenclaveci.westus.cloudapp.azure.com/blue/organizations/jenkins/Mystikos%2FStandalone-Pipelines%2FSolutions-Tests-Pipeline/detail/Solutions-Tests-Pipeline/2994/pipeline

Main issue: The client sends the corrupted image as a payload to the server, which fails out. The issue is that the server is a background process and does not have a check on it.

Fixes tried (unsuccessfully):

  1. Checking the exit code for the sample and client binaries.
  2. Putting an exception error and exit 1 in the inference_service.py file (server)
  3. Checking the status of the pid of the server process before killing it

Note: Corrupted images have now been fixed, but this limitation persists.