using manifest-tool in Github actions

[geoffrey-vl]

I've successfully setup a Github Actions CI/CD pipeline, that builds my firmware. Next I want to auto upload the firmware and manifest to Pelion, so I started adding the manifest-tool to the pipeline. I've was successful in installing the manifest-tool from source, and the tool runs. As a side note, I'm using an older version of the manifest-tool (1.5.2) since I have that one on my desktop working for few years now, combined with python 2.7.

While on my desktop the manifest-tool runs fine on Windows and WSL(Ubuntu), I'm having trouble using the manifest-tool in a Github Workflow runner based upon ubuntu. Underneath is a glimpse on how I did define that manifest part. Note that I'm storing the signing certificate and key in Github secrets.

- name: Upload to Pelion
              env:
                  SIGN_CERT_PK: ${{ secrets.MY_ORG_MANIFEST_SIGNING_CERT_PRIVATEKEY }}
                  SIGN_CERT_HEX: ${{ secrets.MY_ORG_MANIFEST_SIGNING_CERT_HEX }}
              run: |
                  echo "Restoring Manifest Signing certificate"
                  echo $SIGN_CERT_HEX | xxd -r -ps > firmware-integrity-certificate.der
                  echo $SIGN_CERT_PK > firmware-integrity-certificate.key

                  # do some stuff such as uploading firmware to Pelion and retrieving the download URL into $FWURL

                  echo "Creating manifest..."
                  manifest-tool create -k firmware-integrity-certificate.key -p firmware.bin -u $FWURL -i manifest.json -o manifest.bin

The log:

Creating manifest...
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/2.7.18/x64/bin/manifest-tool", line 30, in <module>
    sys.exit(main())
  File "/opt/hostedtoolcache/Python/2.7.18/x64/bin/manifest-tool", line 27, in main
    return clidriver.main()
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/manifesttool/clidriver.py", line 33, in main
    return driver.main()
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/manifesttool/clidriver.py", line 63, in main
    }[self.options.action](self.options) or 0
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/manifesttool/create.py", line 52, in main
    return create(options, manifestInput)
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/manifesttool/v1/create.py", line 810, in main
    manifest = create_signed_resource(options, manifestInput)
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/manifesttool/v1/create.py", line 761, in create_signed_resource
    signature = get_signature(options, manifestInput, enc_data = resource_encoded)
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/manifesttool/v1/create.py", line 654, in get_signature
    sk = ecdsa.SigningKey.from_pem(options.private_key.read())
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/ecdsa/keys.py", line 930, in from_pem
    valid_curve_encodings,
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/ecdsa/keys.py", line 996, in from_der
    s, empty = der.remove_sequence(s)
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/ecdsa/der.py", line 147, in remove_sequence
    raise UnexpectedDER("Empty string does not encode a sequence")
ecdsa.der.UnexpectedDER: Empty string does not encode a sequence

The error state as if the private key is empty. However when I try to print the private key the github shell prevents my from exposing the secret in every single way. I've even tried to manually tier appart the .key file and compose it again using echo but GH every times detect that I'm willing to expose some secret and hides using using **** **** ****. In the end I'm only guessing that the .key file is correctly restored, but I can't prove it. How checked it multiple times on my desktop and I see no problem there. The fact that GH is hiding the output whenever and however I'm reading the .key file makes my think the content is OK.

So now I'm guessing maybe the security shell also prevents the manifest-tool from running well. Any idea what could be wrong?

[teetak01]

@moshe-shahar could you check this issue?

[geoffrey-vl]

As an update, I've found that adding the key directly into the repo (for testing purposes only, I'll remove it afterwards by overriding the history) gives no issues. So it seems that the stage of storing the encrypted secret from the environment into a file is the root of all evil. Still digging further for a workaround, but as for now it doesn't seem to be related to manifest-tool in any way.

[teetak01]

@geoffrey-vl I understood that you have some older system you are running. It would still recommend moving to python3 and latest manifest-tool (v2.2.0) if possible. For older devices, the manifest-tool v2.0.0 has the manifest-header -v1 commands (like create-v1) to support the older update client implementation.

Latest device management client releases (v4.8.0 and later) use the manifest header -v3 by default, which is also the default for manifest-tool.

[geoffrey-vl]

As a final solution I'm following the github instructions for storing large secrets that involves using GPG to encrypt files within the repo. Even though the PK isn't too large to store directly within a GH-secret, Github is apparently highly protecting any way of exposing the secrets even if the users is willing to write them into a file. In the end that is a good thing, I just hoped the docs from GH were a bit clearer on what is and isn't allowed. Case closed.

I understood that you have some older system you are running. It would still recommend moving to python3 and latest manifest-tool (v2.2.0) if possible. For older devices, the manifest-tool v2.0.0 has the manifest-header -v1 commands (like create-v1) to support the older update client implementation.

Thanks, apart from python3 support what would be the reason the upgrade a working tool set?

[moshe-shahar]

Hi @geoffrey-vl, Check the changelog for the list of changes that come with each release. In short, from v1.5.2 to v2.2.0, we changed the tool architecture and interface, introduced manifest schema v3, and fixed stability issues.