Content file commit fails - Upload state is always commitFileFailed

[Acrawford-Adesa]

Background

I am trying to create a script that will update an Intune app automatically as part of a continuous deployment pipeline. I have been replicating the steps found here so that they work in our environment.

Issue

After calling deviceAppManagement/mobileApps/${application.id}/microsoft.graph.iosLobApp/contentVersions/${contentVersion.id}/files/${contentFile.id}/commit with the appropriate variables, I call deviceAppManagement/mobileApps/${application.id}/microsoft.graph.iosLobApp/contentVersions/${contentVersion.id}/files/${contentFile.id} in a loop to wait for the file commit to be finished. The problem is that UploadState is always commitFileFailed and isCommited is always false.

More details

I think my issue may be with how I am encrypting the file or fileEncryptionInfo since the docs seem to be missing some details about those steps. I am encrypting the file using AES-256, generating the initialization vector using let iv: any = crypto.randomBytes(16); and the cipher key with crypto.createHash("sha256").update(password).digest(); where passwrod is a 16 byte long buffer. I construct the fileEncryptionInfo object used in the body of the commit post request using the following:

const macKey = crypto.randomBytes(32);
const temp = Buffer.concat([encrypted, iv]).toString();
const hmac = crypto.createHmac("sha256", macKey);
const fileDigest = crypto.createHash("sha256").update(unencryptedContent).digest("base64");

const fileEncryptionInfo = {
    fileDigestAlgorithm: "SHA256",
    encryptionKey: CIPHER_KEY.toString("base64"),
    initializationVector: iv.toString("base64"),
    fileDigest,
    mac: hmac.update(temp).digest().toString("base64"),
    profileIdentifier: "ProfileVersion1",
    macKey: macKey.toString("base64"),
};

I am not sure if this is a bug with the SDK or user error, but I have searched the docs, the issues pages, and the wider internet for more information and come up empty. I do not know why the commit is resulting in an error.

[nikithauc]

@Acrawford-Adesa Can you please provide the following -

1. Code making the request calls to the Graph API using the Graph JS Library?
2. Response from the Graph API with the request ids.

[Acrawford-Adesa]

I replaced any potentially secret fields in the responses with "fake".

Create application:

application = await graphClient
    .api("deviceAppManagement/mobileApps")
    .post(newMobileApp);

Response:

{
  '@odata.context': 'https://graph.microsoft.com/v1.0/$metadata#deviceAppManagement/mobileApps/$entity',
  '@odata.type': '#microsoft.graph.iosLobApp',
  id: '79b0345d-180b-4cb6-bb3c-ba3763ccbf7e',
  displayName: 'HiddenDeployment2-Test',
  description: 'HiddenDeployment2-Test',
  publisher: 'fake',
  largeIcon: null,
  createdDateTime: '2021-06-09T11:48:04.7328287Z',
  lastModifiedDateTime: '2021-06-09T11:48:04.7328287Z',
  isFeatured: false,
  privacyInformationUrl: null,
  informationUrl: null,
  owner: 'fake',
  developer: null,
  notes: null,
  publishingState: 'notPublished',
  committedContentVersion: null,
  fileName: 'hidden.ipa',
  size: 0,
  bundleId: 'fake',
  expirationDateTime: null,
  versionNumber: '1.0.0',
  buildNumber: '1.0.0',
  applicableDeviceType: { iPad: true, iPhoneAndIPod: true },
  minimumSupportedOperatingSystem: {
    v8_0: true,
    v9_0: false,
    v10_0: false,
    v11_0: false,
    v12_0: false,
    v13_0: false
  }
}

Create content version

const contentVersion = await graphClient
    .api(
      `deviceAppManagement/mobileApps/${application.id}/microsoft.graph.iosLobApp/contentVersions`
    )
    .post({
      "@odata.type": "#microsoft.graph.mobileAppContent",
    });

Response:

Content version {
  '@odata.context': "https://graph.microsoft.com/v1.0/$metadata#deviceAppManagement/mobileApps('fake')/microsoft.graph.iosLobApp/contentVersions/$entity",
  id: '1'
}

Add content file

const unencryptedContent: Buffer = fs.readFileSync(filename);
  const encrypted = encrypt(unencryptedContent);

  const fileMetadata = {
    sizeEncrypted: encrypted.length,
    name: filename,
    size: unencryptedContent.length,
    "@odata.type": "#microsoft.graph.mobileAppContentFile",
  };

  const contentFile = await graphClient
    .api(
      `deviceAppManagement/mobileApps/${application.id}/microsoft.graph.iosLobApp/contentVersions/${contentVersion.id}/files`
    )
    .post(fileMetadata);

Response:

{
  '@odata.context': "https://graph.microsoft.com/v1.0/$metadata#deviceAppManagement/mobileApps('fake')/microsoft.graph.iosLobApp/contentVersions('1')/files/$entity",
  azureStorageUri: null,
  isCommitted: false,
  id: '28caed19-ca7d-4492-8ac3-63d7c241a0e6',
  createdDateTime: '0001-01-01T00:00:00Z',
  name: 'hidden.ipa',
  size: 75716629,
  sizeEncrypted: 75716656,
  azureStorageUriExpirationDateTime: null,
  manifest: 'fake',
  uploadState: 'azureStorageUriRequestPending'
}

Upload encrypted file

 const blockBlobClient = new BlockBlobClient(azureStorageUri, newPipeline());
  const uploadBlobResponse = await blockBlobClient.upload(
    encrypted,
    encrypted.length
  );

Wait for file processing and get storage url

while (!azureStorageUri) {
     updatedContentFile = await graphClient
      .api(
        `deviceAppManagement/mobileApps/${application.id}/microsoft.graph.iosLobApp/contentVersions/${contentVersion.id}/files/${contentFile.id}`
      )
      .get();
    azureStorageUri = updatedContentFile.azureStorageUri;
  }

Response:

{
  '@odata.context': "https://graph.microsoft.com/v1.0/$metadata#deviceAppManagement/mobileApps('fake')/microsoft.graph.iosLobApp/contentVersions('1')/files/$entity",
  azureStorageUri: 'fake',
  isCommitted: false,
  id: '28caed19-ca7d-4492-8ac3-63d7c241a0e6',
  createdDateTime: '0001-01-01T00:00:00Z',
  name: 'hidden.ipa',
  size: 75716629,
  sizeEncrypted: 75716656,
  azureStorageUriExpirationDateTime: '2021-06-09T12:03:08.0187611Z',
  manifest: 'hidden',
  uploadState: 'azureStorageUriRequestSuccess'
}

Commit file encryption info

 await graphClient
    .api(
      `deviceAppManagement/mobileApps/${application.id}/microsoft.graph.iosLobApp/contentVersions/${contentVersion.id}/files/${contentFile.id}/commit`
    )
    .post({
      fileEncryptionInfo,
    });

Response:

PassThrough {
  _readableState: ReadableState {
    objectMode: false,
    highWaterMark: 16384,
    buffer: BufferList { head: null, tail: null, length: 0 },
    length: 0,
    pipes: [],
    flowing: null,
    ended: false,
    endEmitted: false,
    reading: false,
    sync: false,
    needReadable: false,
    emittedReadable: false,
    readableListening: false,
    resumeScheduled: false,
    errorEmitted: false,
    emitClose: true,
    autoDestroy: true,
    destroyed: false,
    errored: null,
    closed: false,
    closeEmitted: false,
    defaultEncoding: 'utf8',
    awaitDrainWriters: null,
    multiAwaitDrain: false,
    readingMore: false,
    decoder: null,
    encoding: null,
    [Symbol(kPaused)]: null
  },
  _events: [Object: null prototype] {
    prefinish: [Function: prefinish],
    unpipe: [Function: onunpipe],
    error: [ [Function: onerror], [Function (anonymous)] ],
    close: [Function: bound onceWrapper] { listener: [Function: onclose] },
    finish: [Function: bound onceWrapper] { listener: [Function: onfinish] }
  },
  _eventsCount: 5,
  _maxListeners: undefined,
  _writableState: WritableState {
    objectMode: false,
    highWaterMark: 16384,
    finalCalled: false,
    needDrain: false,
    ending: false,
    ended: false,
    finished: false,
    destroyed: false,
    decodeStrings: true,
    defaultEncoding: 'utf8',
    length: 0,
    writing: false,
    corked: 0,
    sync: true,
    bufferProcessing: false,
    onwrite: [Function: bound onwrite],
    writecb: null,
    writelen: 0,
    afterWriteTickInfo: null,
    buffered: [],
    bufferedIndex: 0,
    allBuffers: true,
    allNoop: true,
    pendingcb: 0,
    prefinished: false,
    errorEmitted: false,
    emitClose: true,
    autoDestroy: true,
    errored: null,
    closed: false
  },
  allowHalfOpen: true,
  [Symbol(kCapture)]: false,
  [Symbol(kTransformState)]: {
    afterTransform: [Function: bound afterTransform],
    needTransform: false,
    transforming: false,
    writecb: null,
    writechunk: null,
    writeencoding: null
  }
}

Wait till file is commited

let isCommitted = false;
  let uploadState = "";
  while (!isCommitted && uploadState !== "commitFileFailed") {
    const updatedApp = await graphClient
      .api(
        `deviceAppManagement/mobileApps/${application.id}/microsoft.graph.iosLobApp/contentVersions/${contentVersion.id}/files/${contentFile.id}`
      )
      .get();
    console.log(updatedApp);
    isCommitted = updatedApp.isCommitted;
    uploadState = updatedApp.uploadState;
  }
  if (uploadState === "commitFileFailed") {
    console.error("Commit failed");
  }

Response

 '@odata.context': "https://graph.microsoft.com/v1.0/$metadata#deviceAppManagement/mobileApps('fake')/microsoft.graph.iosLobApp/contentVersions('1')/files/$entity",
  azureStorageUri: 'fake',
  isCommitted: false,
  id: '28caed19-ca7d-4492-8ac3-63d7c241a0e6',
  createdDateTime: '0001-01-01T00:00:00Z',
  name: 'hidden.ipa',
  size: 75716629,
  sizeEncrypted: 75716656,
  azureStorageUriExpirationDateTime: '2021-06-09T12:03:08.0187611Z',
  manifest: 'hidden',
  uploadState: 'commitFileFailed'
}

[nikithauc]

@jaiprakashmb Can you please verify this issue?

[jaiprakashmb]

@Acrawford-Adesa , From above thread details there seems to be issue in the way you are uploading the file. Above Response does indicates incorrect encryptedfile size is being passed (75716656).

Ideally it should be 48 + 16*(Floor(75716629 / 16) + 1) = 75716688 (HMAC hash + Initialization vector + AES CBC 256 encrypted content).

Also from code snippet you have share it is bit difficult to pin point the issues. As a suggestion in case you can share source code where were you are performing file upload operations than it will help. Can you also confirm if you are doing anything different from the sample/ example code take for reference?

[benbitrise]

@jaiprakashmb thank you for your comment on this thread. I am experiencing the same error as the original poster (commitFileFailed). I am attempting to follow the powershell sample code. I must re-write this in Js to run in a CI/CD environment that doesn't support powershell.

Below is the encryption code and upload code I'm using. Anything you can see here that I'm doing wrong?

const originalFileBuffer = fs.readFileSync(originalPath);

const key = crypto.randomBytes(32);
const iv = crypto.randomBytes(16);
const encryptor = crypto.createCipheriv('aes-256-cbc', key, iv);

let encrypted = encryptor.update(originalFileBuffer);
encrypted = Buffer.concat([iv, encrypted, encryptor.final()])

const hmacKey = crypto.randomBytes(32);
const hmac = crypto.createHmac('sha256', hmacKey);
hmac.update(encrypted)
const hmacHash = hmac.digest();

let encryptedWithHmac = Buffer.concat([hmacHash, encrypted])
fs.writeFileSync(encryptedPath, encryptedWithHmac)

const encryptionInfo = {
        encryptionKey: key.toString('base64'),
        macKey: hmacKey.toString('base64'),
        initializationVector: iv.toString('base64'),
        mac: hmacHash.toString('base64'),
        profileIdentifier: 'ProfileVersion1',
        fileDigest: crypto.createHash('sha256').update(originalFileBuffer).digest('base64'),
        fileDigestAlgorithm: 'SHA256'
};

console.log("***ENCRYPTION INFO***")
console.log(encryptionInfo)

const originalSize = fs.statSync(originalPath).size;
const encryptedSize =  fs.statSync(encryptedPath).size;

console.log(`Original size ${originalSize} and encrypted size ${encryptedSize}`)

Log output: Original size 40331844 and encrypted size 40331904

const { newPipeline, BlockBlobClient } = require("@azure/storage-blob")

...

const blockBlobClient = new BlockBlobClient(azureStorageUri, newPipeline())
await blockBlobClient.upload(encryptedFileBuffer, encryptedFileBuffer.length);

[petrhollayms]

Thank you for reporting this issue. This appears to be an issue or limitation with the service APIs. Unfortunately, as the Microsoft Graph SDK team, we do not have ownership of the APIs that are causing you issues. We invite you to create a question about the service API to Microsoft Q&A and tagged with one of the [microsoft-graph-*] tags, that way it will get routed to the appropriate team for them to triage:

https://aka.ms/msgraphsupport or directly https://aka.ms/askgraph

For now, we will close the issue on our side but feel free to open it in the relevant repository if you think the issue is specific to SDK. Please let us know if this helps!

Note: We will close this repository on April 19, 2024.