Can not decrypt 'EncryptedInnertubeResponsePart' follow the readme

[Maasea]

Hello, I have been trying to decrypt the EncryptedInnertubeResponsePart data following the guidelines in initplayback.md.

Here are the steps I took:

1. I first deserialized the ump file and separated theEncryptedInnertubeResponsePart according to the OnesieHeaderType.
2. I used the EncryptedInnertubeResponsePart.proto file and generated the corresponding serialization code.
3. I retrieved the Client_key from the /config file.
4. I wrote a decrypt method following the encrypt_request method, but I found that the signature verification failed, and the decrypted values could not be decompressed using GZIP and BROTLI.

Relevant Data:：

1. The EncryptedInnertubeResponsePart that I extracted

   EncryptedInnertubeResponsePart {
   encryptedContent: <Buffer 5e af b0 f2 f5 3d c3 2f 44 11 5b ed 7f 79 03 28 0e 24 2b d4 66 bf 58 77 48 9c 00 b7 b7 8f 74 4e 6e a7 37 7d 64 f6 10 5b 3f 29 f4 14 c2 16 7c ec bd 9f ... 197587 more bytes>,
   hmac: <Buffer b0 ff 2c cf ad e5 56 4c 00 1f 51 c5 43 b7 db 4d 8f b2 a1 60 64 bd 3a 7f 32 91 69 4c 7f 2a b9 a4>,
   iv: <Buffer 7f e0 7c b6 76 f1 8e 2e ea 74 c9 9d 34 59 a8 57>,
   compressionAlgorithm: 1
   }

2. The client_key (sanitized) obtained from the config file.

   <Buffer 2a ?? 98 9d 68 cb cb f2 fd 29 ?? d0 04 a3 0f 4d 5b 02 ca 18 0e 35 ?? e7 f5 cf 83 61 0d ?? ?? ??>

3. The decryption and verification functions I used, along with the method of calling them, but it always throws an exception.

   
   function decryptAndVerify (client_key, encryptedData, hmacSignature, iv) {
   const aesKey = client_key.slice(0, 16)
   const hmacKey = client_key.slice(16, 32)
   
   const hmac = crypto.createHmac('sha256', hmacKey)
   hmac.update(encryptedData)
   hmac.update(iv)
   const calculatedHmac = hmac.digest()
   
   if (!crypto.timingSafeEqual(calculatedHmac, hmacSignature)) {
   throw new Error('HMAC verification failed')
   }
   
   const decipher = crypto.createDecipheriv('aes-128-ctr', aesKey, iv)
   return Buffer.concat([decipher.update(encryptedData), decipher.final()])
   }

const message = EncryptedInnertubeResponsePart.fromBinary(raw) const config = Config.fromBinary(configRaw) const clientKey = config.content.f16.f7.f138536474.onesieHotConfig.clientKey

const encryptedData = message.encryptedContent const hmac = message.hmac const iv = message.iv const decryptedData = decryptAndVerify(clientKey, encryptedData, hmac, iv)

[davidzeng0]

From the wording of your issue, I'm guessing that you are trying to decrypt the response that you got after setting up a MITM for the YouTube app. This likely means that your key is incorrect. Also note that the servers send back a compression_algorithm field regardless of whether you set enable_compression to true or false in the EncryptedInnertubeRequest. If enable_compression is not set to true, do not decompress regardless of the algorithm field being set.

As for the keys, it's worth checking the other requests to see if there is a field global_config_group within ResponseContext and use the latest key you find, and try to ensure the encrypted_client_key matches before decrypting.

What confuses me is why you have a proto file named Config rather than ConfigRequest or ConfigResponse. I'm assuming this is ConfigResponse, but I'm not aware of any field content

[Maasea]

Yes, I use MITM to obtain these data.

To add to the previous information:

I obtained the configResponse from the https://youtubei.googleapis.com/youtubei/v1/config , and I extracted the client_key using the following path:

configResponse.responseContext.globalConfigGroup.hotConfigGroup.mediaHotConfig.onesieHotConfig.clientKey

Since the config request is not always sent, I retrieve the configResponse by reinstalling the Youtube App when needed. I am not sure if this method ensures that I get the most up-to-date client_key, as I found that the key I just obtained is the same as yesterday's. I noticed a field keyExpiresInSeconds: 259200, does this mean the key is valid for 72 hours?

I am not clear on how the encrypted_client_key is generated, but I compared the OnesieRequestProto.EncryptedInnertubeRequest.encrypted_client_key in the initPlayback request with the onesieHotConfig.encrypted_client_key in the Config response, and they are not the same.

Additionally, I parsed the UMP, but I was unable to play the video files directly as mentioned by davidzeng0/UMP_Format. The parsing process should be correct, as during the MITM process, I used the defined Reeder method to read, and the Writer method to write back the UMP, after which the Youtube App was able to play the video using the modified response body.

[davidzeng0]

I found that the key I just obtained is the same as yesterday's

A wipe of the app's data may be required

I noticed a field keyExpiresInSeconds: 259200, does this mean the key is valid for 72 hours?

Yes

I am not clear on how the encrypted_client_key is generated

It's in the OnesieHotConfig

I was unable to play the video files directly as mentioned

ONESIE_ENCRYPTED_MEDIA must be decrypted, and the header id (generally the first byte) must be stripped from the rest of the media data

[Maasea]

Thank you for your help.

[davidzeng0]

Were you successful in decrypting the response? Sometimes the key can from log_event and other innertube responses

You might also want to see https://github.com/davidzeng0/innertube/commit/5ce2241cbfc130ead5d31e3d4a64f0704c34c8cf

[Maasea]

Were you successful in decrypting the response?

No

I attempted to locate the Client_key in the response.ResponseContext from URLs such as https://youtubei.googleapis.com/youtubei/v1/(browse|get_watch), but most of what I found was tracking data, and I did not obtain the Client_key I was looking for.

Additionally, following the hints from earlier, I separated out the MEDIA type headerId. I discovered that initPlayback contains multiple videoIds (which I suspect might be advertisements?), and the arrangement of MEDIA data based on headerId appears to be chaotic.

I am quite puzzled by the initplayback data I analyzed (ignoring other OnesieHeaderType):

Case1:
MEDIA-HEAD (headerId=0, videoId=abc)
MEDIA (headerId=0)
MEDIA-END

Case:2
MEDIA-HEAD(headerId=1, videoId=efg)
MEDIA (headerId=1)
MEDIA (headerId=0)
MEDIA (headerId=1)
MEDIA-END

Case3
MEDIA-HEAD(headerId=2,videoId=efg)
MEDIA (headerId=2)
....

I believe MEDIA-HEAD~MEDIA-END should represent a segment, and my confusion lies in:

1. The appearance of different headerIds within the segment identified by headerId 1, as seen in cases 1 and 2.
2. The same videoId being associated with different headerIds, as in cases 2 and 3.

Although I saw two possible mp4 file identifiers in the file's hex, I still cannot directly play the files separated by headerId.

Moreover, I do not understand the role of headerId, nor do I know how to assemble the files from the response due to this strange arrangement.

[davidzeng0]

You might be interested in setting a fixed response, to log_event for example, and inserting the keys you want to use in there. If the config is set in any ResponseContext it gets updated for the whole app.

The MEDIA's headerId corresponds to a MediaHeader with the same headerId. That's all. By checking the video id and the itag you can reassemble individual streams. You can concatenate, in order, all buffers for a specific video id and itag pair to get a playable stream. Make sure you strip the headerId from each MEDIA before concatenating

[Maasea]

inserting the keys you want to use in there

I will try this. I plan to insert the keys fromconfigResponse into log_event.

Additionally, do you know how the client_key is used to generate the encrypt_client_key? What is the purpose of the encrypt_client_key field?

The merging of video streams is not as appealing to me because I am primarily interested in removing ad videos through MITM. However, I have noticed that the data returned for ad videos and regular videos are essentially identical. It might be necessary to decrypt the EncryptedInnertubeResponsePart to obtain more details.

[davidzeng0]

The encrypt_client_key is generated by google servers and returned in a OnesieHotConfig. It's paired with the client_key. Since the client_key isn't sent in the request, the encrypted_client_key is used by the backend to decrypt the request. You can imagine the backend decrypts the encrypted_client_key, which gets the original client_key, so now the request can be decrypted. Neither field is generated by the client.

[Maasea]

I encountered an interesting situation.

After inserting the client_key and encrypted_client_key obtained from configResponse into responseContext.globalConfigGroup.hotConfigGroup.mediaHotConfig.onesieHotConfig, the YouTube App no longer retrieves videos through the InitPlayback interface, but instead reverts to the older Player interface.

It's worth noting that I injected the data into browse rather than log_event, for convenience reasons. I might try injecting into log_event at a later time.

I speculate there could be two possible reasons:

1. There was an issue with the injected client_key and encrypted_client_key, causing the YouTube App to fail in constructing the EncryptedInnertubeRequest and thus reverting to the older Player interface.
2. I only injectedclient_key and encrypted_client_key, but onesieHotConfig actually contains many settings. During protobuf deserialization, the un-injected values were read as false, which might have disabled certain client settings.

The following is a partial response body parsed by protoc --decode_raw, where globalConfigGroup is the content I injected.

1{<responseContext>
  6{...}
  6{...}
  6{...}
 16{<globalConfigGroup>
     7{<hotConfigGroup>
        138536474 {<mediaHotConfig>
           146311580 {<onesieHotConfig>
              1: "\376\266E\266\355\335\241sH\007X\245\274\371,\240\021......."  <client_key>
              2: "\000\252L\301\314+\251\007\230I\202\202P\266\0316])\020<\211....."<encrypted_client_key>
              3: 259200 <key_expires_in_seconds>
  }

Update: I checked the log_event, where I found the client_key in the response.

[davidzeng0]

OnesieHotConfig also has flags that tell the app if it should use onesie requests or not. You must inject the full hot config group at once.