borisbabic / browser_cookie3

This is a fork of browser_cookie
GNU Lesser General Public License v3.0
856 stars 144 forks source link

Broken decryption in Chrome #210

Open dr-DNR opened 2 months ago

dr-DNR commented 2 months ago

I have a script that has been running fine over the last couple of years, but has suddenly stopped tonight (20240907). This is the error that I got

""" Error message: Unable to get key for cookie decryption Traceback (most recent call last): File "browsercookie3_init.py", line 539, in _decrypt File "browsercookie3_init.py", line 530, in _decrypt_windows_chromium File "browsercookie3_init.py", line 89, in _crypt_unprotect_data RuntimeError: Failed to decrypt the cipher text with DPAPI

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "browsercookie3_init.py", line 555, in _decrypt File "Cryptodome\Cipher_mode_gcm.py", line 567, in decrypt_and_verify File "Cryptodome\Cipher_mode_gcm.py", line 508, in verify ValueError: MAC check failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "", line 918, in File "", line 821, in main File "", line 256, in simulation_dependent_variables File "browsercookie3_init.py", line 1160, in chrome File "browsercookie3_init.py", line 515, in load File "browsercookie3_init.py", line 557, in _decrypt browser_cookie3.BrowserCookieError: Unable to get key for cookie decryption """

I had to previously implement the --disable-features=LockProfileCookieDatabase in my chrome shortcut, but other than that I have had no issues. I am currently running browser-cookie3==0.19.1 in python 3.10.12. I am guessing that it's probably a chrome update that lead to this error.

Thank you, and I appreciate all the help that you can provide.

timothy-miron commented 2 months ago

I'm just starting to sleuth around the code to find out what actually broke, and one thing I've noticed is that if one looks at what the 3 characters getting chopped off around this line of code, the error that's being encountered happens when the program finds a cookie that does not start with v10 but rather starts with v20 -- you can quickly test this by adding a print statement BEFORE this line, like print(f'first 3 chars: {encrypted_value[:3]}').
... you'll see it spit out

first 3 chars: v10
first 3 chars: v10
first 3 chars: v10
...
...
first 3 chars: v20

.. and then immediately hit the exception ..

I suspect that a new encryption scheme has been adopted, and the 'fix' will be to understand what needs to be handled differently, and to branch the logic depending on whether it's v10 vs. v20.

timothy-miron commented 2 months ago

Further sleuthing.. Seems that v20 prefix is indicative of 'App Bound Encryption' at least from what I can make out from a cursory look at this module in the Chromium code base (link to actual line where v20 is defined as such a prefix), and this sort of lines up with this recent announcement about Chrome beefing up its security against scripts being able to access Cookies and impersonate sessions, etc. by employing app-bound encryption (link to article).

In summary, I have no answer for how to get around this, save for potentially disabling 'Application Bound Encryption' in Chrome (likely something not recommended but possible). Perhaps someone smarter than me can develop a patch, but I suspect it would be an extremely large lift to get this "fixed".

dr-DNR commented 2 months ago

Thanks for all the information and help. I would be up for developing such as patch, but alas, I am not that bright. I am going to switch to firefox for the the time being and might consider switching back if there are any new developments. I am going to leave this open in case someone else ends up having the same issue, or is able to do something about it.

trikosm186 commented 2 months ago

After I updated Chrome earlier today (128.0.6613.120) I have exactly the same problem. Unfortunately I don't have the knowledge to look more into it. Searched a bit about app-bound encryption but I can't find "Software\Policies\Google\Chrome\ApplicationBoundEncryptionEnabled" in my PC's (Win10) registry.

dr-DNR commented 2 months ago

The solution will probably be something similar to --disable-features=LockProfileCookieDatabase specific to Chrome, not Windows, but I might be wrong

uu7157 commented 2 months ago

This happened after the latest chrome update. Likely related issue

specman333 commented 2 months ago

After I updated Chrome earlier today (128.0.6613.120) I have exactly the same problem. Unfortunately I don't have the knowledge to look more into it. Searched a bit about app-bound encryption but I can't find "Software\Policies\Google\Chrome\ApplicationBoundEncryptionEnabled" in my PC's (Win10) registry.

I encounter the same problem with Chrome version 128.0.6613.120: " browser_cookie3.BrowserCookieError: Unable to get key for cookie decryption". I guess the latest version Chrome change some things for cookie decryption.

specman333 commented 2 months ago

Chrome 128 Release notes:

Cross-site ancestor chain bit for CookiePartitionKey of partitioned cookies

Chrome 128 adds a cross-site ancestor bit to the key ring of the partitioned cookie's CookiePartitionKey. This change unifies the partition key with the partition key values used in storage partitioning and adds protection against clickjacking attacks by preventing cross-site embedded frames from having access to the top-level-site's partitioned cookies.

I think this maybe lead to browser_cookie3 error in getting Chrome 128's Cookie.

Iansus commented 2 months ago

Any update on this? Is this somehow patchable or we're lacking information on what exactly changed?

9DA73860 commented 2 months ago

in the meantime I use rookiepy in requests

import rookiepy

rcookies = rookiepy.chrome()
cookies = rookiepy.to_cookiejar(rcookies)
mic-user commented 2 months ago

@9DA73860 Your solution only works with 'v10' (which is DPAPI before app bound). So your reply totally mis-point out. @timothy-miron is pointing right matters which I also found and strugling. If you feel working find with the package for specific cookies, the cookie your are accessing is not expired after created by v10. Once it expired and refreshed on newer chrome version, will be face the/this/same matter. For now, no published open source solution for app-bounded 'v20' encryption at this moment(which means all open source people screaming out all around globe including me).

By the way, I found, for the app bounded encryption, there is new key for this. ` local_state_path = os.path.join(os.environ["USERPROFILE"], "AppData", "Local", "Google", "Chrome", "User Data", "Local State") with open(local_state_path, "r", encoding="utf-8") as f: local_state = f.read() local_state = json.loads(local_state)

    # decode the encryption key from Base64
    _key_legacy = base64.b64decode(local_state["os_crypt"]["encrypted_key"])
    _key_app_bound_encrypted_key = base64.b64decode(local_state["os_crypt"]["app_bound_encrypted_key"])`

With this, any one can decrypt the for v20 encryption like below?

` def _decrypt_data(self, data, key): if data == bytearray(b''): return ''

    if data[:2] == b'v1': #V10 V11
        try:

            iv = data[3:15]
            ciphertext = data[3+12:-16]
            tag = data[-16:]
            cipher = AES.new(key, AES.MODE_GCM, iv)
            decrypted_value = cipher.decrypt_and_verify(ciphertext, tag)
            return decrypted_value
        except:
            try:
                _list : list = win32crypt.CryptUnprotectData(data, None, None, None, 0)
                _dec : str = _list[1]
                return str(_dec)
            except:
                # not supported
                return ""
    else: #Google Chrome Adds App-Bound Encryption HERE!!!!
        try:
                            ### THE SOLUTION SHOULD BE LOOKS LIKE BELOW
            ### _list : list = win32crypt.CryptProtectData(data, None, None, None, 0)
            ### _dec : str = _list[1]
            ### return str(_dec)
        except:
            return ""

        pass

`

mic-user commented 2 months ago

and.. the related source code group at https://source.chromium.org/chromium/chromium/src/+/main:chrome/browser/os_crypt/app_bound_encryption_provider_win.cc;drc=c0265133106c7647e90f9aaa4377d28190b1a6a9;l=30

mic-user commented 2 months ago

and... what does 'v10', 'v20' means, the encrypted cookie data in the Chromium cookie db(which is sqlite) as below. All value have 'v20' header is now the problem.

[v10 examples in the cookie db] 화면 캡처 2024-09-16 112648

[v20 examples in the cookie db] 화면 캡처 2024-09-16 112734

mic-user commented 2 months ago

Only the workaround that compatible mode as v10 legacy format for Windows I found.

[Disclaimer]

[Steps]

  1. Register the registry to disable the 'App bound encryption' as the Google's official document (https://chromeenterprise.google/policies/#ApplicationBoundEncryptionEnabled)

    [HKEY_LOCAL_MACHINE\Software\Policies\Google\Chrome] "ApplicationBoundEncryptionEnabled"=dword:00000000

  2. Kill all chrome process in task manager include every single background invisible process.

  3. Now, you can delete saved cookie database that encrypted by v20-App bounded method. C:\Users\ (user name)\AppData\Local\Google\Chrome\User Data\Default\Network\Cookies <! delete it or back up. Now, ready to save v10 old passion cookie data into empty cookie storage.

  4. Delete pre generated v20-app bounded key at C:\Users\ (user name)\AppData\Local\Google\Chrome\User Data\Local State <! delete it or backup. Now, nothing distrupt v10 legacy format.

  5. Ready to Launch Chrome. Launch Chrome. Now, you will see the newly installation welcome page. Everything is good to go. (Only but lost/backedup all previously saved cookie infomation.)

  6. Confirm/check the 'ApplicationBoundEncryptionEnabled' registry setting properly entered by entering address as 'chrome://policy/'. You will see 'Policy' and value which you manually set at the Step '1' above.

  7. Done. And you can start gather login informations again by conducting gmail login and other your usual activity as previous days. From now on, it will be saved as v10 legacy format cookies unless Google block the compatible method.

화면 캡처 2024-09-16 201135


And..please any one find the v20-app bound mode python logic for all around globe.

Have a good day.

paraxx commented 2 months ago

It works! Thanks a lot

tp1415926535 commented 2 months ago

And..please any one find the v20-app bound mode python logic for all around globe.

Yes, we can't make every user who uses the software do that. I'd like to see the solution at the development level

VanhLegend commented 2 months ago

Tôi chỉ tìm được giải pháp thay thế ở chế độ tương thích với định dạng cũ v10 cho Windows.

[Tuyên bố miễn trừ trách nhiệm]

  • Bằng cách thực hiện giải pháp thay thế này, bạn có thể mất toàn bộ tính năng của cấp độ bảo mật và tăng nguy cơ tấn công mà Google đã công bố ( https://www.bleepingcomputer.com/news/security/ google-chrome-adds-app-bound-encryption-to-block-infostealer-malware/ )
  • Bằng cách thực hiện giải pháp thay thế này, bạn sẽ mất mọi dữ liệu 'tất cả cookie' đã lưu ngay lập tức bao thông tin đăng nhập gmail, youtube, tiktok và bất kỳ cookie cài đặt nào khác. Nó sẽ được xóa sạch. NHƯNG, nếu bạn đăng nhập hoặc cài đặt lại, bạn có thể bắt đầu lại cookie mới trong vòng đời.
  • Mọi việc xảy ra khi thực hiện giải pháp này đều khiến bạn tự chịu rủi ro.

[Các bước]

  1. Đăng ký đăng ký để vô hiệu hóa 'Ứng dụng liên kết hóa hóa' theo tài liệu chính thức của Google ( https://chromeenterprise.google/policies/#ApplicationBoundEncryptionEnabled )

[HKEY_LOCAL_MACHINE\Software\Policies\Google\Chrome] "ApplicationBoundEncryptionEnabled"=dword:00000000

  1. Tắt tất cả tiến trình Chrome trong tác vụ quản lý, bao gồm mọi tiến trình ẩn trong nền.
  2. Bây giờ, bạn có thể xóa cookie cơ sở dữ liệu đã được mã hóa bằng phương pháp giới hạn v20-App. C:\Users\ (tên người dùng)\AppData\Local\Google\Chrome\User Data\Default\Network\Cookies <! xóa hoặc sao lưu. Bây giờ, sẵn sàng lưu trữ cũ niềm đam mê cookie dữ liệu v10 vào bộ nhớ trống cookie.
  3. Xóa khóa giới hạn v20-app được tạo trước đó tại C:\Users\ (tên người dùng)\AppData\Local\Google\Chrome\User Data\Local State <! xóa hoặc sao lưu. Bây giờ, không có gì làm gián đoạn định dạng cũ v10.
  4. Chrome launcher sẵn sàng. Khởi động Chrome. Bây giờ, bạn sẽ thấy trang web mới cài đặt chào mừng. Mọi thứ đã có sẵn. (Chỉ mất/sao lưu tất cả các cookie thông tin đã lưu trước đó.)
  5. Xác nhận/kiểm tra đăng ký cài đặt 'ApplicationBoundEncryptionEnabled' đã nhập đúng bằng cách nhập địa chỉ là 'chrome://policy/'. Bạn sẽ thấy 'Chính sách' và giá trị mà bạn đã đặt thủ công ở Bước '1' ở trên.
  6. Xong. Và bạn có thể bắt đầu thu thập lại thông tin đăng nhập bằng cách thực hiện đăng nhập gmail và các hoạt động thông thường khác của bạn như những ngày trước. Từ bây giờ, thông tin sẽ được lưu dưới dạng cookie định dạng cũ v10 trừ khi Google chặn phương pháp tương thích.

화면 캡처 2024-09-16 201135

Và...làm ơn đó có thể tìm ra logic python chế độ buộc v20-app cho toàn cầu.

Chúc bạn một ngày tốt lành.

Any solution for v20 yet?

blul1ghtz commented 1 month ago

Im interested in decrypting v20 and have average experience in c++ but im not sure if its specific to the pid or the chrome application or if you can even inject into chrome and decrypt it, Anybody know if v20 uses the same encryption?

mic-user commented 1 month ago

@VanhLegend @blul1ghtz A guy made some sample code for v20, try it. https://github.com/runassu/chrome_v20_decryption/blob/31ff1e97a42bc548aa9894f72108b922b3cdba1f/decrypt_chrome_v20_cookie.py

from

https://github.com/yt-dlp/yt-dlp/issues/10927#issuecomment-2412854548

mic-user commented 1 month ago

Only the workaround that compatible mode as v10 legacy format for Windows I found.

[Disclaimer]

  • By performing this workaround, you will may lose security level integrity and increase hacking risk that announced by Google (https://www.bleepingcomputer.com/news/security/google-chrome-adds-app-bound-encryption-to-block-infostealer-malware/)
  • By performing this workaround, you will lose every already 'saved all cookie' data instantly include gmail, youtube, tiktok login infomation and any other cookie settings. It will be wiped out. BUT, if you login or setting again, it will be fine to re-start new cookie life.
  • Everything happen by performing this workaround at your own risk.

[Steps]

  1. Register the registry to disable the 'App bound encryption' as the Google's official document (https://chromeenterprise.google/policies/#ApplicationBoundEncryptionEnabled)

[HKEY_LOCAL_MACHINE\Software\Policies\Google\Chrome] "ApplicationBoundEncryptionEnabled"=dword:00000000

  1. Kill all chrome process in task manager include every single background invisible process.
  2. Now, you can delete saved cookie database that encrypted by v20-App bounded method. C:\Users\ (user name)\AppData\Local\Google\Chrome\User Data\Default\Network\Cookies <! delete it or back up. Now, ready to save v10 old passion cookie data into empty cookie storage.
  3. Delete pre generated v20-app bounded key at C:\Users\ (user name)\AppData\Local\Google\Chrome\User Data\Local State <! delete it or backup. Now, nothing distrupt v10 legacy format.
  4. Ready to Launch Chrome. Launch Chrome. Now, you will see the newly installation welcome page. Everything is good to go. (Only but lost/backedup all previously saved cookie infomation.)
  5. Confirm/check the 'ApplicationBoundEncryptionEnabled' registry setting properly entered by entering address as 'chrome://policy/'. You will see 'Policy' and value which you manually set at the Step '1' above.
  6. Done. And you can start gather login informations again by conducting gmail login and other your usual activity as previous days. From now on, it will be saved as v10 legacy format cookies unless Google block the compatible method.

화면 캡처 2024-09-16 201135

And..please any one find the v20-app bound mode python logic for all around globe.

Have a good day.

From the chrome version '130.0.6723.59'(on 16th Oct 2024), seems google made some changes not so big. So that even v10 data contain extra 32bytes of byte-stream at the begining of decrypted string. Therefore, you guys need to trim out the data at the begining like this.

Patch needed like below

  1. commenting - db.text_factory setting

  2. decrypted_str : str = decrypted_value[32:].decode("utf-8", errors='ignore')
mcazim98 commented 1 month ago

For an alternative way to retrieve the cookie is using Firefox to login to the specific website. I needed the cookies to have a way to login to sites without manually logging in, and currently instead of egde or chrome i just login to the site on firefox and extract the cookie from there.

berkinkadiroglu commented 1 month ago

Patch needed like below

  1. commenting - db.text_factory setting

  2. decrypted_str : str = decrypted_value[32:].decode("utf-8", errors='ignore')

@mic-user where exactly should this patch be applied?

mic-user commented 1 month ago

@berkinkadiroglu Usually, ppl using sample code from below two. https://thepythoncode.com/article/extract-chrome-cookies-python https://thepythoncode.com/code/extract-chrome-cookies-python There is the function that 'decrypt_data()' and there is decodiong line which returning value. There is it. (Sorry about don't know where the same position in the borisbabic's 'browser_cookie3'. I'm just sharing what I found within the new Chrome update. The guy who maintaining the browser_cookie3 will/should know where it is(I hope he/she understand what is going on here).)

teddywing commented 1 month ago

It looks like @mic-user's patch would go here:

diff --git a/browser_cookie3/__init__.py b/browser_cookie3/__init__.py
index 33bf364..e65a427 100644
--- a/browser_cookie3/__init__.py
+++ b/browser_cookie3/__init__.py
@@ -574,17 +574,17 @@ class ChromiumBased:
         for key in keys:
             cipher = AES.new(key, AES.MODE_CBC, self.iv)

             # will rise Value Error: invalid padding byte if the key is wrong,
             # probably we did not got the key and used peanuts
             try:
                 decrypted = unpad(cipher.decrypt(
                     encrypted_value), AES.block_size)
-                return decrypted.decode('utf-8')
+                return decrypted[32:].decode('utf-8')
             except ValueError:
                 pass
         raise BrowserCookieError('Unable to get key for cookie decryption')

 class Chrome(ChromiumBased):
     """Class for Google Chrome"""
mic-user commented 1 month ago

@teddywing Hey, Will you put this on some where around there?

        #-----------------------------
        local_VERSION_path : str = os.path.join(os.environ["USERPROFILE"],
                                        "AppData", "Local", "Google", "Chrome",
                                        "User Data", "Last Version")
        with open(local_VERSION_path, "r", encoding="utf-8") as f:
            _version_raw : str = f.read()
        version : list =  [int(n) for n in _version_raw.split('.')] #_version_raw.split('.')
        if (version[2] < 6723): #130.0.6723.59
            decrypted_str : str = decrypted_value.decode()
        else:
            decrypted_str : str = decrypted_value[32:].decode("utf-8", errors='ignore') 
teddywing commented 4 weeks ago

@mic-user Thanks for the backward compatibility suggestion, but I’m afraid I won’t have time to make a real patch and pull request for this fix.

mimi89999 commented 3 weeks ago

There is a way to read cookies from Chrome that does not require admin permissions. A PoC was published in https://github.com/thewh1teagle/rookie/issues/81