"UnicodeDecodeError" when running pentestgpt-cookie on Kali

Af7eR9l0W commented 1 year ago

Running the most recent version of kali. Tried using pip installs via user, anaconda, and running it from the repos /pentestgpt/extract_cookie.py directory. Changing the site-packages chatgpt_config.py and using the expected export methods. export BROWSER_COOKIE_DB=/home/poob/.config/google-chrome/Default/Cookies pentestgpt-cookie

Traceback (most recent call last): File "/home/poob/anaconda3/bin/pentestgpt-cookie", line 8, in sys.exit(main()) File "/home/poob/anaconda3/lib/python3.10/site-packages/pentestgpt/extract_cookie.py", line 29, in main cookies = chrome_cookies(url, cookie_file=cookie_file) File "/home/poob/anaconda3/lib/python3.10/site-packages/pycookiecheat/pycookiecheat.py", line 327, in chrome_cookies val = chrome_decrypt( File "/home/poob/anaconda3/lib/python3.10/site-packages/pycookiecheat/pycookiecheat.py", line 71, in chrome_decrypt return clean(decrypted) File "/home/poob/anaconda3/lib/python3.10/site-packages/pycookiecheat/pycookiecheat.py", line 43, in clean return decrypted[:-last].decode("utf8") UnicodeDecodeError: 'utf-8' codec can't decode byte 0xdb in position 2: invalid continuation byte

After testing the script on mint, (and changing the user agent) it works no problem. Adding some debug prints shows the cookie begins with a ga instead of the usual intercom

Example:

This cookie wont work, even if you omit the ga part. Opening chrome and obtaining the cookie the old way works just fine. Comparing the curl command from the browser vs the one the script outputs, it appears they use different cookies. Changing the url in extract_cookies.py to https://chat.openai.com/api/auth/session' fixes the script and outputs the same cookie as the browser. Applying this change to the mint install did nothing. I'm not sure if its an issue with my kali install or an issue with kali in general.

GreyDGL commented 1 year ago

Yeah I also notice this issue. Will debug and figure out how to do it. For now, please just use the copy&paste method.

vmayoral commented 1 year ago

+1, Just tested natively in OS X.

I can also reproduce this issue. It appears that pentestgpt-cookie has turned invalid from the point wherein I submited https://github.com/GreyDGL/PentestGPT/pull/100 until now. Something might've changed on the server side. I can also confirm to get it to work by "manually copying".

Did a bit of further research, here're some observations (cookies below invalidated already 😉, but using them for the sake of depicting the issues):

Cookie generated by pentestgpt-cookie

``` _puid=user-fFZg2hsobZhDgUv7gbA6Uune:1685299168-t2hfcDN3h%2BQSEa3s5i2BzJVpK8mT9gOhYcD3NvafAJk%3D; intercom-device-id-dgkjq2bp=06c13abe-1fb7-4fb3-b653-82dc2c580e38; intercom-session-dgkjq2bp=ZTFQMkFMQlg0ZHJOd2owK2ZuNnM1L3J0clIwdmZ1M0hJelo5RERSWFdhdXcwczlLYmFXZHlubG1ad1J6TUxsSS0tSGdWdkRzakRtYmhkVC92OEdPNytlUT09--3710aaeea2c9fa77c1744c9470ba7ccbd33b3bcd; __Secure-next-auth.session-token=eyJhbGciOiJkaXIiLCJlbmMiOiJBMjU2R0NNIn0..0NuEDqT63XjJ8nB7.ctcmwUtKpCdQPLcDTIQ_38cDE-FpRS0DvjAyQSGzgRdhbCBJBefot93gp78VsnQ5qMmxvTmIALEsH-QMsX5cVhlhFCiIA0hjlAM o6ZCtQL29zDM_V07vcyb1VSLgnuT1UVLRwzfDWmZ9Sut_Un5H9tNDk_r9A_wgGQ72uYP5rg7RvvtM9fDXcAcczLQ9Qn8tteFcBR86T-JDBM_ZmHrzXqYwlFlVRQ8UQWw498sMSby_bOdnsxumMWbonx8In2Qg3HJHyfNh-FRKXWZKd3LfvGqN9LQiSNj7l R0vCuEbRHICPpr4z_sZOBvF4wX7vKLVhw8rbAsTqxqN7kLC63Wys--v962nFdF0FKSSO1duWGE6m07t_rfA55dRcE5ScuBbqui9NIHGJZEmQXPxarT4nptP601LEcQ2pJynUruF9R3Nfe2XBThDeDV1-bGGRG8YQN5K12PUyM1j3bXEau4CGa3ZKQRqlhT 1p0YBPilY15cCPPBvFZA9LV5eo6AnqIVkylQKpMedypWmJLzme5JyHjURCsZjoGQGfmEVR3TqWoAgo7eb45zrPTHyiWLMnV9bYSnEinJ3gYeFLHbhqTxnSQ3Sehu2zRGs660ETHn68UVqXNfnkIQyja8OCYo5u8kqcJUZTCd-ReUsjZU_p_SBLhJ-711X_ bBcE9VicGRI9jNZJ7J4eQwJXU5eeLoyTuBa07CLTF1RuhoVz51sOJeZn1ECbpK69hjCNu-_8b-xNQdh0b1WsJovLRicuCWtV3HAplNOBc5YLBFtpanfa3lIVR7lZq2CQA4lBpzYQno2L_lsHnhw1ipCcEsNAvowQ5IBCk5U1Ba1tDWgMW8WR2uLUcK9lzL 9erNn4dA7muPN0u69gqzc4lUhKPckqhpsfjVcB4FwvGUL9qYFQsU4r_eh_Ed1yHVqlqTLudEchDK7mWORp7sPNcPt3UKnyX6Z9UmhhK8N3sjkDRB6sWNg9OoD-Wc_NkWLqiMkf-6ZbGM5YBRF3P5L09tLRbdeOisYBABxQWiRl2lcjmPXAm6OR9uhYNy-h d4LnKQ833dbag8IM9QBiyHIXNJEQLd8sfZ27iCf3MF8gyn6eOv_xfXuMe6MQMegyIOzU_3IjNNok8RK5PxJU_DHyDUy5vV0nbByTh9rJyU_o1sFo9ZKeOHD2mip6KCgGw87TG_G07PbCWsU8hpzrFTbP7SjqnqjXCFXiBwJcLgQfFD-weuIiUu7bc7081J 8TzEtMNHG2XwPhSbJYw_U0xAd8AKinq7ebmrDuhFNK_QAQSC4bvkpg3M6L5RYcjdHQqC_ROXoh6c2dV1uPvVuwQJjFVQEmbXwGR5iK4udkdkYWB1XIbtFP_hYGPXmgXf8rpDtvzv3ovDDZGv7O1NPOAtw5eP5ppkdmsrhJr-QZ3XrHFhUpb0VxqbJ_u1au FAzbhGnNfHWCdBj2jK-hG_6ixDl8aQ9GQYudc9n4ITsnD3GKiI5jqHRhzfBye_OEqlZH-xO7wuZ4K2NxPAwlGK_MS7Psw8lEElpiLa8RdXHi93Nzrz0Yr_JR4-6iU__vyn9UjQEmn_7vrK3lGbMx52JF8MAxrjSWSvpuoKE5ytJsuWiztRDv8kN-ecB0SF gzKEi98FROQmZF1vC01tLHrkChTN7wYlGRrlaoEjq6dx8brWo3ThCYfSCnXg6r8va2C1znztbq2c0z_t1kuDP8fEMw4cpNO2rd-9EO3h9ONr6fiC_SQ03suEDCHcLkXh1rMY_7ReHLTRlcT22fkqfoGWGekeLodvpa3KWxw5-O0qD3BKtcb1c19hZqE4ov efYO0XfbtFHK390wxgZpjxNcPMbsaHCPv9kiB4xyPQ6Ijg2Y40H70cDyXxCLh0T3EZH7P-V_B1J3pHQynprI8xH1eXuwxN5QA1pngI3O-xFRIsfbMT1LTx9XlnrCa9vL5PeFarUaMzUM3FvF3NSkUjqyj-HOl2kjbxzz4fpuz2BFqLQq0pLffyJkPBOcAk xr6e04CICzjW0duLewdBTR9mwvVjjNQTiPvV7ku8mpcO69mQrL5V8RQ7oTTmL-MHr9Jtv11PqUx4a3VLlkehmOVN1RHT9Hgxp3TZ55uzx9ofdiu7J9Umb7vXOHqf_6cTrs7QjCorf2pRa8iNb9dCvJmazYCyAFPSEb942Bx3p_6j9sFh4-HKQ7K6_Ywl-j PA0I5o8hSL1lFgIyQyi_R6Y_PZhBDU8fZryJe5WiXdl9rKZxGwNQtKhMHkCoIrfTGiZShWXG3KevB7mBmyjbprEcUJCmjPAZk8pKO9XE-Hd8-Moft5LXc_i-eGmDF9i_0VXL44LhEEfstoVCfuxgILQbJlig2K_nYaw_jYa3uvmWl7MkqPPIRjJpYM_uTc 4yEmoUcveHaEVv-ODCHpn28XxgwP6v4yAKF38XLD8PPiy4b1dmsCAT0iadYwStRDUw5nok9fYb8lUTF6QUNaD2m0hEe7hzi97ccnRzyqz8nFW0Zv7dg_QkYCuxBTEW9nHtDIa96Wrda8Z7b9nTbWOWVu982lzqMbxam9QT2Se7Vqj1FBI7ihnrJGHuSVAe NL1cMxGnALBk5YqmuXje9bKsPMS0l0ng0hpdwyAG6g0VSs3eg.4758yZOa06a0I-hnht4G_Q; _dd_s=rum=0&expire=1685347362999 ```

Cookie manually extracted from the browser

``` intercom-device-id-dgkjq2bp=06c13abe-1fb7-4fb3-b653-82dc2c580e38; __Host-next-auth.csrf-token=162b9ff011f265a676ddaeab196336fc0a895950b195ab254b877b8e816e7fb7%7Ca8b545393dfecd5521a18d7ce7803e5014112f54f03f6be99dc8a6435c3f40b6; __Secure-next-auth.callback-url=https%3A%2F%2Fchat.openai.com; _cfuvid=Sp21cqHKPZOdpxoW5R673npRiuCqnDOzJGxCD1NJBfk-1685506792098-0-604800000; __cf_bm=WXmnbpZmeUehKPJJjKPAojxCzwPfBqn6DQZjo_r27ds-1685506793-0-AU3D4C0TUSls8Ze90swEaHPsU8FVXnjvbppzxXFpRh0gw6JXauX6Z+13uvtE9ROXNFIeIrQKnVvcXD51FTjhDhJVAwXpHS26xSprwOzX5nCRIxCNndd8LwSpVqkAkn6v1FD4cWOvtI8PtX9s2iDVFQA=; intercom-session-dgkjq2bp=OFJ4bXlmT291K0Z6L3ZWWmlNL3pUUXdoN1BsWWhUSit4ZW1zV2FyREd0eHkrdDR0NlZMTU1CakpPdGMxNWNrdS0tL2RJNWFTQjR3cEFoRjg3YnFVSWRNUT09--871e1b40cdb8055d69d52b514f169da7194aabed; _dd_s=rum=0&expire=1685507850622; __Secure-next-auth.session-token=eyJhbGciOiJkaXIiLCJlbmMiOiJBMjU2R0NNIn0..m-F4hpNSDOXKm1nl.-tk3Ap3N3z7XQL7vof2G_GwJBB4Drkc8_ZJDfLt9r2jKZC9Rr3G8vdML8C2KW-jSRhPVM-hq6ryWf5qqZk0h8IfbqVb1iXgZK9BiXZ01jfCeIts80e5h1SGiJAA5rGOIdLWxUWfWKcodl1Z-tnUcOQWIDeN-umYQ6NqJRFjr4NCyaAiNVC3x8QJij3SE_7KcC4Q86vvBQg4wLoCm2U0XwowKvVpd0OJwAWZUmrjuM9ET-62NOBlOUHxYluB7hljs97RFxCfxxtGaLkgciZm21oXZclAEdEMDDQByvxAupTRBKa8X_orNbdreMu5thAvbqPT0AOHTbAJK3SmPfKfijzRLwL8ybhlOwYvlQuR356gI7tm75DPb_hsNvjIGs1CuoNpyNb779nBuiJ3yBPddlOsBhGFe7m766HksOLYjANJnku9GUzgxxH-BI05RHd7ZfE6L2uxBQq8yd_2HLPQDr2w5V6RjWKjlSAZGFIz8K9Pxid-L4Q0CQfVmVgTPYJY2ZkhX-52DOVv7hHgV4LKtzNhpsTK0pXrPTFSurlVSFeNA6M5XbVWbPobtpurcsTabDy39Tcg4lKKnP7X8L7Nnr4ZyKe8xhhSqRH21aLQaJaK1FO897XpCd7ZAmNu9xyM9sWMXGcAjKEHy3QYBT4dJaMVctgXXJ9VnOMLr5gU91IV1xNJOBPGZ6vAwy9NY_iSBsr6_bC9xkYpxvvvCf4dkxOabun_Ho6aTq7d94aHFD58Q2Uvq8P17-8GRn4YB3Mm_r_dd9Xdbj9ab1x2s0yDUXLX0HVt8CdYoMC-e6_2cmuOhu2gMDCxazlOzH1-GAncyIpsKizBcBV2LHktG10E_fLHCkPD-6UoLEyD4hAMOO2x_ZtQd7emo79HgWsle5v4KVWr627ZbLS22RuqWnFLUE2rAO608J779rAHvU9TegMIFZE7sbEhPfDg46nUcZ1AlSMsl4MS9iMY25e_ny27ZzI8yZfdbxsTB1PS24Md4CjrStAyyxxQ7LY1A-3OaYtrno7IneU-rD1dhncv98p4f8wlmcRXNEa9jEaBg0NZimCluAkuAGdimyr5axNktZ5UupWsxc_OZ5SZm68Z3riT8A22gozTkSQsQCSi6N1HZWdIM7UaFpoHau04JdMCH07t_V63WQinGQ3gR_mDnKnETe8nRgsCBbdQuQcXrZoet1AqUNjTk0uNl_zylepys3sT0QEAK47obThqCfCX1uRMImPSlzLTckRy3ZkqcmpttOl4Zdb8TMeP7zN3WQCcv-FUMXt5STecc96zRT9RJnYODpKtNQ29qiZLkeEyJE0TGF3Ne30kuU0CDaP-rqh7AqeQkvOfQf4i7q0AW64rabuOy7iriw3T7W3e4yvFTXGZUlYQQkVyph87xRJNe8neKzLy956ie6BRsQO3tytNwj0QYY-nDIz5lrkCIRJ5l_-hWcqwni8ZC2cxHgwoj8tewRVFfTILCtA37hsL5cC3i-km_i74AbwtkDd583JJyNH2vxIs4FQEySFOI3IPzOhO7iwB4Cn6_-IySe1lUxcsDKMy_dBubZ9_UCXWfcevOkpXwo9RhjgYxpJU77ZWVTPPYdT_79PpbkFKaQLcdmS4gSR5oj0SFQpWrjWHWH65VKsBJIw6Ff2dblodGMk_yjLMqzhdj-Ao1QbjUOEuyu7dBGeOk0H8j5G0NKEFzutxCoy_Jmee0IhIwTRhhgdFoSo0EWEdpn7FwJGdexn_GHt272rDnngUIkHTWioiPR9SsJZmjHGPmLNAay8zpCw0DTdsvooDofiUoPb58mGZMP3bV5HTmUizsAH9Gb7q7zXuFKDhVp7urU2tfIlCIbAG1raF9sNF7_mfUfC0k8ILPnYdN1RQGqXptCy8VNcdjBKJo0i8unCgihNcpwvnUpHrZiM4JyydN93L3w15RKCrwqV9wS9SQPWepgsJIpf9lSsfni8UCXSu0tFWs0SwR2JWSjIludAvHk3OtpImTrYAWfOJv4erAlxQIMUMhO9BXoxYvLiOCN0QXKZztDvIBOPzsRdg1fFabh4adqzCMR_c934p3Cj4QkuBn6t7KWL5hb7Ncr2FGABUWTykPVyiGxiDbOTTEBMb6Vq7ZvC82WdmAoC6nsEgyzUpkBJOVcQ7MOtNzLVGl7eeefnYqLAfRsD81LuE_ojPLoQGMm52evmRfazaEEcYA02jYCmqW8mNR3AFsoQEohWEr0TDrNJ5MivHezrNHzWzxQS6xuVkewPr_zq6efjiTJprd6eRFhrqd6bim_ZaYvIUsN537XOFGjzOAX0KorhEwD02oCaMXIycSrwhZWU1N5-MBQdHldOsuHPt0DLFO9y_34PvhH6D86F1916rBdwG12qDJDuxvQRsA6jZg9DRoz2KyKmFPdVrT13qQK9f7bJoZUgBIwoSgOT6JCzHk8HnA_LbA5HRkyK4GCWLE_DbMhXuJ2sGeB4W35X6y-vbYC52NbA5zDB8DeScnk1F_hxdjuOMk0wA6ymU-gxd0-3elMFy9vpxxJsqrzMQNqi8D1pQZfbLWcwv7_nj0_OEg-sX-3trsy4VPloYjq8bcWF3sJ-Y_i5EEVsl92WZ11Smko0IY8NkablBTLyP3DA.QpJOnjRwkmLrg0j0A93xoQ; _puid=user-fFZg2hsobZhDgUv7gbA6Uune:1685506951-oU84zvw%2FsMsf7dJd6J2BsQNv33%2B8bzXjR%2B5CJ6jjAUE%3D ```

⚠️ Cookies are ordered differently. This should be fine, I believe.
❌ Cookies content is different https://chat.openai.com/share/d3f52223-4235-4421-a359-73070ea6d652. This is somewhat weird, as I confirmed they were identical just a couple of days ago.
⚠️ Cookies are formatted differently. Newlines, etc.
⚠️ This discussion is in the open, so OpenAI IT/security folks are likely monitoring which means that any countermeasure we implement might be directly translated into more hurdles. After all, we're bypassing their business model

GreyDGL commented 1 year ago

In my previous examination, I noticed that the varying cookie fields might be due to some third-party antibot verification (from Cloudflare) adopted by OpenAI. This prevents effective cookie retrieval/generation from browser simulation (Selenium, Playwright, etc.). For different OS/browsers, Cloudflare may have some custom verification methods, and the generated cookies are different. I think they're not directly related to OpenAI, but yeah we're sort of bypassing their verification. Anyway I think we shall at least maintain a valid cookie-based access to ChatGPT until the bypassing method becomes too complex. In the future, we may focus more on the API-based solution.

Af7eR9l0W commented 1 year ago

It seems like pycookiecheat may be culprit behind the unicode decode errors. Switching to browser_cookie3 solved it for me. I've included it my most recent pull request. As long as the varying cookie fields don't get too crazy, keeping an exclusion list inside a variable should suffice. I added that with my latest request as well. There is probably a better way to do it tho

vmayoral commented 1 year ago

It seems like pycookiecheat may be culprit behind the unicode decode errors. Switching to browser_cookie3 solved it for me. I've included it my most recent pull request. As long as the varying cookie fields don't get too crazy, keeping an exclusion list inside a variable should suffice. I added that with my latest request as well. There is probably a better way to do it tho

Nice approach!

I gave your fork a try at https://github.com/Af7eR9l0W/PentestGPT. Tested it in Darwin. Cookie produced with pentestgpt-cookie didn't work for me neither. Any thoughts?

Af7eR9l0W commented 1 year ago

Nice approach!

I gave your fork a try at https://github.com/Af7eR9l0W/PentestGPT. Tested it in Darwin. Cookie produced with pentestgpt-cookie didn't work for me neither. Any thoughts?

Try changing this url to the variable URL. If that doesn't work you can view the CookieJar by adding a print statement on the following line. As a last resort try getting it through your browser since it could be an issue with my code. Modifying the BROWSER_COOKIE_DB to another chromium based browser (edge works great for me) is worth a shot.

Here recently it seems no matter how you get the cookies or which one you use it doesn't want to work. It definitely feels like browser verification stuff has ramped up. Since creating pull request I've gotten 3 new cookie fields _cfuvid, _gauf,_cfu_. However, I wouldn't be surprised to learn its been user error all along.

If only the api wasn't so damn expensive

CarlosCarlos109 commented 1 year ago

Hi there. Nice work Grey. Does it work on Kali? Kind regards;

GreyDGL commented 1 year ago

@CarlosCarlos109 Was working on some other features. Will start to implement support for Kali this week. My testing shows that Kali with Python3.10 works, and there's some compatibility issue with Python3.11

CarlosCarlos109 commented 1 year ago

Thank you very much for the information @GreyDGL. I will try to use venv python then. Good work!! @GreyDGL I'll be waiting for new developments, gladly would help. Kind Regards;

GreyDGL / PentestGPT

"UnicodeDecodeError" when running pentestgpt-cookie on Kali #105