search

AlexCSDev / PatreonDownloader

Powerful tool for downloading content posted by creators on patreon.com. Supports content hosted on patreon itself as well as external sites (additional plugins might be required).
MIT License
985 stars 98 forks source link

"Unable to retrieve campaign id" and chrome browser issues #123

Closed AlexCSDev closed 2 years ago

AlexCSDev commented 2 years ago

Please keep the reports and discussions of all "Unable to retrieve campaign id" errors here, thank you.

If you are encountering this issue, please try running this test build and report if it fixed the issue for you: https://mega.nz/file/211WVIoL#CSkmHdALPMVluaxUW4oHlaPLG8l_Kakj-83MGI5Srdo

Make sure to extract it to separate folder and make sure you are not running any kind of VPN or other software/hardware which can alter the way requests are sent from your computer

Liz-chan commented 2 years ago

Still getting the same error:

2022-04-25 22:06:37.8697 INFO Retrieving cookies... 2022-04-25 22:07:03.7015 INFO Preparing to download... 2022-04-25 22:07:42.6162 FATAL Fatal error, application will be closed: UniversalDownloaderPlatform.Common.Exceptions.UniversalDownloaderException: Unable to retrieve campaign id: Only part of a ReadProcessMemory or WriteProcessMemory request was completed. ---> System.ComponentModel.Win32Exception (299): Only part of a ReadProcessMemory or WriteProcessMemory request was completed. at System.Diagnostics.NtProcessManager.EnumProcessModulesUntilSuccess(SafeProcessHandle processHandle, IntPtr[] modules, Int32 size, Int32& needed) at System.Diagnostics.NtProcessManager.GetModules(Int32 processId, Boolean firstModuleOnly) at System.Diagnostics.NtProcessManager.GetFirstModule(Int32 processId) at System.Diagnostics.Process.get_MainModule() at PatreonDownloader.PuppeteerEngine.PuppeteerEngine.<>c.b__10_0(Process x) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.PuppeteerEngine\PuppeteerEngine.cs:line 75 at System.Linq.Enumerable.WhereArrayIterator`1.ToArray() at PatreonDownloader.PuppeteerEngine.PuppeteerEngine.KillChromeIfRunning() in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.PuppeteerEngine\PuppeteerEngine.cs:line 74 at PatreonDownloader.PuppeteerEngine.PuppeteerEngine.Initialize(Uri remoteBrowserAddress, Boolean headless) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.PuppeteerEngine\PuppeteerEngine.cs:line 63 at PatreonDownloader.PuppeteerEngine.PuppeteerEngine..ctor(Boolean headless) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.PuppeteerEngine\PuppeteerEngine.cs:line 48 at PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver..ctor() in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.PuppeteerEngine\PuppeteerCaptchaSolver.cs:line 24 at PatreonDownloader.Implementation.PatreonWebDownloader.SolveCaptchaAndUpdateCookies(String url) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.Implementation\PatreonWebDownloader.cs:line 82 at PatreonDownloader.Implementation.PatreonWebDownloader.DownloadString(String url) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.Implementation\PatreonWebDownloader.cs:line 63 at PatreonDownloader.Implementation.PatreonWebDownloader.DownloadString(String url) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.Implementation\PatreonWebDownloader.cs:line 66 at PatreonDownloader.Implementation.PatreonWebDownloader.DownloadString(String url) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.Implementation\PatreonWebDownloader.cs:line 66 at PatreonDownloader.Implementation.PatreonWebDownloader.DownloadString(String url) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.Implementation\PatreonWebDownloader.cs:line 66 at PatreonDownloader.Implementation.PatreonWebDownloader.DownloadString(String url) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.Implementation\PatreonWebDownloader.cs:line 66 at PatreonDownloader.Implementation.PatreonCrawlTargetInfoRetriever.GetCampaignId(String url) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.Implementation\PatreonCrawlTargetInfoRetriever.cs:line 36 --- End of inner exception stack trace --- at PatreonDownloader.Implementation.PatreonCrawlTargetInfoRetriever.GetCampaignId(String url) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.Implementation\PatreonCrawlTargetInfoRetriever.cs:line 49 at PatreonDownloader.Implementation.PatreonCrawlTargetInfoRetriever.RetrieveCrawlTargetInfo(String url) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.Implementation\PatreonCrawlTargetInfoRetriever.cs:line 24 at UniversalDownloaderPlatform.Engine.UniversalDownloader.Download(String url, String downloadDirectory, IUniversalDownloaderPlatformSettings settings) in F:\Sources\BigProjects\PatreonDownloader\submodules\UniversalDownloaderPlatform\UniversalDownloaderPlatform.Engine\UniversalDownloader.cs:line 176 at PatreonDownloader.App.Program.RunPatreonDownloader(CommandLineOptions commandLineOptions) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.App\Program.cs:line 143 at PatreonDownloader.App.Program.Main(String[] args) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.App\Program.cs:line 69

bboshoven commented 2 years ago

I just used the tool successfully from the dev code with some edits from me, but the code isn't nice, so I'll just share my findings.

In the DownloadStringInternal-method of the WebDownloader, when it tries to download the initial given url (in my case https://www.patreon.com/#creator_name#/posts), I was getting a forbidden status code with content: javascript needs to be enabled and/or adblocker to be disabled. So I hardcoded it to use puppeteer instead of the HttpClient in the DownloadStringInternal-method and then everything worked.

AlexCSDev commented 2 years ago

Hm... not the best solution, but might be the most efficient one. Thanks, this gives me some food for thought.

AlexCSDev commented 2 years ago

Another thought I've had - try installing Cloudflare WARP. Make sure "1.1.1.1 with WARP" mode is enabled.

The-Beebz commented 2 years ago

Hello! I just want to report that initially trying to run this tool on my own patreon (for the purposes of archiving and then closing the page) I also got the same error Unable to retrieve campaign id

I tried Cloudflare WARP and turned on the 1.1.1.1 with Warp Setting and ran it again and it worked without needing the test build. So just reporting I did not download the test build whatsoever yet, WARP was enough. Not only did I get all my images since I started my Patreon in 2016 but all zip files and attachments as well.

I noticed that any images uploaded in the "gallery" format were not retrieved in full size, but the id will make it easy for me to go back and find them to retrieve them myself :)

I'm not sure if this is a side effect of me scraping my own Patreon, but every image that wasnt added as an attachment was downloaded twice, one with post and one with media. I doublechecked the checksums of a a couple of the same pictures and they seemed the same. Unsure if this has happened to anyone downloading a Patreon that wasnt their own.

Thank you so much! Hope this is useful for you.

shinji257 commented 2 years ago

I just wanted to drop my 2 cents here as I'm also getting the above error (and probably was last time I tried this). I tried warp speed and the dev build and it doesn't really help. If anything I just get stuck in a loop.

aksskl commented 2 years ago

I've been experiencing a different browser issue. The captcha window doesn't pop up so it gets stuck in a loop when the captcha is triggered and no captcha is returned. I saw the issue being brought up somewhere else. Tried running on another windows machine (fresh windows 11), same issue. Tried two different ISPs. Also tested two VPNs. I'll try linux tomorrow and whatever I can think of. Weird behavior started around the 26th overnight. Maybe my patreon account is acting weird, but I do get the captchas while regularly browsing. Really odd. The browser window does initiate for logins just not the captcha anymore.

I also tried the two latest test and debug builds and it gave me "Please enable JS and disable any ad blocker" in the debug folder. I used to see that message sometimes when the browser window pops up then it loads the captcha after five or ten seconds. It was working on the 25th.

DarthPasma commented 2 years ago

Mine didn't work even with the Cloudflare WARP method.

ArkahnX commented 2 years ago

Running the linked test build from Mega with this command PatreonDownloader.App.exe --download-directory "E:\patreon\hushabyevalley" --verbose --use-sub-directories --url "https://www.patreon.com/hushabyevalley/posts"

After the initial sign on, PatreonDownloader appears to recognize that there is a captcha, but never opens a browser to allow the captcha to be entered, loops a few times, and then crashes. I've posted the log below

2022-05-28 08:55:56.2522 DEBUG [UniversalDownloaderPlatform.Engine.UniversalDownloader] Initializing UniversalDownloader...
2022-05-28 08:55:56.2810 DEBUG [UniversalDownloaderPlatform.Engine.UniversalDownloader] Initializing ninject kernel
2022-05-28 08:55:56.4353 DEBUG [UniversalDownloaderPlatform.Engine.UniversalDownloader] Loading custom ninject module
2022-05-28 08:55:56.4353 DEBUG [UniversalDownloaderPlatform.Engine.UniversalDownloader] Loading ICrawlTargetInfoRetriever
2022-05-28 08:55:56.4879 DEBUG [UniversalDownloaderPlatform.Engine.UniversalDownloader] Configuring IPageCrawler
2022-05-28 08:55:57.0037 DEBUG [UniversalDownloaderPlatform.Engine.PluginManager] New plugin found: UniversalDownloaderPlatform.GoogleDriveDownloader.dll
2022-05-28 08:55:57.0037 FATAL [UniversalDownloaderPlatform.GoogleDriveDownloader.Plugin] !!!![GOOGLE DRIVE]: gd_credentials.json not found, google drive files will not be downloaded! Refer to documentation for additional information. !!!!
2022-05-28 08:55:57.0037 INFO [UniversalDownloaderPlatform.Engine.PluginManager] Loaded plugin: Google Drive Downloader
2022-05-28 08:55:57.0514 DEBUG [UniversalDownloaderPlatform.Engine.PluginManager] New plugin found: UniversalDownloaderPlatform.MegaDownloader.dll
2022-05-28 08:55:57.0514 WARN [UniversalDownloaderPlatform.MegaDownloader.Plugin] !!!![MEGA]: mega_credentials.json not found, mega downloading will be limited! Refer to documentation for additional information. !!!!
2022-05-28 08:55:58.6916 INFO [UniversalDownloaderPlatform.Engine.PluginManager] Loaded plugin: Mega.nz Downloader
2022-05-28 08:55:58.6916 DEBUG [UniversalDownloaderPlatform.Engine.UniversalDownloader] Initializing plugin manager
2022-05-28 08:55:58.6916 DEBUG [UniversalDownloaderPlatform.Engine.UniversalDownloader] Initializing download manager
2022-05-28 08:55:58.6916 DEBUG [PatreonDownloader.Implementation.PatreonCrawledUrlProcessor] KemonoCrawledUrlProcessor initialized
2022-05-28 08:55:58.6916 DEBUG [UniversalDownloaderPlatform.Engine.UniversalDownloader] Initializing crawl results exporter
2022-05-28 08:55:58.6916 DEBUG [UniversalDownloaderPlatform.Engine.UniversalDownloader] Crawl results exporter not provided
2022-05-28 08:55:58.6916 DEBUG [UniversalDownloaderPlatform.Engine.UniversalDownloader] Initializing url checker
2022-05-28 08:55:58.6916 DEBUG [UniversalDownloaderPlatform.Engine.UniversalDownloader] Initializing web downloader
2022-05-28 08:55:58.6916 DEBUG [UniversalDownloaderPlatform.Engine.UniversalDownloader] Initializing cookie validator
2022-05-28 08:55:58.6916 DEBUG [UniversalDownloaderPlatform.Engine.UniversalDownloader] Initializing crawled url processor
2022-05-28 08:55:58.7399 INFO [PatreonDownloader.App.Program] Retrieving cookies...
2022-05-28 08:55:58.7399 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Initializing PuppeteerEngine with parameters , True
2022-05-28 08:55:58.7643 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Found 33 chrome processes (not sure which one yet)
2022-05-28 08:55:58.8467 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Calling login check
2022-05-28 08:55:58.8467 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Retrieving browser
2022-05-28 08:55:58.9114 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Downloading browser
2022-05-28 08:56:44.0542 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Launching browser
2022-05-28 08:56:50.0115 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Opening new page
2022-05-28 08:56:50.2772 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Creating IWebBrowser
2022-05-28 08:56:50.3511 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Checking login status
2022-05-28 08:56:50.8238 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] We are NOT logged in, opening login page
2022-05-28 08:56:50.8238 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Puppeteer is in headless mode, restarting in full mode
2022-05-28 08:56:52.1888 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Initializing PuppeteerEngine with parameters , False
2022-05-28 08:56:52.1888 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Found 34 chrome processes (not sure which one yet)
2022-05-28 08:56:52.2474 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Downloading browser
2022-05-28 08:56:52.2474 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Launching browser
2022-05-28 08:56:52.6783 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Opening new page
2022-05-28 08:56:52.9102 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Creating IWebBrowser
2022-05-28 08:58:08.9731 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Checking login status
2022-05-28 08:58:09.3496 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] We are logged in
2022-05-28 08:58:11.5963 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Initializing PuppeteerEngine with parameters , True
2022-05-28 08:58:11.5963 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Found 33 chrome processes (not sure which one yet)
2022-05-28 08:58:11.6257 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Downloading browser
2022-05-28 08:58:11.6257 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Launching browser
2022-05-28 08:58:11.7764 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Opening new page
2022-05-28 08:58:11.8705 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Creating IWebBrowser
2022-05-28 08:58:11.9446 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Retrieving browser
2022-05-28 08:58:11.9446 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Retrieving cookies
2022-05-28 08:58:14.4043 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Adding cookie: _dd_s
2022-05-28 08:58:14.4043 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Adding cookie: datadome
2022-05-28 08:58:14.4043 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Adding cookie: session_id
2022-05-28 08:58:14.4043 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Adding cookie: patreon_locale_code
2022-05-28 08:58:14.4043 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Adding cookie: _sctr
2022-05-28 08:58:14.4043 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Adding cookie: amplitude_idpatreon.com
2022-05-28 08:58:14.4064 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Adding cookie: _ttp
2022-05-28 08:58:14.4064 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Adding cookie: _fbp
2022-05-28 08:58:14.4064 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Adding cookie: _gat
2022-05-28 08:58:14.4064 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Adding cookie: _scid
2022-05-28 08:58:14.4064 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Adding cookie: patreon_location_country_code
2022-05-28 08:58:14.4064 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Adding cookie: _rdt_uuid
2022-05-28 08:58:14.4064 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Adding cookie: _gcl_au
2022-05-28 08:58:14.4064 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Adding cookie: _tt_enable_cookie
2022-05-28 08:58:14.4064 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Adding cookie: _ga
2022-05-28 08:58:14.4064 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Adding cookie: _ALGOLIA
2022-05-28 08:58:14.4064 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Adding cookie: G_ENABLED_IDPS
2022-05-28 08:58:14.4064 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Adding cookie: _gid
2022-05-28 08:58:14.4064 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Adding cookie: __cf_bm
2022-05-28 08:58:14.4064 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Adding cookie: __ssid
2022-05-28 08:58:14.4064 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCookieRetriever] Adding cookie: patreon_device_id
2022-05-28 08:58:14.4064 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Disposing puppeteer engine
2022-05-28 08:58:15.4373 DEBUG [UniversalDownloaderPlatform.Engine.UniversalDownloader] Universal Downloader Platform settings: SaveDescriptions=False,SaveEmbeds=False,SaveJson=False,SaveAvatarAndCover=False,DownloadDirectory=E:\patreon\hushabyevalley,OverwriteFiles=False,UseSubDirectories=True,MaxFilenameLength=100
2022-05-28 08:58:15.4387 INFO [PatreonDownloader.App.Program] Preparing to download...
2022-05-28 08:58:15.9602 WARN [DynamicInjector70f1a5adfd404e48ae7968c00d9af8f2] Captcha has been triggered, the browser window will be opened now. Please solve the captcha there.
2022-05-28 08:58:15.9602 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Initializing PuppeteerEngine with parameters , True
2022-05-28 08:58:15.9602 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Found 33 chrome processes (not sure which one yet)
2022-05-28 08:58:15.9804 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Calling captcha check
2022-05-28 08:58:15.9804 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Retrieving browser
2022-05-28 08:58:15.9848 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Downloading browser
2022-05-28 08:58:15.9848 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Launching browser
2022-05-28 08:58:16.1552 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Opening new page
2022-05-28 08:58:16.2531 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Creating IWebBrowser
2022-05-28 08:58:16.3182 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Checking if captcha is returned
2022-05-28 08:58:21.0280 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Captcha was not returned, done
2022-05-28 08:58:21.0321 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Retrieving browser
2022-05-28 08:58:21.0321 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Retrieving cookies
2022-05-28 08:58:23.2345 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: _dd_s
2022-05-28 08:58:23.2345 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: datadome
2022-05-28 08:58:23.2345 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: session_id
2022-05-28 08:58:23.2345 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: patreon_locale_code
2022-05-28 08:58:23.2345 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: _sctr
2022-05-28 08:58:23.2345 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: amplitude_idpatreon.com
2022-05-28 08:58:23.2345 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: _ttp
2022-05-28 08:58:23.2345 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: _fbp
2022-05-28 08:58:23.2345 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: _gat
2022-05-28 08:58:23.2345 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: _scid
2022-05-28 08:58:23.2345 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: patreon_location_country_code
2022-05-28 08:58:23.2345 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: _rdt_uuid
2022-05-28 08:58:23.2345 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: _gcl_au
2022-05-28 08:58:23.2345 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: _tt_enable_cookie
2022-05-28 08:58:23.2345 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: _ga
2022-05-28 08:58:23.2345 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: _ALGOLIA
2022-05-28 08:58:23.2345 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: G_ENABLED_IDPS
2022-05-28 08:58:23.2345 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: _gid
2022-05-28 08:58:23.2345 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: __cf_bm
2022-05-28 08:58:23.2345 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: __ssid
2022-05-28 08:58:23.2345 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: patreon_device_id
2022-05-28 08:58:23.2345 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Disposing puppeteer engine
2022-05-28 08:58:23.2968 WARN [DynamicInjector70f1a5adfd404e48ae7968c00d9af8f2] Captcha has been triggered, the browser window will be opened now. Please solve the captcha there.
2022-05-28 08:58:23.2968 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Initializing PuppeteerEngine with parameters , True
2022-05-28 08:58:23.3010 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Found 36 chrome processes (not sure which one yet)
2022-05-28 08:58:23.3171 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] 3 chrome processes are in patreondownloader's folder
2022-05-28 08:58:23.3171 WARN [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Running PatreonDownloader's Chrome detected. Attempting to close it...
2022-05-28 08:58:23.3171 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Attempting to kill PID 23472
2022-05-28 08:58:23.3171 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Attempting to kill PID 22884
2022-05-28 08:58:23.3171 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Attempting to kill PID 4012
2022-05-28 08:58:23.3171 INFO [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Successfully killed all PatreonDownloader's Chrome instances.
2022-05-28 08:58:23.3171 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Calling captcha check
2022-05-28 08:58:23.3171 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Retrieving browser
2022-05-28 08:58:23.3171 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Downloading browser
2022-05-28 08:58:23.3171 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Launching browser
2022-05-28 08:58:23.4458 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Opening new page
2022-05-28 08:58:23.5431 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Creating IWebBrowser
2022-05-28 08:58:23.6135 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Checking if captcha is returned
2022-05-28 08:58:27.7956 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Captcha was not returned, done
2022-05-28 08:58:27.7983 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Retrieving browser
2022-05-28 08:58:27.7983 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Retrieving cookies
2022-05-28 08:58:29.8426 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: _dd_s
2022-05-28 08:58:29.8426 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: datadome
2022-05-28 08:58:29.8426 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: session_id
2022-05-28 08:58:29.8426 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: patreon_locale_code
2022-05-28 08:58:29.8426 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: _sctr
2022-05-28 08:58:29.8426 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: amplitude_idpatreon.com
2022-05-28 08:58:29.8440 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: _ttp
2022-05-28 08:58:29.8440 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: _fbp
2022-05-28 08:58:29.8440 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: _gat
2022-05-28 08:58:29.8440 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: _scid
2022-05-28 08:58:29.8440 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: patreon_location_country_code
2022-05-28 08:58:29.8440 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: _rdt_uuid
2022-05-28 08:58:29.8440 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: _gcl_au
2022-05-28 08:58:29.8440 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: _tt_enable_cookie
2022-05-28 08:58:29.8440 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: _ga
2022-05-28 08:58:29.8440 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: _ALGOLIA
2022-05-28 08:58:29.8440 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: G_ENABLED_IDPS
2022-05-28 08:58:29.8440 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: _gid
2022-05-28 08:58:29.8440 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: __cf_bm
2022-05-28 08:58:29.8440 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: __ssid
2022-05-28 08:58:29.8440 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver] Adding cookie: patreon_device_id
2022-05-28 08:58:29.8440 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Disposing puppeteer engine
2022-05-28 08:58:29.8887 WARN [DynamicInjector70f1a5adfd404e48ae7968c00d9af8f2] Captcha has been triggered, the browser window will be opened now. Please solve the captcha there.
2022-05-28 08:58:29.8887 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Initializing PuppeteerEngine with parameters , True
2022-05-28 08:58:29.8921 DEBUG [PatreonDownloader.PuppeteerEngine.PuppeteerEngine] Found 36 chrome processes (not sure which one yet)
2022-05-28 08:58:30.1648 FATAL [PatreonDownloader.App.Program] Fatal error, application will be closed: UniversalDownloaderPlatform.Common.Exceptions.UniversalDownloaderException: Unable to retrieve campaign id: Only part of a ReadProcessMemory or WriteProcessMemory request was completed.
 ---> System.ComponentModel.Win32Exception (299): Only part of a ReadProcessMemory or WriteProcessMemory request was completed.
   at System.Diagnostics.NtProcessManager.EnumProcessModulesUntilSuccess(SafeProcessHandle processHandle, IntPtr[] modules, Int32 size, Int32& needed)
   at System.Diagnostics.NtProcessManager.GetModules(Int32 processId, Boolean firstModuleOnly)
   at System.Diagnostics.NtProcessManager.GetFirstModule(Int32 processId)
   at PatreonDownloader.PuppeteerEngine.PuppeteerEngine.<>c.<KillChromeIfRunning>b__10_0(Process x) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.PuppeteerEngine\PuppeteerEngine.cs:line 75
   at System.Linq.Enumerable.WhereArrayIterator`1.ToArray()
   at PatreonDownloader.PuppeteerEngine.PuppeteerEngine.KillChromeIfRunning() in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.PuppeteerEngine\PuppeteerEngine.cs:line 74
   at PatreonDownloader.PuppeteerEngine.PuppeteerEngine.Initialize(Uri remoteBrowserAddress, Boolean headless) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.PuppeteerEngine\PuppeteerEngine.cs:line 63
   at PatreonDownloader.PuppeteerEngine.PuppeteerEngine..ctor(Boolean headless) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.PuppeteerEngine\PuppeteerEngine.cs:line 48
   at PatreonDownloader.PuppeteerEngine.PuppeteerCaptchaSolver..ctor() in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.PuppeteerEngine\PuppeteerCaptchaSolver.cs:line 24
   at PatreonDownloader.Implementation.PatreonWebDownloader.SolveCaptchaAndUpdateCookies(String url) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.Implementation\PatreonWebDownloader.cs:line 82
   at PatreonDownloader.Implementation.PatreonWebDownloader.DownloadString(String url) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.Implementation\PatreonWebDownloader.cs:line 63
   at PatreonDownloader.Implementation.PatreonWebDownloader.DownloadString(String url) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.Implementation\PatreonWebDownloader.cs:line 66
   at PatreonDownloader.Implementation.PatreonWebDownloader.DownloadString(String url) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.Implementation\PatreonWebDownloader.cs:line 66
   at PatreonDownloader.Implementation.PatreonCrawlTargetInfoRetriever.GetCampaignId(String url) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.Implementation\PatreonCrawlTargetInfoRetriever.cs:line 36
   --- End of inner exception stack trace ---
   at PatreonDownloader.Implementation.PatreonCrawlTargetInfoRetriever.GetCampaignId(String url) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.Implementation\PatreonCrawlTargetInfoRetriever.cs:line 49
   at PatreonDownloader.Implementation.PatreonCrawlTargetInfoRetriever.RetrieveCrawlTargetInfo(String url) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.Implementation\PatreonCrawlTargetInfoRetriever.cs:line 24
   at UniversalDownloaderPlatform.Engine.UniversalDownloader.Download(String url, String downloadDirectory, IUniversalDownloaderPlatformSettings settings) in F:\Sources\BigProjects\PatreonDownloader\submodules\UniversalDownloaderPlatform\UniversalDownloaderPlatform.Engine\UniversalDownloader.cs:line 176
   at PatreonDownloader.App.Program.RunPatreonDownloader(CommandLineOptions commandLineOptions) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.App\Program.cs:line 143
   at PatreonDownloader.App.Program.Main(String[] args) in F:\Sources\BigProjects\PatreonDownloader\PatreonDownloader.App\Program.cs:line 69
aksskl commented 2 years ago

@ArkahnX That's the same captcha loop I'm seeing, but I don't get the memory error and crash. Mine captcha loops forever.

Though within the last hour or so, the captcha window does pop up if I just leave it looping for a while. Sadly solving the captcha doesn't start the crawling. It keeps looping.

jruiz94 commented 2 years ago

I'm having the same issue, enabling the --debug option shows that a captcha should be solved but the window to solve it never opens. Using Cloudflare WARP does not seem to fix the issue, nor using the test build.

Liz-chan commented 2 years ago

Personally the old version here still works for me, but hopefully the new version can be fixed as well since the old version won't last forever.

There's also an alternative Patreon downloader here on Github if you do some cursory searching.

aksskl commented 2 years ago

@Liz-chan Yeah, for the time being I'm using 0.9.4.2 after reading your comment.

Mustached-Man commented 2 years ago

Same here, didn't have to use a VPN or CloudFlare for version 0.9.4.2

AlexCSDev commented 2 years ago

So let me get this straight, you guys are not having any kind of issues on 0.9.4.2 including captcha triggering?

aksskl commented 2 years ago

So far, no issues. Captchas are not triggering for me, but I haven't used it extensively in the last day. Captchas used to come up very often when it was introduced, almost every time I loaded up a page. Now not as much. Maybe patreon eased up on the captchas.

Mustached-Man commented 2 years ago

The only thing I encountered was the puzzle piece captcha that I only had to solve once. After that, it works like a charm.

1 2

Djfe commented 2 years ago

Only just started using this tool 0.9.4.2 works fine. 0.10.2.0 doesn't work (Unable to retrieve campaign id). test release doesn't work (same error).

my antivirus is windows defender, my dns is 9.9.9.11 (Quad 9) I tried 0.10.2.0 on two pcs (one was reinstalled a few months ago), the rest only on my main pc.

AlexCSDev commented 2 years ago

Really interesting... I guess I'll have to figure out what has changed in the application behavior between 0.9 and 0.10.

FIocker commented 2 years ago

Weird because I was using the latest version 2 weeks ago and it worked fine.

Now it doesn't anymore, neither does the test version. Same creators.

That old version works perfectly fine tho.

AlexCSDev commented 2 years ago

Some progress on my side: I have been able to reproduce the endless captcha loop. Captcha is only being triggered for patreondownloader's internal download engine and not browser used to solve it. Probably some required data is missing from the request.

AlexCSDev commented 2 years ago

Please try version 0.10.3.0. I have improved browser mimicking in that version. Please note, this will probably not solve "ReadProcessMemory or WriteProcessMemory" thing, I have yet to figure out why it happens and how to reproduce it.

Liz-chan commented 2 years ago

The new release works great, thanks Aleksey!

aksskl commented 2 years ago

0.10.3.0 works for me. Thank you!

Mustached-Man commented 2 years ago

Same here, much appreciated!

AlexCSDev commented 2 years ago

Closed. If someone is still having issues with "ReadProcessMemory or WriteProcessMemory" thing, please open a new issue.