TheNetsky / Microsoft-Rewards-Script

Automated Microsoft Rewards script, using TypeScript, Cheerio and Playwright.
232 stars 52 forks source link

Mobile Searches are not working #155

Closed DestroyerV closed 6 days ago

DestroyerV commented 1 month ago

After the update to 1.4.11 the mobile search are not working. If the desktop searches complete the script reopen the chromium in desktop mode not in mobile browser mode because of this the mobile search point is not getting collected.

kianfotovat commented 1 month ago

Adding on to this: the "mobile" browser is opening in desktop mode and the user agent string is the same as the desktop one.

After some investigation, I found that passing the browserListQuery option to generateFingerprint() seems to be causing the devices and operatingSystems options to be ignored. I tried two workarounds:

  1. Passing only browserListQuery with the string "last 1 ChromeAndroid versions"
    • Didn't seem to do anything at all: resulted in a desktop-sized browser with an outdated Chrome on Windows user agent
    • I used ChromeAndroid because Browserslist doesn't have a specific flag for mobile Edge
  2. Using the old way to specify browsers (passing browsers: ['edge'] instead of browserListQuery)
    • This does create a mobile-sized browser with an (old) Edge on mobile Android user agent, but...
    • Doesn't fix it either unfortunately, searches still don't give any points

I haven't tried rolling back to an earlier version of the script, but that should be the same thing as the second workaround I mentioned anyways. I'll report back if I figure out a solution.

TheNetsky commented 1 month ago

Indeed seems to be an issue, browserlist doesn't seem to have support for the mobile version of Edge.

Thinking of just dropping chromium and using actual MS Edge instead,

kianfotovat commented 1 month ago

Thinking of just dropping chromium and using actual MS Edge instead,

I had to wait until today for my searches to be reset so I could test but I just got around to it:

I uncommented the channel: 'msedge' line (which produced an Edge browser as expected) but I still didn't get any points for mobile searches. This test was done with browsers: ['edge'] instead of browserListQuery because I still haven't figured out why fingerprint-generator ignores the other parameters when its used.

The mobile check-in and read-to-earn still work. Not really sure why, I'm guessing maybe those pages aren't as strict when it comes to checking fingerprints (since they technically should only be accessible through the Bing app anyways), but that's a total shot in the dark.

Either way, I'm fairly confident in saying this breakage didn't have anything to do with the 1.4.11 update. I've tried rolling back the whole script to 1.4.10 as well as just rolling back the dependencies that were updated but I still saw the exact same behavior.

TheNetsky commented 1 month ago

check-in and read-to-earn use the API, fingerprints are only for the actual browser, Having the fingerprint AND msedge is counteractive since the fingerprint will override the user-agent of the actual browser.

kianfotovat commented 1 month ago

I figured as much but didn't think it'd matter. But I realize now that Playwright will use the system-installed Edge which is most likely better than using a generated fingerprint anyways.

devallareddyb commented 1 month ago

so mobile search won't work for now right?

TheNetsky commented 1 month ago

Correct, downgrade to .10 for now, when I have time I'll see if I can switch it up to make it work again.

devallareddyb commented 1 month ago

Correct, downgrade to .10 for now, when I have time I'll see if I can switch it up to make it work again. Thank you

Just now I updated mobile finger-print manually and started testing, I will update shortly in 1 hour.

devallareddyb commented 1 month ago

It is working now after updating fingerprint and getting points for mobile search.

[10/5/2024, 9:37:42 AM] [PID: 29388] [LOG] [SEARCH-BING] Completed searches [10/5/2024, 9:37:43 AM] [PID: 29388] [LOG] [BROWSER] Created browser with User-Agent: "Mozilla/5.0 (iPhone; CPU iPhone OS 17_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.0 Mobile/15E148 Safari/604.1" [10/5/2024, 9:37:43 AM] [PID: 29388] [LOG] [MAIN] Starting MOBILE browser [10/5/2024, 9:37:55 AM] [PID: 29388] [LOG] [LOGIN] Email entered successfully [10/5/2024, 9:37:58 AM] [PID: 29388] [LOG] [LOGIN] Password entered successfully [10/5/2024, 9:38:05 AM] [PID: 29388] [LOG] [LOGIN] Successfully logged into the rewards portal [10/5/2024, 9:38:05 AM] [PID: 29388] [LOG] [LOGIN-BING] Verifying Bing login [10/5/2024, 9:38:05 AM] [PID: 29388] [LOG] [LOGIN] Logged into Microsoft successfully [10/5/2024, 9:38:11 AM] [PID: 29388] [LOG] [LOGIN] Logged in successfully [10/5/2024, 9:38:11 AM] [PID: 29388] [LOG] [LOGIN-BING] Bing login verification passed! [10/5/2024, 9:38:20 AM] [PID: 29388] [LOG] [GO-HOME] Visited homepage successfully [10/5/2024, 9:38:22 AM] [PID: 29388] [LOG] [DAILY-CHECK-IN] Starting Daily Check In [10/5/2024, 9:38:22 AM] [PID: 29388] [LOG] [READ-TO-EARN] Starting Read to Earn [10/5/2024, 9:38:22 AM] [PID: 29388] [LOG] [DAILY-CHECK-IN] Already claimed today [10/5/2024, 9:38:22 AM] [PID: 29388] [LOG] [READ-TO-EARN] Completed Read to Earn [10/5/2024, 9:38:22 AM] [PID: 29388] [LOG] [READ-TO-EARN] Read all available articles [10/5/2024, 9:38:30 AM] [PID: 29388] [LOG] [GO-HOME] Visited homepage successfully [10/5/2024, 9:38:30 AM] [PID: 29388] [LOG] [SEARCH-BING] Starting Bing searches [10/5/2024, 9:38:32 AM] [PID: 29388] [LOG] [SEARCH-GOOGLE-TRENDS] Generating search queries, can take a while! | GeoLocale: GB [10/5/2024, 9:38:34 AM] [PID: 29388] [LOG] [SEARCH-BING] 60 Points Remaining | Query: daniel day-lewis | Mobile: true

This is the fingerprint i used { "fingerprint": { "screen": { "availHeight": 844, "availWidth": 390, "pixelDepth": 24, "height": 844, "width": 390, "availTop": 0, "availLeft": 0, "colorDepth": 24, "innerHeight": 844, "outerHeight": 844, "outerWidth": 390, "innerWidth": 390, "screenX": 0, "pageXOffset": 0, "pageYOffset": 0, "devicePixelRatio": 3, "clientWidth": 390, "clientHeight": 844, "hasHDR": true }, "navigator": { "userAgent": "Mozilla/5.0 (iPhone; CPU iPhone OS 17_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.0 Mobile/15E148 Safari/604.1", "userAgentData": { "brands": [ {"brand": "Safari", "version": "17"}, {"brand": "Apple", "version": "15"} ], "mobile": true, "platform": "iOS", "architecture": "ARM", "bitness": "64", "fullVersionList": [ {"brand": "Safari", "version": "17.0"}, {"brand": "Apple", "version": "15.0"} ], "model": "iPhone 15", "platformVersion": "17.0", "uaFullVersion": "17.0.0" }, "language": "en-US", "languages": ["en-US"], "platform": "iPhone", "deviceMemory": 4, "hardwareConcurrency": 6, "maxTouchPoints": 10, "product": "Gecko", "productSub": "20100101", "vendor": "Apple Inc.", "vendorSub": null, "doNotTrack": null, "appCodeName": "Mozilla", "appName": "Netscape", "appVersion": "5.0 (iPhone; CPU iPhone OS 17_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.0 Mobile/15E148 Safari/604.1", "oscpu": null, "extraProperties": { "vendorFlavors": ["safari"], "isBluetoothSupported": true, "globalPrivacyControl": null, "pdfViewerEnabled": true, "installedApps": [] }, "webdriver": false }, "audioCodecs": { "ogg": "", "mp3": "probably", "wav": "probably", "m4a": "maybe", "aac": "probably" }, "videoCodecs": { "ogg": "", "h264": "probably", "webm": "" }, "pluginsData": { "plugins": [ { "name": "QuickTime Plugin", "description": "Handles video, sound, and interactive content", "filename": "QuickTime Plugin.plugin", "mimeTypes": [ {"type": "video/quicktime", "suffixes": "mov", "description": "QuickTime Movie"} ] } ], "mimeTypes": [ "QuickTime Movievideo/quicktimemov" ] }, "battery": { "charging": true, "chargingTime": null, "dischargingTime": null, "level": 0.85 }, "videoCard": { "vendor": "Apple", "renderer": "Apple A17 Pro GPU" }, "multimediaDevices": { "speakers": [{"deviceId": "", "kind": "audiooutput", "label": "", "groupId": ""}], "micros": [{"deviceId": "", "kind": "audioinput", "label": "", "groupId": ""}], "webcams": [{"deviceId": "", "kind": "videoinput", "label": "", "groupId": ""}] }, "fonts": ["San Francisco", "SF Pro Text", "SF Pro Display"], "mockWebRTC": false, "slim": true }, "headers": { "sec-ch-ua-mobile": "?1", "sec-ch-ua-platform": "\"iOS\"", "user-agent": "Mozilla/5.0 (iPhone; CPU iPhone OS 17_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.0 Mobile/15E148 Safari/604.1", "accept-encoding": "gzip, deflate, br, zstd", "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.7", "sec-ch-ua": "\"Safari\";v=\"17\", \"Apple\";v=\"15\"", "upgrade-insecure-requests": "1", "accept-language": "en-US", "sec-fetch-site": "same-site", "sec-fetch-mode": "navigate", "sec-fetch-user": "?1", "sec-fetch-dest": "document" } }

image

basalisco commented 1 month ago

mobile_fingerpint.json

I tried this fingerprint and it works properly

kianfotovat commented 1 month ago

mobile_fingerpint.json

I tried this fingerprint and it works properly

Thank you. I'm curious, how did you make this?

devallareddyb commented 1 month ago

mobile_fingerpint.json I tried this fingerprint and it works properly

Thank you. I'm curious, how did you make this?

I did same, I replaced mobile_fingerprint.json with the fingerprint I provided in my previous reply.

I got that fingerprint from chatGPT.

image

TheNetsky commented 1 month ago

The problem is that it's a "static" fingerprint, rather than a new one being generated. So not much use for this project. The one you provided almost looks identical to the ones being generated in 0.10, since that too uses an outdated version of Edge, .124

basalisco commented 1 month ago

Il problema è che si tratta di un'impronta digitale "statica", piuttosto che di una nuova che viene generata. Quindi non è molto utile per questo progetto. Quello che hai fornito sembra quasi identico a quelli generati nella versione 0.10, poiché anche quello utilizza una versione obsoleta di Edge, .124

yes friend it's a static from the previous version. have find this library

TheNetsky commented 1 month ago

The project is already using that package.

kianfotovat commented 1 month ago

Correct, downgrade to .10 for now, when I have time I'll see if I can switch it up to make it work again.

Just to let you know, downgrading to 1.4.10 did not resolve the issue for me. Not sure if that's due to a problem on my end but my setup hasn't changed (running in a Win10 VM) apart from Playwright requiring me to re-install its Chrome browser (v1138 -> v1440).

Edit: I tried to do a desktop-only run like I have been for a couple days and I'm not getting desktop search points either. Maybe my account is flagged?

TheNetsky commented 1 month ago

I already seem to have fixed the issue in regards to points. What I'm currently struggling with is if there's a a new kind of cooldown for searches, and if so what is it, can that data be fetched somewhere. Or are cooldowns to result of the scripts searches. Since it seems It's around 3 searches (9 points) per 15min? But that also seems to vary.

Searches also no longer the requirement to be the Edge browser, from testing this works fine without.

I hope someone can help me figure this one out, with some more information on the situation.

Also fun fact, whilst peaking around I found that MS does fingerprint, however it doesn't seem to be enabled for some reason. You can see it within this script, this is loaded when you do a search on bing.com.

//<![CDATA[ sj_evt && sj_evt.bind( "onRALoad", function() { var reportActivityModule = new ModernRewards.ReportActivity({"AnimateHeader":true,"IsAuthenticated":true,"DashboardUrl":"https://rewards.bing.com/?signin=1","IsMobileClient":false,"RewardsIncrement":1,"RewardsSessionData":{"IsRewardUser":true,"IsLinkedUser":false,"IsTenantEnabled":false,"IsAadUser":false,"IsTrialUser":false,"TrialUserClaimBalance":-1.0,"IsRebatesUser":false,"IsRebatesDeniedUser":false,"Balance":10087,"RewardsBalance":10087,"GiveBalance":0,"RebatesBalance":0.0,"IsGiveModeOn":false,"GiveModeCid":"","GiveModeCName":"","PreviousBalance":10087,"GoalTrackBalance":0,"IsLevel2":true,"IsOptOut":false,"IsSuspended":false,"ShowAnimation":false,"EPuid":"REDACTED","IsRedirectedFromOldDashboard":false,"ImpressionLifeTimeCount":0,"DailyImpressionHPCount":0,"DailyImpressionSerpCount":1,"VisitedCount":35,"LastVisitTime":"2024-10-08T07:00:00Z","LastAutoOpenFlyoutTime":"0001-01-01T00:00:00","LastRewardsDashboardVisitTimeEpoch":1728378636,"LastRewardsFlyoutLoadTimeEpoch":1728378635,"LastRewardsL2AutoGoBigFlyoutLoadTime":0,"AutoOpenFlyoutFlag":false,"IsCLOUser":false,"Waitlist":"2","IsAdultMSA":"0","IsMSA18":"0","IsCcpEligible":"2","HasUsedCcpTrial":"0","SERPTheme":"","DailyCheckInProgress":0,"DailyCheckInAnimate":0,"LastAadRedDotShown":"0001-01-01T00:00:00","LastMedallionRedemptionAnimationImpression":"0001-01-01T00:00:00+00:00","IsRedemptionReadyAndAnimationEnabled":false,"IsUserFirstMedallionImpressionToday":true,"MedallionGiftHoverTooltipText":"Ready to redeem","ShowMedallionFadeAnimation":false},"RewardsHeader":"Rewards","NeedUpdateRewardsHeaderLink":false,"AnimationAltText":"Animation","AutoOpenFlyout":false,"AutoOpenGoBigL2Scenario":"","IsAutoOpenGoBigL2Treatment":false,"BnpReportActivityProperties":{"IsBnpReportActivityEligible":false,"NotificationsInsertionPointId":2000,"PartnerId":"RewardsSerpPostLoad"},"AutoOpenFlyoutSuppressionReason":"","IsGiveHeaderTextEnabled":false,"IsGiveSerpHeaderTealHeartEnabled":false,"IsRewardsEntryPointEnabled":false,"BalanceMessage":null,"ShowNonMemberUpsellMessage":false,"LogWaitlistAutoJoin":false,"IsDailyCheckInMedallionAnimationEnabled":false,"UseRedDotFromBingDomain":true,"SendFingerprint":false,"SendFingerprintV4":false}); reportActivityModule.initialize(); }, 1 );; //]]>

mgrimace commented 1 month ago

Correct, downgrade to .10 for now, when I have time I'll see if I can switch it up to make it work again.

Just to let you know, downgrading to 1.4.10 did not resolve the issue for me. Not sure if that's due to a problem on my end but my setup hasn't changed (running in a Win10 VM) apart from Playwright requiring me to re-install its Chrome browser (v1138 -> v1440).

Edit: I tried to do a desktop-only run like I have been for a couple days and I'm not getting desktop search points either. Maybe my account is flagged?

My mobile searches did appear to fix after downgrading from 1.4.11 to 1.4.10. That being said, I'm using docker, so 'downgrading' was a full deletion of the script and full rebuild (noting that accounts.json, compose.yaml, and config.json persist so no need to reconfigure).

All of that to say, maybe delete your /dist folder and completely rebuild the script from to make sure there isn't any lingering weirdness with sessions, script pieces, etc (if you haven't already done so)?

mgrimace commented 1 month ago

Or are cooldowns to result of the scripts searches. Since it seems It's around 3 searches (9 points) per 15min? But that also seems to vary.

The best that I can find is that cooldowns appear to be inconsistently and seemingly randomly applied to accounts. Seems to be a few r/microsoftrewards posts about it lately so it might be triggering more often/randomly for people lately.

All I know: it's a '15 minute' cooldown after 3 maybe 4 searches before points can be earned again. Application of the cooldown appears random, with some part being country-related. It can last for days/weeks/months, it can randomly disappear. I have no idea how to know or tell if the cooldown is applied to an account, and it isn't clear what (if any) behaviour triggers it.

My solution has been to use larger timeouts that result in roughly 3-4 searches every 15 mins just-in-case (and more randomness in time between). I accidentally tried the default timeouts when I downgraded and it seemed to work just fine, which I take to mean that I don't have the cooldown applied to my account at the moment, but I keep the larger timeouts on just in case. The drawback of course is that the script takes much longer to complete (which I do headless, so less of an impact for me personally)

Thats about all I know and all I can dig up on my end!

mgrimace commented 1 month ago

my 'safe' timeouts are:

    "searchDelay": {
      "min": 180000,
      "max": 270000
    },

I have had cooldown in my region for a while and these values do seem to 'bypass' it without triggering a cooldown.

TheNetsky commented 1 month ago

Yeah I've been fiddling with that, but sometimes it's hard to distinct if it's the timeout at play or a bad user-agent, since I'm still trying to find a stable way to generate valid user-agents with proper fingerprints.

However that sometimes result in PC searches working but not mobile, or the browser being detected as automation software.

So I've been tinkering a lot with it, the current package doesn't seem to be working well, very outdated browsers and randomly not respecting the parameters set.

However generating a fingerprint and just overriding the user-agent will get flagged as fake, since it doesn't match the generated fingerprint.

basalisco commented 1 month ago

my 'safe' timeouts are:

    "searchDelay": {
      "min": 180000,
      "max": 270000
    },

I have had cooldown in my region for a while and these values do seem to 'bypass' it without triggering a cooldown.

        "searchDelay": {
            "min": 300000,
            "max": 600000

I use these values, and in the last release anyway I had no points. fixed only by importing the old fingerprint. For me, going below those values means not having points

kianfotovat commented 1 month ago

All of that to say, maybe delete your /dist folder and completely rebuild the script from to make sure there isn't any lingering weirdness with sessions, script pieces, etc (if you haven't already done so)?

Hmm, the only files I re-used were account/config.json and the session folder but maybe a new session would help.

Seems to be a few r/microsoftrewards posts about it lately so it might be triggering more often/randomly for people lately.

I saw the same thing. I dodged the last timeout wave but it seems like I got caught up in this one. Haven't found anything about detecting whether your account is subject to the cooldown or not though (aside from observing the behavior obviously).

mgrimace commented 1 month ago

fixed only by importing the old fingerprint.

is it better practice to keep fingerprint? I've had my saveFingerprint set to false

TheNetsky commented 1 month ago

There's a lot stuff unknown about this. however the issue mostly seems to be with generating valid ones at this point.

rodolfohro commented 1 month ago

sorry if it is a dumb quest, but where i place the mobile_fingerprint.json ? cant find one on my folder to replace

TheNetsky commented 1 month ago

sorry if it is a dumb quest, but where i place the mobile_fingerprint.json ? cant find one on my folder to replace

Enable "saveFingerprint" in config.json, then put the file in the /browser/sessions/ where the desktop_cookies.json etc are located.

rodolfohro commented 1 month ago

sorry if it is a dumb quest, but where i place the mobile_fingerprint.json ? cant find one on my folder to replace

Enable "saveFingerprint" in config.json, then put the file in the /browser/sessions/ where the desktop_cookies.json etc are located.

thx for the quick reply, that didn't quite fix my problem, i believe is a Node.js essue for today i will just do it manually

TheNetsky commented 1 month ago

I think I've fixed the issue, going to be running some more tests before releasing the new release. image

TheNetsky commented 1 month ago

https://github.com/TheNetsky/Microsoft-Rewards-Script/tree/1.4.12 See if this fixes the issue

mgrimace commented 1 month ago

https://github.com/TheNetsky/Microsoft-Rewards-Script/tree/1.4.12 See if this fixes the issue

I'll test, though my activities are completed for the day so please give me a 24hr cycle to see if mobile completes. Testing with doMorePromotions and doPunchCards both set to false (for #157 and #159). I also have saveFingerprint set to default (false) in case that matters.

TheNetsky commented 1 month ago

https://github.com/TheNetsky/Microsoft-Rewards-Script/tree/1.4.12 See if this fixes the issue

I'll test, though my activities are completed for the day so please give me a 24hr cycle to see if mobile completes. Testing with doMorePromotions and doPunchCards both set to false (for #157 and #159). I also have saveFingerprint set to default (false) in case that matters.

What county are you in, so when I have time, I can make an account for that area, see if I can replicate those activities to add support for these.

Also what specifically in https://github.com/TheNetsky/Microsoft-Rewards-Script/issues/159?

mgrimace commented 1 month ago

Just tried to run 1.4.12, got the following error. Note I am using Docker, destroyed the container and rebuilt from scratch:

=> ERROR [microsoft-rewards-script  8/12] RUN ./node_modules/.bin/playwright install chromium                   0.5s 
------                                                                                                                
 > [microsoft-rewards-script  8/12] RUN ./node_modules/.bin/playwright install chromium:                              
0.420 node:internal/modules/cjs/loader:1143                                                                           
0.420   throw err;                                                                                                    
0.420   ^                                                                                                             
0.420                                                                                                                 
0.420 Error: Cannot find module 'playwright/lib/transform/esmLoader'
0.420 Require stack:
0.420 - /usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/transform/esmUtils.js
0.420 - /usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/runner/processHost.js
0.420 - /usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/runner/loaderHost.js
0.420 - /usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/runner/loadUtils.js
0.420 - /usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/runner/reporters.js
0.420 - /usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/runner/runner.js
0.420 - /usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/program.js
0.420 - /usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/cli.js
0.420     at Module._resolveFilename (node:internal/modules/cjs/loader:1140:15)
0.420     at Function.resolve (node:internal/modules/helpers:188:19)
0.420     at Object.<anonymous> (/usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/transform/esmUtils.js:26:114)
0.420     at Module._compile (node:internal/modules/cjs/loader:1364:14)
0.420     at Module._extensions..js (node:internal/modules/cjs/loader:1422:10)
0.420     at Module.load (node:internal/modules/cjs/loader:1203:32)
0.420     at Module._load (node:internal/modules/cjs/loader:1019:12)
0.420     at Module.require (node:internal/modules/cjs/loader:1231:19)
0.420     at require (node:internal/modules/helpers:177:18)
0.420     at Object.<anonymous> (/usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/runner/processHost.js:10:17) {
0.420   code: 'MODULE_NOT_FOUND',
0.420   requireStack: [
0.420     '/usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/transform/esmUtils.js',
0.420     '/usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/runner/processHost.js',
0.420     '/usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/runner/loaderHost.js',
0.420     '/usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/runner/loadUtils.js',
0.420     '/usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/runner/reporters.js',
0.420     '/usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/runner/runner.js',
0.420     '/usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/program.js',
0.420     '/usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/cli.js'
0.420   ]
0.420 }
0.420 
0.420 Node.js v18.20.4
------
failed to solve: process "/bin/sh -c ./node_modules/.bin/playwright install chromium" did not complete successfully: exit code: 1
mgrimace commented 1 month ago

https://github.com/TheNetsky/Microsoft-Rewards-Script/tree/1.4.12 See if this fixes the issue

I'll test, though my activities are completed for the day so please give me a 24hr cycle to see if mobile completes. Testing with doMorePromotions and doPunchCards both set to false (for #157 and #159). I also have saveFingerprint set to default (false) in case that matters.

What county are you in, so when I have time, I can make an account for that area, see if I can replicate those activities to add support for these.

Also what specifically in #159?

Ah my bad, I didn't check to see if PunchCards were fixed in 1.4.12, I'll enable. I had experienced the same error where if Punchcards were enabled searches wouldn't complete.

I'm in Canada - for #157 it seems like you have to search for specific keywords related to the task (example screenshots below):

Screen Shot 2024-10-15 at 3 21 06 PM Screen Shot 2024-10-15 at 3 21 00 PM
TheNetsky commented 1 month ago

Just tried to run 1.4.12, got the following error. Note I am using Docker, destroyed the container and rebuilt from scratch:

=> ERROR [microsoft-rewards-script  8/12] RUN ./node_modules/.bin/playwright install chromium                   0.5s 
------                                                                                                                
 > [microsoft-rewards-script  8/12] RUN ./node_modules/.bin/playwright install chromium:                              
0.420 node:internal/modules/cjs/loader:1143                                                                           
0.420   throw err;                                                                                                    
0.420   ^                                                                                                             
0.420                                                                                                                 
0.420 Error: Cannot find module 'playwright/lib/transform/esmLoader'
0.420 Require stack:
0.420 - /usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/transform/esmUtils.js
0.420 - /usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/runner/processHost.js
0.420 - /usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/runner/loaderHost.js
0.420 - /usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/runner/loadUtils.js
0.420 - /usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/runner/reporters.js
0.420 - /usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/runner/runner.js
0.420 - /usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/program.js
0.420 - /usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/cli.js
0.420     at Module._resolveFilename (node:internal/modules/cjs/loader:1140:15)
0.420     at Function.resolve (node:internal/modules/helpers:188:19)
0.420     at Object.<anonymous> (/usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/transform/esmUtils.js:26:114)
0.420     at Module._compile (node:internal/modules/cjs/loader:1364:14)
0.420     at Module._extensions..js (node:internal/modules/cjs/loader:1422:10)
0.420     at Module.load (node:internal/modules/cjs/loader:1203:32)
0.420     at Module._load (node:internal/modules/cjs/loader:1019:12)
0.420     at Module.require (node:internal/modules/cjs/loader:1231:19)
0.420     at require (node:internal/modules/helpers:177:18)
0.420     at Object.<anonymous> (/usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/runner/processHost.js:10:17) {
0.420   code: 'MODULE_NOT_FOUND',
0.420   requireStack: [
0.420     '/usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/transform/esmUtils.js',
0.420     '/usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/runner/processHost.js',
0.420     '/usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/runner/loaderHost.js',
0.420     '/usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/runner/loadUtils.js',
0.420     '/usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/runner/reporters.js',
0.420     '/usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/runner/runner.js',
0.420     '/usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/lib/program.js',
0.420     '/usr/src/microsoft-rewards-script/node_modules/rebrowser-playwright/cli.js'
0.420   ]
0.420 }
0.420 
0.420 Node.js v18.20.4
------
failed to solve: process "/bin/sh -c ./node_modules/.bin/playwright install chromium" did not complete successfully: exit code: 1

Yeah I switched libs, so you gotta basically remove everything and reinstall all the packages, including chromium.

mgrimace commented 1 month ago

Yeah I switched libs, so you gotta basically remove everything and reinstall all the packages, including chromium.

Ok, hmm, then there's likely an issue with the dockerfile since the script and all the packages should be completely removed and completely rebuilt on destroying/rebuilding the container - likely a separate issue specifically for docker.

edit: it looks like the dockerfile may need an update to use rebrowser-playwright to install chromium. I can't figure out how to do that in my testing yet.

TheNetsky commented 1 month ago

Yeah I switched libs, so you gotta basically remove everything and reinstall all the packages, including chromium.

Ok, hmm, then there's likely an issue with the dockerfile since the script and all the packages should be completely removed and completely rebuilt on destroying/rebuilding the container - likely a separate issue specifically for docker.

edit: it looks like the dockerfile may need an update to use rebrowser-playwright to install chromium. I can't figure out how to do that in my testing yet.

I accidently removed the playwright package, added it back, should work now.

mgrimace commented 1 month ago

I accidently removed the playwright package, added it back, should work now.

Yep, docker container installs now as expected. The issue I'm having is when the script runs, I get the error browserType.launch: Executable doesn't exist at /root/.cache/ms-playwright/chromium-1134/chrome-linux/chrome When I browse /root/.cache/ms-playwright its showing chromium-1140

TheNetsky commented 1 month ago

I accidently removed the playwright package, added it back, should work now.

Yep, docker container installs now as expected. The issue I'm having is when the script runs, I get the error browserType.launch: Executable doesn't exist at /root/.cache/ms-playwright/chromium-1134/chrome-linux/chrome When I browse /root/.cache/ms-playwright its showing chromium-1140

Try again, it needs version of playwright "1.47.2" explicitly

mgrimace commented 1 month ago

I accidently removed the playwright package, added it back, should work now.

Yep, docker container installs now as expected. The issue I'm having is when the script runs, I get the error browserType.launch: Executable doesn't exist at /root/.cache/ms-playwright/chromium-1134/chrome-linux/chrome When I browse /root/.cache/ms-playwright its showing chromium-1140

Try again, it needs version of playwright "1.47.2" explicitly

That worked, changed the dependency is package.json for playwright from "^1.47.2" to "1.47.2", script is running now as expected. Doing desktop searches just fine, I'll see if there's mobile left to do and report back. So far so good thanks!

mgrimace commented 1 month ago

Mobile searches do appear to be working on my end with 1.4.12!

TheNetsky commented 1 month ago

Good stuff! Thanks for testing. Am going to change up some more stuff, likely change this to 1.5 instead. Will require some docker testing again once I finished the "parallel" functionality.

mgrimace commented 1 month ago

Good stuff! Thanks for testing. Am going to change up some more stuff, likely change this to 1.5 instead. Will require some docker testing again once I finished the "parallel" functionality.

Happy to help, thanks for all your work! I'll happily test/contribute to the docker implementation once it's ready.

MatadorProBr commented 1 month ago

1.4.12 See if this fixes the issue

Didn't fix it for my side sadly, it tries to search but no points are gained so it just changes the user-agent and restarts this whole loop. Here is a example:

[17/10/2024, 19:00:41] [PID: 3180] [LOG] [BROWSER] Created browser with User-Agent: "Mozilla/5.0 (Android 10; Linux; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Mobile Safari/537.36 EdgA/129.0.2792.84"
[17/10/2024, 19:00:41] [PID: 3180] [LOG] [MAIN] Starting MOBILE browser
[17/10/2024, 19:00:43] [PID: 3180] [LOG] [LOGIN] Already logged in
[17/10/2024, 19:00:43] [PID: 3180] [LOG] [LOGIN-BING] Verifying Bing login
[17/10/2024, 19:00:50] [PID: 3180] [LOG] [LOGIN-BING] Bing login verification passed!
[17/10/2024, 19:00:50] [PID: 3180] [LOG] [LOGIN] Logged in successfully
[17/10/2024, 19:00:57] [PID: 3180] [LOG] [GO-HOME] Visited homepage successfully
[17/10/2024, 19:00:58] [PID: 3180] [LOG] [DAILY-CHECK-IN] Starting Daily Check In
[17/10/2024, 19:00:58] [PID: 3180] [LOG] [DAILY-CHECK-IN] Already claimed today
[17/10/2024, 19:00:58] [PID: 3180] [LOG] [READ-TO-EARN] Starting Read to Earn
[17/10/2024, 19:00:59] [PID: 3180] [LOG] [READ-TO-EARN] Read all available articles
[17/10/2024, 19:00:59] [PID: 3180] [LOG] [READ-TO-EARN] Completed Read to Earn
[17/10/2024, 19:01:06] [PID: 3180] [LOG] [GO-HOME] Visited homepage successfully
[17/10/2024, 19:01:06] [PID: 3180] [LOG] [SEARCH-BING] Starting Bing searches
[17/10/2024, 19:01:07] [PID: 3180] [LOG] [SEARCH-GOOGLE-TRENDS] Generating search queries, can take a while! | GeoLocale: BR
[17/10/2024, 19:01:11] [PID: 3180] [LOG] [SEARCH-BING] 51 Points Remaining | Query: banco central valores a receber | Mobile: true
[17/10/2024, 19:03:27] [PID: 3180] [LOG] [SEARCH-BING] 51 Points Remaining | Query: uruguai x equador | Mobile: true
[17/10/2024, 19:05:14] [PID: 3180] [LOG] [SEARCH-BING] 51 Points Remaining | Query: oab | Mobile: true
[17/10/2024, 19:06:58] [PID: 3180] [LOG] [SEARCH-BING] 51 Points Remaining | Query: espanha x sérvia | Mobile: true
[17/10/2024, 19:08:46] [PID: 3180] [LOG] [SEARCH-BING] 51 Points Remaining | Query: taiwan | Mobile: true
[17/10/2024, 19:11:08] [PID: 3180] [LOG] [SEARCH-BING] 51 Points Remaining | Query: cher | Mobile: true
[17/10/2024, 19:13:25] [PID: 3180] [WARN] [SEARCH-BING-MOBILE] Search didn't gain point for 5 iterations, likely bad User-Agent
TheNetsky commented 1 month ago

Sadly I can't replicate this, it's been working fine for me since the changes

mgrimace commented 1 month ago

1.4.12 See if this fixes the issue

Didn't fix it for my side sadly, it tries to search but no points are gained so it just changes the user-agent and restarts this whole loop. Here is a example:

Try changing the search delay, it's possible you've had cooldown implemented on your account, see this post above.

kianfotovat commented 1 month ago

Didn't fix it for my side sadly, it tries to search but no points are gained so it just changes the user-agent and restarts this whole loop.

@MatadorProBr This is almost certainly because MS has enabled the cooldown for your account. Our only options are configure the script so it has a minimum of 5 minutes delay between every search (although you'd probably want at least 6-7 minutes to be safe) or forget about searches for the time being and hope the cooldown expires.

Personally I'm going with option #2. I can't be bothered to have this script/an extra browser running for 4-6 hours a day just for a handful of reward points, plus I suspect that continuing to use any script for searches while under the cooldown will result in my account getting banned eventually.

If you want to go with the other option, the search delay config linked in the comment above will probably work, though it may be excessively long.

In any case, I don't think our issue is related to the script.

mgrimace commented 1 month ago

Our only options are configure the script so it has a minimum of 5 minutes delay between every search

It’s probably different region-to-region, but you should be able to get away with 3-4 searches every 5 minutes rather than 1 search/5mins. I posted my cooldowns here, which is my attempt to bring the time down as low as possible. It still takes hours. basalico also posted their cooldowns which were higher/more conservative numbers which would be even safer. It still takes hours to complete, but generally works regardless of cooldown.

kianfotovat commented 1 month ago

It’s probably different region-to-region, but you should be able to get away with 3-4 searches every 5 minutes rather than 1 search/5mins.

Yeah probably, I wouldn't be surprised if different accounts within the same region had different cooldowns either tbh. There's almost no info out there on how MS determines or applies these limits.

FWIW I tested your suggestion (3 searches per 5 minutes) just now but no luck. The first 3 searches gave points like normal, but after waiting 6 minutes and trying to search again I don't get any points. Guessing the restriction on my account is stricter than yours. I'm sure those cooldowns you/basalico posted would work though, so anyone else reading this thread and having the same issues should 100% check those out.

basalisco commented 1 month ago

1.4.12 See if this fixes the issue

Didn't fix it for my side sadly, it tries to search but no points are gained so it just changes the user-agent and restarts this whole loop. Here is a example:

[17/10/2024, 19:00:41] [PID: 3180] [LOG] [BROWSER] Created browser with User-Agent: "Mozilla/5.0 (Android 10; Linux; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Mobile Safari/537.36 EdgA/129.0.2792.84"
[17/10/2024, 19:00:41] [PID: 3180] [LOG] [MAIN] Starting MOBILE browser
[17/10/2024, 19:00:43] [PID: 3180] [LOG] [LOGIN] Already logged in
[17/10/2024, 19:00:43] [PID: 3180] [LOG] [LOGIN-BING] Verifying Bing login
[17/10/2024, 19:00:50] [PID: 3180] [LOG] [LOGIN-BING] Bing login verification passed!
[17/10/2024, 19:00:50] [PID: 3180] [LOG] [LOGIN] Logged in successfully
[17/10/2024, 19:00:57] [PID: 3180] [LOG] [GO-HOME] Visited homepage successfully
[17/10/2024, 19:00:58] [PID: 3180] [LOG] [DAILY-CHECK-IN] Starting Daily Check In
[17/10/2024, 19:00:58] [PID: 3180] [LOG] [DAILY-CHECK-IN] Already claimed today
[17/10/2024, 19:00:58] [PID: 3180] [LOG] [READ-TO-EARN] Starting Read to Earn
[17/10/2024, 19:00:59] [PID: 3180] [LOG] [READ-TO-EARN] Read all available articles
[17/10/2024, 19:00:59] [PID: 3180] [LOG] [READ-TO-EARN] Completed Read to Earn
[17/10/2024, 19:01:06] [PID: 3180] [LOG] [GO-HOME] Visited homepage successfully
[17/10/2024, 19:01:06] [PID: 3180] [LOG] [SEARCH-BING] Starting Bing searches
[17/10/2024, 19:01:07] [PID: 3180] [LOG] [SEARCH-GOOGLE-TRENDS] Generating search queries, can take a while! | GeoLocale: BR
[17/10/2024, 19:01:11] [PID: 3180] [LOG] [SEARCH-BING] 51 Points Remaining | Query: banco central valores a receber | Mobile: true
[17/10/2024, 19:03:27] [PID: 3180] [LOG] [SEARCH-BING] 51 Points Remaining | Query: uruguai x equador | Mobile: true
[17/10/2024, 19:05:14] [PID: 3180] [LOG] [SEARCH-BING] 51 Points Remaining | Query: oab | Mobile: true
[17/10/2024, 19:06:58] [PID: 3180] [LOG] [SEARCH-BING] 51 Points Remaining | Query: espanha x sérvia | Mobile: true
[17/10/2024, 19:08:46] [PID: 3180] [LOG] [SEARCH-BING] 51 Points Remaining | Query: taiwan | Mobile: true
[17/10/2024, 19:11:08] [PID: 3180] [LOG] [SEARCH-BING] 51 Points Remaining | Query: cher | Mobile: true
[17/10/2024, 19:13:25] [PID: 3180] [WARN] [SEARCH-BING-MOBILE] Search didn't gain point for 5 iterations, likely bad User-Agent

try this option in config.json file

        "searchDelay": {
            "min": 300000,
            "max": 600000
        },

It will be very slow to complete about 6h/account. But I think it will work