search

Makepad-fr / fbjs

Tooling that automates your Facebook interactions.
https://www.npmjs.com/package/@makepad/fbjs
GNU General Public License v3.0
64 stars 23 forks source link

TimeoutError: waiting for selector "#login_form" failed: timeout 30000ms exceeded #13

Closed wootwoot1234 closed 3 years ago

wootwoot1234 commented 3 years ago

Hello.

I'm excited to try out this tool. I just installed it and ran it but got this error:

(node:47502) UnhandledPromiseRejectionWarning: TimeoutError: waiting for selector "#login_form" failed: timeout 30000ms exceeded.

This is the command I used:

fgps --group-ids ##########

I then ran it with the --headful param and can see the browser open and load the facebook.com but then doesn't fill in the username/password.

kaanyagci commented 3 years ago

Hi there,

Apologies for the late response.

Did you initialize the scraper with fgps init ?

We will take a look as soon as possible.

wootwoot1234 commented 3 years ago

Thanks for responding. Yeah, I ran the init.

I think the issue might be with the selectors? I don't have a "#login_form". So I changed it to this:

  'login_form': {
    'email': '#email',
    'password': '#pass',
    'submit': '#u_0_b',
    'parent': '#u_0_a',
  }

But that didn't fix it so I'm not sure what's up. Here is the HTMl that I'm seeing:

<form class="_featuredLogin__formContainer" data-testid="royal_login_form" action="/login/?privacy_mutation_token=eyJ0eXBlIjowLCJjcmVhdGlvbl90sdvr6v6ssv6jY0Nzg0sdfgRRsdfgWxsc2l0ZV9pZCI6MzgxMjI5MDc5NTc1OTQ2fQ%3D%3D" method="post" onsubmit="" id="u_0_a">
    <input type="hidden" name="jazoest" value="2884" autocomplete="off"><input type="hidden" name="lsd" value="AVpha3BE7Do" autocomplete="off">
    <div>
        <div class="_6lux"><input type="text" class="inputtext _55r1 _6luy" name="email" id="email" data-testid="royal_email" placeholder="Email or Phone Number" autofocus="1" aria-label="Email or Phone Number"></div>
        <div class="_6lux"><input type="password" class="inputtext _55r1 _6luy" name="pass" id="pass" data-testid="royal_pass" placeholder="Password" aria-label="Password"></div>
    </div>
    <input type="hidden" autocomplete="off" name="login_source" value="comet_headerless_login"><input type="hidden" autocomplete="off" name="next" value="">
    <div class="_6ltg"><button value="1" class="_42ft _4jy0 _6lth _4jy6 _4jy1 selected _51sy" name="login" data-testid="royal_login_button" type="submit" id="u_0_b">Log In</button></div>
    <div class="_6ltj"><a href="https://www.facebook.com/recover/initiate/?ars=facebook_login&amp;privacy_mutation_token=eyJ0eXBlIjowLCJsdfgssdf64r5rs7l90aW1lIjoxNjA4MjY0Nzg0LCJjYWxsc2l0ZV9pZCI6Mzgx5NTc1OTQ2fQ%3D%3D">Forgot Password?</a></div>
    <div class="_8icz"></div>
    <div class="_6ltg"><a role="button" class="_42ft _4jy0 _6lti _4jy6 _4jy2 selected _51sy" href="#" ajaxify="/reg/spotlight/" id="u_0_2" data-testid="open-registration-form-button" rel="async">Create New Account</a></div>
</form>
kaanyagci commented 3 years ago

Seems like the issue is related to the cookie banner.

Will make the patch to accept all cookies as soon as possible. image

kaanyagci commented 3 years ago

@wootwoot1234 just updated for accepting all cookies and new login selectors. But I need to update selectors for the new Facebook UI. I'll try to push a new version before the end of the day.

wootwoot1234 commented 3 years ago

Thanks! I'll test it in a few hours.

I wasn't seeing the cookie modal but hopefully that does matter.

Thanks for updating it.

Best, Tom

On Fri, Dec 18, 2020 at 6:54 AM Kaan Yagci notifications@github.com wrote:

@wootwoot1234 https://github.com/wootwoot1234 just updated for accepting all cookies and new login selectors. But I don't have time to test with my account as I have 2FA activated. Could you please test on branch facebook-login-issue-fix if you have time? If you don't have time, I'll test it before merging it before the end of the day!

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/Makepad-fr/facebook-group-posts-scraper/issues/13#issuecomment-748069452, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADGOGIMYL4P2ZVAILOGGQTSVNGJXANCNFSM4TN4UHAQ .

wootwoot1234 commented 3 years ago

I downloaded the new code. I'm not seeing the cookie pop so I commented out that code because otherwise, it failed waiting for it.

Then it got stuck on this line:

https://github.com/Makepad-fr/facebook-group-posts-scraper/blob/39418c3452831d170e18de26c262ecf3c8bfcd8a/src/index.js#L358

So I commented out that line too just to see what would happen and got this error.


(node:7715) UnhandledPromiseRejectionWarning: Error: net::ERR_ABORTED at https://m.facebook.com/groups/InstantPotCommunity/
    at navigate (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix/node_modules/puppeteer/lib/FrameManager.js:120:37)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async FrameManager.navigateFrame (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix/node_modules/puppeteer/lib/FrameManager.js:94:17)
    at async Frame.goto (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix/node_modules/puppeteer/lib/FrameManager.js:406:12)
    at async Page.goto (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix/node_modules/puppeteer/lib/Page.js:672:12)
    at async facebookMain (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix/src/index.js:433:3)
    at async main (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix/src/index.js:618:5)
  -- ASYNC --
    at Frame.<anonymous> (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix/node_modules/puppeteer/lib/helper.js:111:15)
    at Page.goto (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix/node_modules/puppeteer/lib/Page.js:672:49)
    at Page.<anonymous> (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix/node_modules/puppeteer/lib/helper.js:112:23)
    at facebookMain (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix/src/index.js:433:14)
    at main (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix/src/index.js:618:11)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:7715) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:7715) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
kaanyagci commented 3 years ago

@wootwoot1234 This's just fixed. Now there are only two issues:

I'm still working on it

wootwoot1234 commented 3 years ago

Ok, great!

I'm still getting this error but maybe it can be fixed by wrapping it in a try{} catch(e){}? That way if you don't have the cookie banner it continues?


(node:9742) UnhandledPromiseRejectionWarning: TimeoutError: waiting for XPath "//button[@data-cookiebanner="accept_button"]" failed: timeout 30000ms exceeded
    at new WaitTask (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix 2/node_modules/puppeteer/lib/DOMWorld.js:549:28)
    at DOMWorld._waitForSelectorOrXPath (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix 2/node_modules/puppeteer/lib/DOMWorld.js:478:22)
    at DOMWorld.waitForXPath (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix 2/node_modules/puppeteer/lib/DOMWorld.js:441:17)
    at Frame.waitForXPath (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix 2/node_modules/puppeteer/lib/FrameManager.js:642:47)
    at Frame.<anonymous> (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix 2/node_modules/puppeteer/lib/helper.js:112:23)
    at Page.waitForXPath (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix 2/node_modules/puppeteer/lib/Page.js:1131:29)
    at facebookLogIn (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix 2/src/index.js:331:14)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async main (/Users/tom/Downloads/facebook-group-posts-scraper-facebook-login-issue-fix 2/src/index.js:600:10)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:9742) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:9742) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
wootwoot1234 commented 3 years ago

After I added the try/catch, it's working! Thanks!

I have a feature request. It would be nice to be able to limit it to a given number of posts. I don't need it to scrape all the posts and for big groups, it's going to take along time. :)

kaanyagci commented 3 years ago

Perfect! For the cookie issue, as you said I'll wrap it in a try/catch block to avoid the fail when there's no cookie banner. It could perfect if you could push your changes!

For the feature request it sounds great! Could you file a new issue for this so we can work on it as soon as we finish these issues!

Thanks again for your contributions Stay tuned 😎

On Fri, Dec 18, 2020, 16:54 Tom notifications@github.com wrote:

It's working great now! Thanks!

I have a feature request. It would be nice to be able to limit it to a given number of posts. I don't need it to scrape all the posts and for big groups, it's going to take along time. :)

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/Makepad-fr/facebook-group-posts-scraper/issues/13#issuecomment-748170493, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACFOZIRTFGDMECCYXAOSZWLSVN3KLANCNFSM4TN4UHAQ .