search

elebumm / RedditVideoMakerBot

Create Reddit Videos with just✨ one command ✨
GNU General Public License v3.0
6.39k stars 1.8k forks source link

Timeout 30000ms exceeded even after new UI layout when taking screenshots #1971

Open mikkoxs opened 4 months ago

mikkoxs commented 4 months ago

Describe the bug

When the bot tries to take screenshots of individual comments, it times out.

Reproduction Steps

After updating to code relevant for the new UI, it worked yesterday however, today it seems to be stuck on taking screenshots and chromium freezes

Expected behavior

Screenshots

image

  File "/Users/mikko/Developer/RedditVideoMakerBot/main.py", line 132, in <module>
    raise err
  File "/Users/mikko/Developer/RedditVideoMakerBot/main.py", line 111, in <module>
    main(post_id)
  File "/Users/mikko/Developer/RedditVideoMakerBot/main.py", line 53, in main
    get_screenshots_of_reddit_posts(reddit_object, number_of_comments)
  File "/Users/mikko/Developer/RedditVideoMakerBot/video_creation/screenshot_downloader.py", line 266, in get_screenshots_of_reddit_posts
    page.locator(f"#t1_{comment['comment_id']}-comment-rtjson-content").screenshot(
  File "/Users/mikko/Developer/RedditVideoMakerBot/venv/lib/python3.10/site-packages/playwright/sync_api/_generated.py", line 17568, in screenshot
    self._sync(
  File "/Users/mikko/Developer/RedditVideoMakerBot/venv/lib/python3.10/site-packages/playwright/_impl/_sync_base.py", line 109, in _sync
    return task.result()
  File "/Users/mikko/Developer/RedditVideoMakerBot/venv/lib/python3.10/site-packages/playwright/_impl/_locator.py", line 529, in screenshot
    return await self._with_element(
  File "/Users/mikko/Developer/RedditVideoMakerBot/venv/lib/python3.10/site-packages/playwright/_impl/_locator.py", line 115, in _with_element
    handle = await self.element_handle(timeout=timeout)
  File "/Users/mikko/Developer/RedditVideoMakerBot/venv/lib/python3.10/site-packages/playwright/_impl/_locator.py", line 315, in element_handle
    handle = await self._frame.wait_for_selector(
  File "/Users/mikko/Developer/RedditVideoMakerBot/venv/lib/python3.10/site-packages/playwright/_impl/_frame.py", line 322, in wait_for_selector
    await self._channel.send("waitForSelector", locals_to_params(locals()))
  File "/Users/mikko/Developer/RedditVideoMakerBot/venv/lib/python3.10/site-packages/playwright/_impl/_connection.py", line 61, in send
    return await self._connection.wrap_api_call(
  File "/Users/mikko/Developer/RedditVideoMakerBot/venv/lib/python3.10/site-packages/playwright/_impl/_connection.py", line 482, in wrap_api_call
    return await cb()
  File "/Users/mikko/Developer/RedditVideoMakerBot/venv/lib/python3.10/site-packages/playwright/_impl/_connection.py", line 97, in inner_send
    result = next(iter(done)).result()
playwright._impl._api_types.TimeoutError: Timeout 30000ms exceeded.
=========================== logs ===========================
waiting for locator("#t1_ksf1z8o-comment-rtjson-content")
============================================================
╭──────────────────────────────────────────────────────────────────────────────╮
│ Sorry, something went wrong with this version! Try again, and feel free to   │
│ report this issue at GitHub or the Discord community.                        │
│ Version: 3.2.1                                                               │
│ Error: Timeout 30000ms exceeded.                                             │
│ =========================== logs ===========================                 │
│ waiting for locator("#t1_ksf1z8o-comment-rtjson-content")                    │
│ ============================================================                 │
│ Config: {'allow_nsfw': False, 'theme': 'light', 'times_to_run': 10,          │
│ 'opacity': 0.9, 'storymode': False, 'storymodemethod': 1,                    │
│ 'storymode_max_length': 1000, 'resolution_w': 1080, 'resolution_h': 1920,    │
│ 'zoom': 1.0, 'background': {'background_video': 'minecraft',                 │
│ 'background_audio': 'lofi', 'background_audio_volume': 0.03,                 │
│ 'enable_extra_audio': False, 'background_thumbnail': True,                   │
│ 'background_thumbnail_font_family': 'arial',                                 │
│ 'background_thumbnail_font_size': 96, 'background_thumbnail_font_color':     │
│ '255,255,255'}, 'tts': {'voice_choice': 'tiktok', 'random_voice': False,     │
│ 'elevenlabs_voice_name': 'Bella', 'elevenlabs_api_key': 'REDACTED',          │
│ 'aws_polly_voice': 'Matthew', 'streamlabs_polly_voice': 'Matthew',           │
│ 'tiktok_voice': 'en_us_001', 'tiktok_sessionid': 'REDACTED', 'python_voice': │
│ '1', 'py_voice_num': '2', 'silence_duration': 0.3, 'no_emojis': False}}      │
╰──────────────────────────────────────────────────────────────────────────────╯

System Information

Operating System : MacOS Python version : Python 3.10 App version / Branch : Pycharm

Checklist

Additional Context

No response

313hemant313 commented 4 months ago

https://github.com/elebumm/RedditVideoMakerBot/pull/1960 Please check this.

krackn88 commented 4 months ago

I changed the playwright script to be headless:fasle and fixed this issue while trying to get this running today.

video_creation\screenshot_downloader.py

yasdev08 commented 4 months ago

it

I changed the playwright script to be headless:fasle and fixed this issue while trying to get this running today.

video_creation\screenshot_downloader.py

it worked at first but got this error after : 

Launching Headless Browser...
Logging in to Reddit...
Skipping translation...
Something went wrong!
Something went wrong with making the screenshots! Do you want to skip the post? (y/n) n
Do you want the error traceback for debugging purposes? (y/n)y
┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Sorry, something went wrong with this version! Try again, and feel free to report this issue at GitHub or the Discord community.  │
│ Version: 3.2.1                                                                                                                    │
│ Error: Timeout 30000ms exceeded.                                                                                                  │
│ =========================== logs ===========================                                                                      │
│ waiting for locator("[data-test-id=\"post-content\"]")                                                                            │
│ ============================================================                                                                      │
│ Config: {'allow_nsfw': False, 'theme': 'light', 'times_to_run': 1, 'opacity': 0.9, 'storymode': False, 'storymodemethod': 1,      │
│ 'storymode_max_length': 1000, 'resolution_w': 1080, 'resolution_h': 1920, 'zoom': 1.0, 'background': {'background_video':         │
│ 'minecraft', 'background_audio': 'lofi', 'background_audio_volume': 0.15, 'enable_extra_audio': False, 'background_thumbnail':    │
│ False, 'background_thumbnail_font_family': 'arial', 'background_thumbnail_font_size': 96, 'background_thumbnail_font_color':      │
│ '255,255,255'}, 'tts': {'voice_choice': 'tiktok', 'random_voice': True, 'elevenlabs_voice_name': 'Bella', 'elevenlabs_api_key':   │
│ 'REDACTED', 'aws_polly_voice': 'Matthew', 'streamlabs_polly_voice': 'Matthew', 'tiktok_voice': 'en_us_001', 'tiktok_sessionid':   │
│ 'REDACTED', 'python_voice': '1', 'py_voice_num': '2', 'silence_duration': 0.3, 'no_emojis': False}}                               │
└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
Traceback (most recent call last):
  File "C:\Users\DIR\Desktop\RedditVideoMakerBot\main.py", line 131, in <module>
    raise err
  File "C:\Users\DIR\Desktop\RedditVideoMakerBot\main.py", line 113, in <module>
    run_many(config["settings"]["times_to_run"])
  File "C:\Users\DIR\Desktop\RedditVideoMakerBot\main.py", line 68, in run_many
    main()
  File "C:\Users\DIR\Desktop\RedditVideoMakerBot\main.py", line 52, in main
    get_screenshots_of_reddit_posts(reddit_object, number_of_comments)
  File "C:\Users\DIR\Desktop\RedditVideoMakerBot\video_creation\screenshot_downloader.py", line 203, in get_screenshots_of_reddit_posts
    raise e
  File "C:\Users\DIR\Desktop\RedditVideoMakerBot\video_creation\screenshot_downloader.py", line 185, in get_screenshots_of_reddit_posts
    page.locator('[data-test-id="post-content"]').screenshot(path=postcontentpath)
  File "C:\Users\DIR\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\sync_api\_generated.py", line 17568, in screenshot
    self._sync(
  File "C:\Users\DIR\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_sync_base.py", line 109, in _sync
    return task.result()
  File "C:\Users\DIR\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_locator.py", line 529, in screenshot
    return await self._with_element(
  File "C:\Users\DIR\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_locator.py", line 115, in _with_element
    handle = await self.element_handle(timeout=timeout)
  File "C:\Users\DIR\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_locator.py", line 315, in element_handle
    handle = await self._frame.wait_for_selector(
  File "C:\Users\DIR\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_frame.py", line 322, in wait_for_selector
    await self._channel.send("waitForSelector", locals_to_params(locals()))
  File "C:\Users\DIR\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_connection.py", line 61, in send
    return await self._connection.wrap_api_call(
  File "C:\Users\DIR\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_connection.py", line 482, in wrap_api_call
    return await cb()
  File "C:\Users\DIR\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_connection.py", line 97, in inner_send
    result = next(iter(done)).result()
playwright._impl._api_types.TimeoutError: Timeout 30000ms exceeded.
=========================== logs ===========================
waiting for locator("[data-test-id=\"post-content\"]")
============================================================
PericoRico commented 4 months ago

I´m having a similar issue when loggin in to Reddit.

image

Traceback (most recent call last): File "D:\Usuarios\Miguel\Documentos\PROGRAMACION\Repos\pythonVideoMaker\main.py", line 132, in <module> raise err File "D:\Usuarios\Miguel\Documentos\PROGRAMACION\Repos\pythonVideoMaker\main.py", line 114, in <module> run_many(config["settings"]["times_to_run"]) File "D:\Usuarios\Miguel\Documentos\PROGRAMACION\Repos\pythonVideoMaker\main.py", line 69, in run_many main() await self._channel.send("fill", locals_to_params(locals())) File "C:\Users\Miguel\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_connection.py", line 61, in send return await self._connection.wrap_api_call( File "C:\Users\Miguel\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_connection.py", line 482, in wrap_api_call return await cb() File "C:\Users\Miguel\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_connection.py", line 97, in inner_send result = next(iter(done)).result() playwright._impl._api_types.TimeoutError: Timeout 30000ms exceeded. =========================== logs =========================== waiting for locator("[name=\"username\"]") ============================================================

I think maybe it´s not finding the username in the config.toml, but it´s actually over there

I would appreciate any help

Green222 commented 4 months ago

Same problem here - Timeout at first (30000ms on waiting for locator username). Changed the browser to headless: False -> Turned intot "Error: strict mode violation, username resolved to 3 elements." Unfortunately wont work.

testervario commented 4 months ago

I have a similar problem:

Logging in to Reddit... ┌──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ │ Sorry, something went wrong with this version! Try again, and feel free to report this issue at GitHub or the Discord community. │ │ Version: 3.2.1 │ │ Error: Timeout 30000ms exceeded. │ │ =========================== logs =========================== │ │ waiting for locator("[name=\"username\"]") │ │ ============================================================ │ │ Config: {'allow_nsfw': False, 'theme': 'dark', 'times_to_run': 1, 'opacity': 0.9, 'storymode': False, 'storymodemethod': 1, 'storymode_max_length': 1000, 'resolution_w': 1080, │ │ 'resolution_h': 1920, 'zoom': 1.0, 'background': {'background_video': 'minecraft', 'background_audio': 'lofi', 'background_audio_volume': 0.15, 'enable_extra_audio': False, │ │ 'background_thumbnail': False, 'background_thumbnail_font_family': 'arial', 'background_thumbnail_font_size': 96, 'background_thumbnail_font_color': '255,255,255'}, 'tts': │ │ {'random_voice': True, 'elevenlabs_voice_name': 'Bella', 'elevenlabs_api_key': 'REDACTED', 'aws_polly_voice': 'Matthew', 'streamlabs_polly_voice': 'Matthew', 'tiktok_voice': │ │ 'en_us_001', 'tiktok_sessionid': 'REDACTED', 'python_voice': '1', 'py_voice_num': '2', 'silence_duration': 0.3, 'no_emojis': False, 'voice_choice': 'pyttsx'}} │ └──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ Traceback (most recent call last): File "C:\OFFLINEPROJECTS\BOTS\RedditVideo\main.py", line 132, in <module> raise err File "C:\OFFLINEPROJECTS\BOTS\RedditVideo\main.py", line 114, in <module> run_many(config["settings"]["times_to_run"]) File "C:\OFFLINEPROJECTS\BOTS\RedditVideo\main.py", line 69, in run_many main() File "C:\OFFLINEPROJECTS\BOTS\RedditVideo\main.py", line 53, in main get_screenshots_of_reddit_posts(reddit_object, number_of_comments) File "C:\OFFLINEPROJECTS\BOTS\RedditVideo\video_creation\screenshot_downloader.py", line 103, in get_screenshots_of_reddit_posts page.locator('[name="username"]').fill(settings.config["reddit"]["creds"]["username"]) File "C:\Users\carlo\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\sync_api\_generated.py", line 16045, in fill self._sync( File "C:\Users\carlo\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_sync_base.py", line 109, in _sync return task.result() File "C:\Users\carlo\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_locator.py", line 212, in fill return await self._frame.fill(self._selector, strict=True, **params) File "C:\Users\carlo\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_frame.py", line 528, in fill await self._channel.send("fill", locals_to_params(locals())) File "C:\Users\carlo\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_connection.py", line 61, in send return await self._connection.wrap_api_call( File "C:\Users\carlo\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_connection.py", line 482, in wrap_api_call return await cb() File "C:\Users\carlo\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_connection.py", line 97, in inner_send result = next(iter(done)).result() playwright._impl._api_types.TimeoutError: Timeout 30000ms exceeded. =========================== logs =========================== waiting for locator("[name=\"username\"]") ============================================================

image

Any solution?

temrb commented 4 months ago

username resolved to 3 elements

I also got this issue and the problem is noted there-- when looking for the username, it found 3 elements. When you resolve username, you notice the same issue happens for password as well as similarly for the login button. To solve this issue, it's better to precisely target the username input where the login info will go to.

Replace the steps code at around line 82 in screenshot_downloader.py to:

...
    with sync_playwright() as p:
        print_substep("Launching Headless Browser...")
        browser = p.chromium.launch(
            headless=False
        )  # headless=False will show the browser for debugging purposes
        # Device scale factor (or dsf for short) allows us to increase the resolution of the screenshots
        # When the dsf is 1, the width of the screenshot is 600 pixels
        # so we need a dsf such that the width of the screenshot is greater than the final resolution of the video
        dsf = (W // 600) + 1

        context = browser.new_context(
            locale=lang or "en-us",
            color_scheme="dark",
            viewport=ViewportSize(width=W, height=H),
            device_scale_factor=dsf,
        )
        cookies = json.load(cookie_file)
        cookie_file.close()

        context.add_cookies(cookies)  # load preference cookies

        # Login to Reddit
        print_substep("Logging in to Reddit...")
        page = context.new_page()
        page.goto("https://www.reddit.com/login", timeout=0)
        page.set_viewport_size(ViewportSize(width=1920, height=1080))
        page.wait_for_load_state()
        page.wait_for_timeout(5000)  # Adjusted timeout for waiting

        # Wait for the username field to be ready and fill it in
        page.wait_for_selector("input#login-username")
        page.locator("input#login-username").fill(
            settings.config["reddit"]["creds"]["username"]
        )

        # Wait for the password field to be ready and fill it in
        page.wait_for_selector("input#login-password")
        page.locator("input#login-password").fill(
            settings.config["reddit"]["creds"]["password"]
        )

        # Click the login button using the updated selector based on the provided HTML structure
        login_button_selector = "button.login:has-text('Log In')"
        page.wait_for_selector(login_button_selector)
        page.click(login_button_selector)

        page.wait_for_timeout(5000)
...
ElyasMoshirpanahi commented 4 months ago

Hey everyone , same problem here as well but i am running the bot in the command line: here is the full log:

╭──────────────────────────────────────────────────────────────────────────────────────────────────╮ │ Downloading screenshots of reddit posts... │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ Launching Headless Browser... Logging in to Reddit... ╭──────────────────────────────────────────────────────────────────────────────────────────────────╮ │ Sorry, something went wrong with this version! Try again, and feel free to report this issue at │ │ GitHub or the Discord community. │ │ Version: 3.2.1 │ │ Error: Timeout 30000ms exceeded. │ │ =========================== logs =========================== │ │ waiting for locator("[name=\"username\"]") │ │ ============================================================ │ │ Config: {'allow_nsfw': False, 'theme': 'dark', 'times_to_run': 3, 'opacity': 0.9, 'storymode': │ │ False, 'storymodemethod': 1, 'storymode_max_length': 1000, 'resolution_w': 1080, 'resolution_h': │ │ 1920, 'zoom': 1.0, 'background': {'background_video': 'minecraft', 'background_audio': 'lofi', │ │ 'background_audio_volume': 0.15, 'enable_extra_audio': False, 'background_thumbnail': False, │ │ 'background_thumbnail_font_family': 'arial', 'background_thumbnail_font_size': 96, │ │ 'background_thumbnail_font_color': '255,255,255'}, 'tts': {'voice_choice': 'elevenlabs', │ │ 'random_voice': False, 'elevenlabs_voice_name': 'Josh', 'elevenlabs_api_key': 'REDACTED', │ │ 'aws_polly_voice': 'Matthew', 'streamlabs_polly_voice': 'Matthew', 'tiktok_voice': 'y', │ │ 'tiktok_sessionid': 'REDACTED', 'python_voice': '1', 'py_voice_num': '2', 'silence_duration': │ │ 0.3, 'no_emojis': False}} │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ Traceback (most recent call last): File "/content/reddit_bot/main.py", line 132, in raise err File "/content/reddit_bot/main.py", line 114, in run_many(config["settings"]["times_to_run"]) File "/content/reddit_bot/main.py", line 69, in run_many main() File "/content/reddit_bot/main.py", line 53, in main get_screenshots_of_reddit_posts(reddit_object, number_of_comments) File "/content/reddit_bot/video_creation/screenshot_downloader.py", line 103, in get_screenshots_of_reddit_posts page.locator('[name="username"]').fill(settings.config["reddit"]["creds"]["username"]) File "/usr/local/lib/python3.10/dist-packages/playwright/sync_api/_generated.py", line 16045, in fill self._sync( File "/usr/local/lib/python3.10/dist-packages/playwright/_impl/_sync_base.py", line 109, in _sync return task.result() File "/usr/local/lib/python3.10/dist-packages/playwright/_impl/_locator.py", line 212, in fill return await self._frame.fill(self._selector, strict=True, **params) File "/usr/local/lib/python3.10/dist-packages/playwright/_impl/_frame.py", line 528, in fill await self._channel.send("fill", locals_to_params(locals())) File "/usr/local/lib/python3.10/dist-packages/playwright/_impl/_connection.py", line 61, in send return await self._connection.wrap_api_call( File "/usr/local/lib/python3.10/dist-packages/playwright/_impl/_connection.py", line 482, in wrap_api_call return await cb() File "/usr/local/lib/python3.10/dist-packages/playwright/_impl/_connection.py", line 97, in inner_send result = next(iter(done)).result() playwright._impl._api_types.TimeoutError: Timeout 30000ms exceeded. =========================== logs =========================== waiting for locator("[name=\"username\"]")

temrb commented 4 months ago

the main issue is reddit shows 2 different login pages on different occasions with different values-- one where it's light and the form is on the left and one where it's dark / form is in the middle.

my solution works for the dark middle form; the original solution works for the light left form. too lazy to work on a solution where it checks which form it is to actually solve this someone can probably fix it

ElyasMoshirpanahi commented 4 months ago

@temrb Thanks for the feedback! how can I help to implement your solution for my project? Are you against my contribution to help you with the implementation for solving the issue for the both version.? :)

temrb commented 4 months ago

i think quick way is for each input, username & password, is to check if the id #login-username / #login-password exists and if it does, run my posted logic. else, run the original logic.

temrb commented 4 months ago

this solves the main issue almost everyone has been reporting-- "After updating to code relevant for the new UI, it worked yesterday however, today it seems to be stuck"

or "sometimes it works sometimes it doesnt" this is due to reddit rendering 1 version over the other at times vice versa

ElyasMoshirpanahi commented 4 months ago

@temrb thanks for your feedback , I will try to implement your solution.

TT5H commented 4 months ago

Facing the issue: waiting for: locator("[data-test-id=\"post-content\"]")

whereas I have managed to fix the login by editing the values: page.locator('input[name="username"]').fill(settings.config["reddit"]["creds"]["username"]) page.locator('input[name="password"]').fill(settings.config["reddit"]["creds"]["password"]) page.locator('.login.button-brand').click()

ElyasMoshirpanahi commented 4 months ago

@TT5H That sounds like a nice solution, thanks for sharing it 🙌👌

PericoRico commented 4 months ago

Facing the issue: waiting for: locator("[data-test-id="post-content"]")

Same issue after aplying @temrb solution

TT5H commented 4 months ago

Facing the issue: waiting for: locator("[data-test-id="post-content"]")

Just hoping that they fix it ASAP.

berlikaliku commented 3 months ago

hi, any idea what was [data-test-id="post-content"] is? i just cloned this repo and never been on reddit before

EDIT#1:

so im just fooling around by replacing [data-test-id="post-content"] with whatever visible on the page, i noticed there is a .png file generated in assets directory called title.png before encountering the next error

if that was really for the title then i made a change

from:

page.locator("[data-test-id="post-content"]").screenshot(path=postcontentpath)

to:

page.locator(f'h1#post-title-t3_{reddit_id}').screenshot(path=postcontentpath)

can anyone confirm? thanks

github-actions[bot] commented 3 months ago

This issue is stale because it has been open 7 days with no activity. Remove stale label or comment, or this will be closed in 10 days.

deinVater94 commented 3 months ago

i've got the same problem as @testervario and havent really realised how to fix it. can anyone help?

github-actions[bot] commented 3 months ago

This issue is stale because it has been open 7 days with no activity. Remove stale label or comment, or this will be closed in 10 days.

ipharksai commented 3 months ago

The issue is still here. Running it with True for headless Mode enabels my script to login to reddit. however it will then run into a similar problem with the content id:

=========================== logs =========================== waiting for locator("[data-test-id=\"post-content\"]")

That also does not work. Question is why.

We can assume that playwright is fine, what else could be the error?

So there is 1) the error of the script being unable to login except when running the non-headless mode and 2) the error of the script not finding (or chromium not loading) the "post-content".

ipharksai commented 3 months ago

Seems like switching from chromium to firefox solved the first issue for me. I can now login to reddit using firefox.

I still run into: playwright._impl._api_types.TimeoutError: Timeout 30000ms exceeded. =========================== logs =========================== waiting for locator("[data-test-id=\"post-content\"]")

so same error as before.

Green222 commented 3 months ago

Are you guys able to use the RedditVideoMaker in any way? Im waiting for the pull request regarding the new UI to be merged but for me until this happened it doesn't work at all - im suprised to see only a few comments and that PR taking so long to be merged?

ipharksai commented 3 months ago

Inspecting the entire reddit website html code lets me know there is "data-test-id" thingy that could be located. there are several "data-testid" thingys, but even when I change my locator to that it wtill cannot be found.

I am stumped.

I wrote this short script for testing:

` import json from playwright.sync_api import sync_playwright, ViewportSize

def main():

load playwright and launch browser

with sync_playwright() as p:
    #launch browser
    browser = p.firefox.launch(headless=False)

    #set viewport size and device scale factor
    dsf = 2
    context = browser.new_context(
        locale="en-us",
        color_scheme="dark",
        viewport=ViewportSize(width=1920,height=1080),
        device_scale_factor=dsf,
        )

    #load cookies
    # cookies = json.load(open("cookies.json"))
    # context.add_cookies(cookies)

    cookie_file_path = "./video_creation/data/cookie-light-mode.json"
    cookies = json.load(open(cookie_file_path, encoding="utf-8"))
    context.add_cookies(cookies)

    # Going to reddit
    page = context.new_page()
    page.goto("https://www.reddit.com/r/AskOldPeople/comments/15pjcgo/what_is_a_true_story_from_your_life_that_sounds/")

    # wait for post content
    locator = ("[data-testid=\"post-content\"]")
    post_content = page.wait_for_selector(locator)

    post_content.screenshot(path="post_content.png")

if name == "main": main() `

that always gives this error:

C:\Users\b.schwarz\RedditVideoMakerBot>python test.py Traceback (most recent call last): File "C:\Users\b.schwarz\RedditVideoMakerBot\test.py", line 46, in main() File "C:\Users\b.schwarz\RedditVideoMakerBot\test.py", line 40, in main post_content = page.wait_for_selector(locator) File "C:\Users\b.schwarz\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\sync_api_generated.py", line 8316, in wait_for_selector self._sync( File "C:\Users\b.schwarz\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright_impl_sync_base.py", line 109, in _sync return task.result() File "C:\Users\b.schwarz\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright_impl_page.py", line 352, in wait_for_selector return await self._main_frame.wait_for_selector(**locals_to_params(locals())) File "C:\Users\b.schwarz\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright_impl_frame.py", line 322, in wait_for_selector await self._channel.send("waitForSelector", locals_to_params(locals())) File "C:\Users\b.schwarz\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright_impl_connection.py", line 61, in send return await self._connection.wrap_api_call( File "C:\Users\b.schwarz\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright_impl_connection.py", line 482, in wrap_api_call return await cb() File "C:\Users\b.schwarz\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright_impl_connection.py", line 97, in inner_send result = next(iter(done)).result() playwright._impl._api_types.TimeoutError: Timeout 30000ms exceeded. =========================== logs =========================== waiting for locator("[data-testid=\"post-content\"]") to be visible

github-actions[bot] commented 2 months ago

This issue is stale because it has been open 7 days with no activity. Remove stale label or comment, or this will be closed in 10 days.

spaidevelopers commented 2 months ago

username resolved to 3 elements

I also got this issue and the problem is noted there-- when looking for the username, it found 3 elements. When you resolve username, you notice the same issue happens for password as well as similarly for the login button. To solve this issue, it's better to precisely target the username input where the login info will go to.

Replace the steps code at around line 82 in screenshot_downloader.py to:

...
    with sync_playwright() as p:
        print_substep("Launching Headless Browser...")
        browser = p.chromium.launch(
            headless=False
        )  # headless=False will show the browser for debugging purposes
        # Device scale factor (or dsf for short) allows us to increase the resolution of the screenshots
        # When the dsf is 1, the width of the screenshot is 600 pixels
        # so we need a dsf such that the width of the screenshot is greater than the final resolution of the video
        dsf = (W // 600) + 1

        context = browser.new_context(
            locale=lang or "en-us",
            color_scheme="dark",
            viewport=ViewportSize(width=W, height=H),
            device_scale_factor=dsf,
        )
        cookies = json.load(cookie_file)
        cookie_file.close()

        context.add_cookies(cookies)  # load preference cookies

        # Login to Reddit
        print_substep("Logging in to Reddit...")
        page = context.new_page()
        page.goto("https://www.reddit.com/login", timeout=0)
        page.set_viewport_size(ViewportSize(width=1920, height=1080))
        page.wait_for_load_state()
        page.wait_for_timeout(5000)  # Adjusted timeout for waiting

        # Wait for the username field to be ready and fill it in
        page.wait_for_selector("input#login-username")
        page.locator("input#login-username").fill(
            settings.config["reddit"]["creds"]["username"]
        )

        # Wait for the password field to be ready and fill it in
        page.wait_for_selector("input#login-password")
        page.locator("input#login-password").fill(
            settings.config["reddit"]["creds"]["password"]
        )

        # Click the login button using the updated selector based on the provided HTML structure
        login_button_selector = "button.login:has-text('Log In')"
        page.wait_for_selector(login_button_selector)
        page.click(login_button_selector)

        page.wait_for_timeout(5000)
...

This issue resolve but another issue arise after login playwright._impl._api_types.Error: TypeError: Cannot set properties of null (setting 'textContent')

image
nullagent07 commented 1 month ago

it

I changed the playwright script to be headless:fasle and fixed this issue while trying to get this running today. video_creation\screenshot_downloader.py

it worked at first but got this error after :

Launching Headless Browser...
Logging in to Reddit...
Skipping translation...
Something went wrong!
Something went wrong with making the screenshots! Do you want to skip the post? (y/n) n
Do you want the error traceback for debugging purposes? (y/n)y
┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Sorry, something went wrong with this version! Try again, and feel free to report this issue at GitHub or the Discord community.  │
│ Version: 3.2.1                                                                                                                    │
│ Error: Timeout 30000ms exceeded.                                                                                                  │
│ =========================== logs ===========================                                                                      │
│ waiting for locator("[data-test-id=\"post-content\"]")                                                                            │
│ ============================================================                                                                      │
│ Config: {'allow_nsfw': False, 'theme': 'light', 'times_to_run': 1, 'opacity': 0.9, 'storymode': False, 'storymodemethod': 1,      │
│ 'storymode_max_length': 1000, 'resolution_w': 1080, 'resolution_h': 1920, 'zoom': 1.0, 'background': {'background_video':         │
│ 'minecraft', 'background_audio': 'lofi', 'background_audio_volume': 0.15, 'enable_extra_audio': False, 'background_thumbnail':    │
│ False, 'background_thumbnail_font_family': 'arial', 'background_thumbnail_font_size': 96, 'background_thumbnail_font_color':      │
│ '255,255,255'}, 'tts': {'voice_choice': 'tiktok', 'random_voice': True, 'elevenlabs_voice_name': 'Bella', 'elevenlabs_api_key':   │
│ 'REDACTED', 'aws_polly_voice': 'Matthew', 'streamlabs_polly_voice': 'Matthew', 'tiktok_voice': 'en_us_001', 'tiktok_sessionid':   │
│ 'REDACTED', 'python_voice': '1', 'py_voice_num': '2', 'silence_duration': 0.3, 'no_emojis': False}}                               │
└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
Traceback (most recent call last):
  File "C:\Users\DIR\Desktop\RedditVideoMakerBot\main.py", line 131, in <module>
    raise err
  File "C:\Users\DIR\Desktop\RedditVideoMakerBot\main.py", line 113, in <module>
    run_many(config["settings"]["times_to_run"])
  File "C:\Users\DIR\Desktop\RedditVideoMakerBot\main.py", line 68, in run_many
    main()
  File "C:\Users\DIR\Desktop\RedditVideoMakerBot\main.py", line 52, in main
    get_screenshots_of_reddit_posts(reddit_object, number_of_comments)
  File "C:\Users\DIR\Desktop\RedditVideoMakerBot\video_creation\screenshot_downloader.py", line 203, in get_screenshots_of_reddit_posts
    raise e
  File "C:\Users\DIR\Desktop\RedditVideoMakerBot\video_creation\screenshot_downloader.py", line 185, in get_screenshots_of_reddit_posts
    page.locator('[data-test-id="post-content"]').screenshot(path=postcontentpath)
  File "C:\Users\DIR\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\sync_api\_generated.py", line 17568, in screenshot
    self._sync(
  File "C:\Users\DIR\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_sync_base.py", line 109, in _sync
    return task.result()
  File "C:\Users\DIR\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_locator.py", line 529, in screenshot
    return await self._with_element(
  File "C:\Users\DIR\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_locator.py", line 115, in _with_element
    handle = await self.element_handle(timeout=timeout)
  File "C:\Users\DIR\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_locator.py", line 315, in element_handle
    handle = await self._frame.wait_for_selector(
  File "C:\Users\DIR\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_frame.py", line 322, in wait_for_selector
    await self._channel.send("waitForSelector", locals_to_params(locals()))
  File "C:\Users\DIR\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_connection.py", line 61, in send
    return await self._connection.wrap_api_call(
  File "C:\Users\DIR\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_connection.py", line 482, in wrap_api_call
    return await cb()
  File "C:\Users\DIR\AppData\Local\Programs\Python\Python310\lib\site-packages\playwright\_impl\_connection.py", line 97, in inner_send
    result = next(iter(done)).result()
playwright._impl._api_types.TimeoutError: Timeout 30000ms exceeded.
=========================== logs ===========================
waiting for locator("[data-test-id=\"post-content\"]")
============================================================

Did you fix it?

Isaakkamau commented 4 weeks ago

https://github.com/elebumm/RedditVideoMakerBot/issues/1971#issuecomment-2008390470 That has solved the first error but now getting:

Launching Headless Browser...
Logging in to Reddit...
Navigating to thread URL...
Checking for NSFW post...
Checking for interest popup...
Skipping translation...
Taking screenshot of the post content...
Downloading screenshots... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% -:--:--Taking screenshot of comment 0...
Downloading screenshots... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% -:--:--
╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Sorry, something went wrong with this version! Try again, and feel free to report this issue at GitHub or the Discord community.            │
│ Version: 3.2.1                                                                                                                              │
│ Error: Timeout 30000ms exceeded.                                                                                                            │
│ =========================== logs ===========================                                                                                │
│ waiting for locator("#t1_l8vpb4q")                                                                                                          │
│ ============================================================                                                                                │
│ Config: {'allow_nsfw': False, 'theme': 'dark', 'times_to_run': 1, 'opacity': 0.9, 'storymode': False, 'storymodemethod': 1,                 │
│ 'storymode_max_length': 1000, 'resolution_w': 1080, 'resolution_h': 1920, 'zoom': 1.0, 'background': {'background_video': 'minecraft',      │
│ 'background_audio': 'lofi', 'background_audio_volume': 0.15, 'enable_extra_audio': False, 'background_thumbnail': False,                    │
│ 'background_thumbnail_font_family': 'arial', 'background_thumbnail_font_size': 96, 'background_thumbnail_font_color': '255,255,255'},       │
│ 'tts': {'voice_choice': 'tiktok', 'random_voice': True, 'elevenlabs_voice_name': 'Bella', 'elevenlabs_api_key': 'REDACTED',                 │
│ 'aws_polly_voice': 'Matthew', 'streamlabs_polly_voice': 'Matthew', 'tiktok_voice': 'en_us_001', 'tiktok_sessionid': 'REDACTED',             │
│ 'python_voice': '1', 'py_voice_num': '2', 'silence_duration': 0.3, 'no_emojis': False}}                                                     │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Traceback (most recent call last):
  File "/Users/adm/Desktop/RedditVideoMakerBot/main.py", line 132, in <module>
    raise err
  File "/Users/adm/Desktop/RedditVideoMakerBot/main.py", line 114, in <module>
    run_many(config["settings"]["times_to_run"])
  File "/Users/adm/Desktop/RedditVideoMakerBot/main.py", line 69, in run_many
    main()
  File "/Users/adm/Desktop/RedditVideoMakerBot/main.py", line 53, in main
    get_screenshots_of_reddit_posts(reddit_object, number_of_comments)
  File "/Users/adm/Desktop/RedditVideoMakerBot/video_creation/screenshot_downloader.py", line 270, in get_screenshots_of_reddit_posts
    page.locator(f"#t1_{comment['comment_id']}").screenshot(
  File "/opt/anaconda3/envs/yuna-ai-env/lib/python3.11/site-packages/playwright/sync_api/_generated.py", line 17568, in screenshot
    self._sync(
  File "/opt/anaconda3/envs/yuna-ai-env/lib/python3.11/site-packages/playwright/_impl/_sync_base.py", line 109, in _sync
    return task.result()
           ^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/yuna-ai-env/lib/python3.11/site-packages/playwright/_impl/_locator.py", line 529, in screenshot
    return await self._with_element(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/yuna-ai-env/lib/python3.11/site-packages/playwright/_impl/_locator.py", line 115, in _with_element
    handle = await self.element_handle(timeout=timeout)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/yuna-ai-env/lib/python3.11/site-packages/playwright/_impl/_locator.py", line 315, in element_handle
    handle = await self._frame.wait_for_selector(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/yuna-ai-env/lib/python3.11/site-packages/playwright/_impl/_frame.py", line 322, in wait_for_selector
    await self._channel.send("waitForSelector", locals_to_params(locals()))
  File "/opt/anaconda3/envs/yuna-ai-env/lib/python3.11/site-packages/playwright/_impl/_connection.py", line 61, in send
    return await self._connection.wrap_api_call(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/yuna-ai-env/lib/python3.11/site-packages/playwright/_impl/_connection.py", line 482, in wrap_api_call
    return await cb()
           ^^^^^^^^^^
  File "/opt/anaconda3/envs/yuna-ai-env/lib/python3.11/site-packages/playwright/_impl/_connection.py", line 97, in inner_send
    result = next(iter(done)).result()
             ^^^^^^^^^^^^^^^^^^^^^^^^^
playwright._impl._api_types.TimeoutError: Timeout 30000ms exceeded.
=========================== logs ===========================
waiting for locator("#t1_l8vpb4q")
============================================================
RequieMa commented 2 weeks ago

Issue still exists

Sorry, something went wrong with this version! Try again, and feel free to report this issue at GitHub or the Discord community. │ │ Version: 3.3.0 │ │ Error: Locator.screenshot: Timeout 30000ms exceeded. │ │ Call log: │ │ waiting for locator("h1#post-title-t3_1dob2zv") │ │ │ │ Config: {'allow_nsfw': True, 'theme': 'dark', 'times_to_run': 1, 'opacity': 0.9, 'storymode': False, 'storymodemethod': 1, 'storymode_max_length': 1000, 'resolution_w': 1080, │ │ 'resolution_h': 1920, 'zoom': 1.0, 'channel_name': 'Reddit Tales', 'background': {'background_video': 'minecraft', 'background_audio': 'lofi', 'background_audio_volume': 0.15, │ │ 'enable_extra_audio': False, 'background_thumbnail': False, 'background_thumbnail_font_family': 'arial', 'background_thumbnail_font_size': 96, 'background_thumbnail_font_color': │ │ '255,255,255'}, 'tts': {'voice_choice': 'googletranslate', 'random_voice': True, 'elevenlabs_voice_name': 'Bella', 'elevenlabs_api_key': 'REDACTED', 'aws_polly_voice': 'Matthew', │ │ 'streamlabs_polly_voice': 'Matthew', 'tiktok_voice': 'en_us_001', 'tiktok_sessionid': 'REDACTED', 'python_voice': '1', 'py_voice_num': '2', 'silence_duration': 0.3, 'no_emojis': False}}

RequieMa commented 2 weeks ago

https://github.com/elebumm/RedditVideoMakerBot/issues/1971#issuecomment-2008390470 did not work for me.

I changed '[data-test-id="post-content"]' to f'[data-fullname="t3_{reddit_id}"]' fixed the first error. Can someone check if it works for others?

Also, after fixing the first error, I encounter https://github.com/elebumm/RedditVideoMakerBot/issues/1971#issuecomment-2171892679 this one

Sorry, something went wrong with this version! Try again, and feel free to report this issue at GitHub or the Discord community. │ │ Version: 3.3.0 │ │ Error: Locator.screenshot: Timeout 30000ms exceeded. │ │ Call log: │ │ waiting for locator("#t1_la6u99b") │ │ │ │ Config: {'allow_nsfw': True, 'theme': 'dark', 'times_to_run': 1, 'opacity': 0.9, 'storymode': False, 'storymodemethod': 1, 'storymode_max_length': 1000, 'resolution_w': 1080, │ │ 'resolution_h': 1920, 'zoom': 1.0, 'channel_name': 'Reddit Tales', 'background': {'background_video': 'minecraft', 'background_audio': 'lofi', 'background_audio_volume': 0.15, │ │ 'enable_extra_audio': False, 'background_thumbnail': False, 'background_thumbnail_font_family': 'arial', 'background_thumbnail_font_size': 96, 'background_thumbnail_font_color': │ │ '255,255,255'}, 'tts': {'voice_choice': 'googletranslate', 'random_voice': True, 'elevenlabs_voice_name': 'Bella', 'elevenlabs_api_key': 'REDACTED', 'aws_polly_voice': 'Matthew', │ │ 'streamlabs_polly_voice': 'Matthew', 'tiktok_voice': 'en_us_001', 'tiktok_sessionid': 'REDACTED', 'python_voice': '1', 'py_voice_num': '2', 'silence_duration': 0.3, 'no_emojis': False}}

RequieMa commented 2 weeks ago

I finally get it work by replacing

'[data-test-id="post-content"]' to f'[data-fullname="t3_{reddit_id}"]' page.locator(f"#t1_{comment['comment_id']}") to page.locator(f"#thing_t1_{comment['comment_id']}").locator("form").first

Can you check if they work for yours?

LatteMill commented 2 weeks ago

[data-test-id="post-content"]

which folder ?

natefillups commented 1 day ago

I finally get it work by replacing

'[data-test-id="post-content"]' to f'[data-fullname="t3_{reddit_id}"]' page.locator(f"#t1_{comment['comment_id']}") to page.locator(f"#thing_t1_{comment['comment_id']}").locator("form").first

Can you check if they work for yours?

I was able to run this with a try catch to cover both cases in the UI, it works locally but when I pushed to a docker image and run it seems to only work 50% of the time. This does not work for all edge cases in the headless reddit UI. Is there another headless UI case that we aren't accounting for?