Issues with Facebook and Instagram link

[mekkkwiz]

I am encountering issues with a bot designed to scrape Facebook and Instagram and post content to Discord. Despite adding necessary configurations in the Dockerfile, there are warnings and failures affecting the bot's performance.

Steps to Reproduce

1. Run the bot in a Docker environment with the following configurations:
   • COPY instagram.sess ./
   • COPY facebook_cookies.txt ./
   • Define INSTAGRAM_USERNAME in the environment variables.
2. Use a Facebook link to scrape content.
3. Use an Instagram link to scrape content.

Expected Behavior

• The bot should scrape Facebook and Instagram content without warnings or errors.
• The bot should post videos from Facebook and Instagram to Discord.

Actual Behavior

1. For Facebook links:
   • Warning: WARNING:facebook_scraper.page_iterators:No raw posts (<article> elements) were found in this page.
   • Videos are not shown on Discord.
2. For Instagram links:
   • Videos are not shown on Discord and have message Failed downloading on Discord.

Logs:

2024-08-02 22:44:11,797 WARNING No raw posts (<article> elements) were found in this page.
2024-08-02 22:44:12,250 INFO User น้อนสามม sent message with url https://www.facebook.com/reel/1037758174583208

Loaded session from instagram.sess.
JSON Query to api/v1/users/web_profile_info/?username=c81maluo6lc: 404 Not Found when accessing https://i.instagram.com/api/v1/users/web_profile_info/?username=c81maluo6lc [retrying; skip with ^C]
JSON Query to api/v1/users/web_profile_info/?username=c81maluo6lc: 404 Not Found when accessing https://i.instagram.com/api/v1/users/web_profile_info/?username=c81maluo6lc [retrying; skip with ^C]
2024-08-02 23:05:38,295 ERROR Failed downloading https://www.instagram.com/reels/C81mALuO6lC/: Profile c81maluo6lc does not exist.
2024-08-02 23:05:39,010 ERROR Ignoring exception in on_message
Traceback (most recent call last):
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.12/site-packages/instaloader/instaloadercontext.py", line 443, in get_json
    raise QueryReturnedNotFoundException(self._response_error(resp))
instaloader.exceptions.QueryReturnedNotFoundException: 404 Not Found when accessing https://i.instagram.com/api/v1/users/web_profile_info/?username=c81maluo6lc

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.12/site-packages/instaloader/instaloadercontext.py", line 443, in get_json
    raise QueryReturnedNotFoundException(self._response_error(resp))
instaloader.exceptions.QueryReturnedNotFoundException: 404 Not Found when accessing https://i.instagram.com/api/v1/users/web_profile_info/?username=c81maluo6lc

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.12/site-packages/instaloader/instaloadercontext.py", line 443, in get_json
    raise QueryReturnedNotFoundException(self._response_error(resp))
instaloader.exceptions.QueryReturnedNotFoundException: 404 Not Found when accessing https://i.instagram.com/api/v1/users/web_profile_info/?username=c81maluo6lc

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.12/site-packages/instaloader/structures.py", line 949, in _obtain_metadata
    metadata = self._context.get_iphone_json(f'api/v1/users/web_profile_info/?username={self.username}',
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.12/site-packages/instaloader/instaloadercontext.py", line 601, in get_iphone_json
    response = self.get_json(path, params, 'i.instagram.com', tempsession, response_headers=response_headers)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.12/site-packages/instaloader/instaloadercontext.py", line 469, in get_json
    return self.get_json(path=path, params=params, host=host, session=sess, _attempt=_attempt + 1,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.12/site-packages/instaloader/instaloadercontext.py", line 469, in get_json
    return self.get_json(path=path, params=params, host=host, session=sess, _attempt=_attempt + 1,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.12/site-packages/instaloader/instaloadercontext.py", line 457, in get_json
    raise QueryReturnedNotFoundException(error_string) from err
instaloader.exceptions.QueryReturnedNotFoundException: JSON Query to api/v1/users/web_profile_info/?username=c81maluo6lc: 404 Not Found when accessing https://i.instagram.com/api/v1/users/web_profile_info/?username=c81maluo6lc

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.12/site-packages/discord/client.py", line 449, in _run_event
    await coro(*args, **kwargs)
  File "/app/bots/discord/client.py", line 73, in on_message
    raise e
  File "/app/bots/discord/client.py", line 66, in on_message
    post = await client.get_post()
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/downloader/instagram.py", line 64, in get_post
    return self._get_profile()
           ^^^^^^^^^^^^^^^^^^^
  File "/app/downloader/instagram.py", line 111, in _get_profile
    profile = instaloader.Profile.from_username(context=self.client.context, username=self.id)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.12/site-packages/instaloader/structures.py", line 880, in from_username
    profile._obtain_metadata()  # to raise ProfileNotExistsException now in case username is invalid
    ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.12/site-packages/instaloader/structures.py", line 966, in _obtain_metadata
    raise ProfileNotExistsException('Profile {} does not exist.'.format(self.username)) from err
instaloader.exceptions.ProfileNotExistsException: Profile c81maluo6lc does not exist.

Additional Information:

Bot version: 0.20.3 Docker version: 4.32.0 (157355) Operating System: windows 11

Attachments:

[amadejkastelic]

Hi, for Facebook, the app expects cookies file to be named cookies.txt. For Instagram, it seems like your username does not exist. You need to specify the username that you log in with. You can also just skip this step and not provide the instagram session and username, which should work for all public and not age restricted posts.

[mekkkwiz]

Thank you for your help! The issue has been resolved.