postaddictme / instagram-php-scraper

Get account information, photos, videos, stories and comments.
https://packagist.org/packages/raiym/instagram-php-scraper
MIT License
3.1k stars 800 forks source link

Getting error 400 Bad request #670

Closed hamzadiaz closed 4 years ago

hamzadiaz commented 4 years ago

It was working very well till later today I started to get this error:

Fatal error: Uncaught InstagramScraper\Exception\InstagramAuthException: Response code is 400. Body: Bad request Something went wrong. Please report issue. in /home2/hamzadiaz/public_html/instagram/vendor/raiym/instagram-php-scraper/src/InstagramScraper/Instagram.php:1664 Stack trace: #0 /home2/hamzadiaz/public_html/instagram/profile.php(26): InstagramScraper\Instagram->login() #1 {main} thrown in /home2/hamzadiaz/public_html/instagram/vendor/raiym/instagram-php-scraper/src/InstagramScraper/Instagram.php on line 1664

I tried using different Instagram account for login but getting same error. Here is what line 1664 Instagram.php of code has:

} elseif ((is_string($response->code) || is_numeric($response->code)) && is_string($response->body)) {
    throw new InstagramAuthException('Response code is ' . $response->code . '. Body: ' . $response->body . ' Something went wrong. Please report issue.', $response->code);
}

Can't figure out how to get it fixed, Maybe Instagram made some update?

Thanks in advance

movAX13h commented 4 years ago

I get the same (already updated to the latest commit) ... since yesterday. I guess there was an update on the IG page somewhere. `Fatal error: Uncaught InstagramScraper\Exception\InstagramAuthException: Response code is 400. Body: Bad request Something went wrong. Please report issue. in /var/www/html/tools/ig/src/InstagramScraper/Instagram.php:1559\nStack trace:

0 /var/www/html/tools/ig/index.php(42): InstagramScraper\Instagram->login()

1 {main}

thrown in /var/www/html/tools/ig/src/InstagramScraper/Instagram.php on line 1559`

gianfolli commented 4 years ago

I have exactly same problem here

eldark commented 4 years ago

Same problem. Nothing to add to the report.

pgrimaud commented 4 years ago

We need to find a way to generate enc_password form field value.

Got the same error on my scraper too : https://github.com/pgrimaud/instagram-user-feed/issues/65

I found this, which could help us : https://www.bountysource.com/issues/87593270-add-enc_password-header-for-requests-version

I'm actually working on it. If someone has good knowledge about crypto in JavaScript (AES), please contact me.

hamzadiaz commented 4 years ago

We need to find a way to generate enc_password form field value.

Got the same error on my scraper too : pgrimaud/instagram-user-feed#65

I found this, which could help us : https://www.bountysource.com/issues/87593270-add-enc_password-header-for-requests-version

I'm actually working on it. If someone has good knowledge about crypto in JavaScript (AES), please contact me.

Hello @pgrimaud could this work?

function encrypt($key, $data) {
    $encryptionKey = base64_decode($key);
    $iv = openssl_random_pseudo_bytes(openssl_cipher_iv_length('aes-256-gcm'));
    $encrypted = openssl_encrypt($data, 'aes-256-gcm', $encryptionKey, 0, $iv, $tag);
    return base64_encode($encrypted . '::' . $iv . '::' . $tag);
}

function decrypt($key, $data) {
    $encryptionKey = base64_decode($key);
    list($encryptedData, $iv, $tag) = explode('::', base64_decode($data), 3);
    return openssl_decrypt($encryptedData, 'aes-256-gcm', $encryptionKey, 0, $iv, $tag);
}

$encrypt = encrypt('key', 'test data');
$decrypt = decrypt('key', $encrypt);

echo $encrypt . ' : ' . $decrypt;

It is basically encryption and decryption of aes-256-gcm with php using openssl, but don't know if it can be integrated in the scrapper?

Source: https://stackoverflow.com/questions/52477247/trying-to-decrypt-with-aes-256-gcm-with-php

pgrimaud commented 4 years ago

Hi @hamzadiaz

Yeah, I read lot of documentation about AES 256 GCM, but I can't find any public key available.

The data I could found are given into this page : https://www.instagram.com/data/shared_data/

"encryption": {
  "key_id": "87",
  "public_key": "8dd9aad29d9a614c338cff479f850d3ec57c525c33b3f702ab65e9e057fc087e",
  "version": "9"
}

Someone adapted JavaScript into Python here : https://github.com/instaloader/instaloader/issues/615#issuecomment-629435096 As you can see, new encryption seems a little bit more tricky than a simple AES encryption.

pgrimaud commented 4 years ago

I tried this shit : https://gist.github.com/pgrimaud/3d2b2f090a7121ba28509db646f3eecb

It has the same byte length as the JavaScript encryption script, but I doesn't work when I used it for login.

I'm gonna be crazy πŸ™‚πŸ”«

ibnux commented 4 years ago

i try to change variable password to enc_password

https://github.com/postaddictme/instagram-php-scraper/blob/6503db29f6bfbf36bbd281da88f73fda279033f5/src/InstagramScraper/Instagram.php#L1540-L1548

$headers = [
                'ig-set-password-encryption-web-key-id' => 33,
                'ig-set-password-encryption-web-pub-key' => '0d85544e7b7b43e0e4465d0a35690925cc7c53c04dc07a6fbffdb45be2718c52',
                'ig-set-password-encryption-web-key-version' => 10,
                'cookie' => $cookieString,
                'referer' => Endpoints::BASE_URL . '/',
                'x-csrftoken' => $csrfToken,
                'X-CSRFToken' => $csrfToken,
                'user-agent' => $this->getUserAgent(),
            ];
            $response = Request::post(Endpoints::LOGIN_URL, $headers,
                ['username' => $this->sessionUsername, 'enc_password' => $this->sessionPassword,'optIntoOneTap'=>false]);

but got respond like this

{"code":400,"raw_body":"{\"message\": \"checkpoint_required\", \"checkpoint_url\": \"\/challenge\/6242737647\/GMoK4dsqH9\/\", \"lock\": false, \"status\": \"fail\"}","body":{"message":"checkpoint_required","checkpoint_url":"\/challenge\/6242737647\/GMoK4dsqH9\/","lock":false,"status":"fail"}

using encoded password from browser login

and then confirmation show up

Jepretan Layar 2020-05-17 pukul 19 10 30
Tom1380 commented 4 years ago

I'm having this same problem.

pgrimaud commented 4 years ago

I published a workaround on my scrapper. It seems working for me, some people could try it : https://github.com/pgrimaud/instagram-user-feed/commit/96ad4cf54d1ad331b337f325c73e664999a6d066 ?

Just clone my repository, edit this file and run it.

If it will conclusive, I will open a PR on this repository.

Thanks for tests and feedback πŸ˜„

hamzadiaz commented 4 years ago

I published a workaround on my scrapper. It seems working for me, some people could try it : pgrimaud/instagram-user-feed@96ad4cf ?

Just clone my repository, edit this file and run it.

If it will conclusive, I will open a PR on this repository.

Thanks for tests and feedback πŸ˜„

Thank you so much @pgrimaud!!!!! I changed Login() function of Instagram.php file to your approach and now I no longer get error 400 Bad Request:

 $headers = [
                'cookie' => $cookieString,
                'referer' => Endpoints::BASE_URL . '/',
                'x-csrftoken' => $csrfToken,
                'X-CSRFToken' => $csrfToken,
                'user-agent' => $this->getUserAgent(),
            ];
            $response = Request::post(Endpoints::LOGIN_URL, $headers,
                ['username' => $this->sessionUsername, 'enc_password' => '#PWD_INSTAGRAM_BROWSER:0:' . time() . ':' . $this->sessionPassword]);
eldark commented 4 years ago

I published a workaround on my scrapper. It seems working for me, some people could try it : pgrimaud/instagram-user-feed@96ad4cf ? Just clone my repository, edit this file and run it. If it will conclusive, I will open a PR on this repository. Thanks for tests and feedback πŸ˜„

Thank you so much @pgrimaud!!!!! I changed Login() function of Instagram.php file to your approach and now I no longer get error 400 Bad Request:

 $headers = [
                'cookie' => $cookieString,
                'referer' => Endpoints::BASE_URL . '/',
                'x-csrftoken' => $csrfToken,
                'X-CSRFToken' => $csrfToken,
                'user-agent' => $this->getUserAgent(),
            ];
            $response = Request::post(Endpoints::LOGIN_URL, $headers,
                ['username' => $this->sessionUsername, 'enc_password' => '#PWD_INSTAGRAM_BROWSER:0:' . time() . ':' . $this->sessionPassword]);

DId you do anything else besides editing that part? Because I still get errors.

hamzadiaz commented 4 years ago

I published a workaround on my scrapper. It seems working for me, some people could try it : pgrimaud/instagram-user-feed@96ad4cf ? Just clone my repository, edit this file and run it. If it will conclusive, I will open a PR on this repository. Thanks for tests and feedback πŸ˜„

Thank you so much @pgrimaud!!!!! I changed Login() function of Instagram.php file to your approach and now I no longer get error 400 Bad Request:

 $headers = [
                'cookie' => $cookieString,
                'referer' => Endpoints::BASE_URL . '/',
                'x-csrftoken' => $csrfToken,
                'X-CSRFToken' => $csrfToken,
                'user-agent' => $this->getUserAgent(),
            ];
            $response = Request::post(Endpoints::LOGIN_URL, $headers,
                ['username' => $this->sessionUsername, 'enc_password' => '#PWD_INSTAGRAM_BROWSER:0:' . time() . ':' . $this->sessionPassword]);

DId you do anything else besides editing that part? Because I still get errors.

Just that part and this is how I log in:

$instagram = \InstagramScraper\Instagram::withCredentials('user', 'pass', new Psr16Adapter('Files'));
$instagram->login(); 
$instagram->saveSession(); 
jangrewe commented 4 years ago

I pulled the latest version, but unfortunately i still get an exception - it's for a 200 now, though...

<b>Fatal error</b>:  Uncaught InstagramScraper\Exception\InstagramException: Response code is 200. Body: 
&lt;!DOCTYPE html&gt;
&lt;html lang=&quot;en&quot; class=&quot;no-js logged-in client-root touch&quot;&gt;
    &lt;head&gt;
        &lt;meta charset=&quot;utf-8&quot;&gt;
        &lt;meta http-equiv=&quot;X-UA-Compatible&quot; content=&quot;IE=edge&quot;&gt;

        &lt;title&gt;
Instagram
&lt;/title&gt;
[...]
in <b>[REDACTED]/vendor/raiym/instagram-php-scraper/src/InstagramScraper/Instagram.php</b> on line <b>385</b><br />
jangrewe commented 4 years ago

If i comment out the throw in line 385, it now works for me, too.

eldark commented 4 years ago

Strange. I tried everything suggested since the temporary fix, but no luck.

ibnux commented 4 years ago

I published a workaround on my scrapper. It seems working for me, some people could try it : pgrimaud/instagram-user-feed@96ad4cf ? Just clone my repository, edit this file and run it. If it will conclusive, I will open a PR on this repository. Thanks for tests and feedback πŸ˜„

Thank you so much @pgrimaud!!!!! I changed Login() function of Instagram.php file to your approach and now I no longer get error 400 Bad Request:

 $headers = [
                'cookie' => $cookieString,
                'referer' => Endpoints::BASE_URL . '/',
                'x-csrftoken' => $csrfToken,
                'X-CSRFToken' => $csrfToken,
                'user-agent' => $this->getUserAgent(),
            ];
            $response = Request::post(Endpoints::LOGIN_URL, $headers,
                ['username' => $this->sessionUsername, 'enc_password' => '#PWD_INSTAGRAM_BROWSER:0:' . time() . ':' . $this->sessionPassword]);

it works using this

But I need to create new account

Old account always ask confirmation everytime script running

eldark commented 4 years ago

Now I'm getting this error: `PHP Fatal error: Uncaught InstagramScraper\Exception\InstagramAuthException: Something went wrong. Please report issue. in D:\Work\Projects\apps\instagram-php-scraper\vendor\raiym\instagram-php-scraper\src\InstagramScraper\Instagram.php:1561 Stack trace:

0 D:\Work\Projects\apps\instagram-php-scraper\getAccountMediasByUsername.php(146): InstagramScraper\Instagram->login()

1 {main}

thrown in D:\Work\Projects\apps\instagram-php-scraper\vendor\raiym\instagram-php-scraper\src\InstagramScraper\Instagram.php on line 1561`

Tried with a few valid usernames, and even changing my IP. No luck.

bocanhcam commented 4 years ago

I published a workaround on my scrapper. It seems working for me, some people could try it : pgrimaud/instagram-user-feed@96ad4cf ?

Just clone my repository, edit this file and run it.

If it will conclusive, I will open a PR on this repository.

Thanks for tests and feedback πŸ˜„

This is working now. But just prepare for future cause it's version 10 now. Is there anyone good at nodejs can convert this one to PHP will be helpful cause I think it work

https://github.com/dilame/instagram-private-api/pull/1052/files#diff-aba9581ea6872bcf7c1aef75405c8783R77

public encryptPassword(password: string): { time: string, encrypted: string } {
    const randKey = crypto.randomBytes(32);
    const iv = crypto.randomBytes(12);
    const rsaEncrypted = crypto.publicEncrypt({
      key: Buffer.from(this.client.state.passwordEncryptionPubKey, 'base64').toString(),
      // @ts-ignore
      padding: crypto.constants.RSA_PKCS1_PADDING,
    }, randKey);
    const cipher = crypto.createCipheriv('aes-256-gcm', randKey, iv);
    const time = Math.floor(Date.now() / 1000).toString();
    cipher.setAAD(Buffer.from(time));
    const aesEncrypted = Buffer.concat([cipher.update(password, 'utf8'), cipher.final()]);
    const sizeBuffer = Buffer.alloc(2, 0);
    sizeBuffer.writeInt16LE(rsaEncrypted.byteLength, 0);
    const authTag = cipher.getAuthTag();
    return {
      time,
      encrypted: Buffer.concat([
        Buffer.from([1, this.client.state.passwordEncryptionKeyId]),
        iv,
        sizeBuffer,
        rsaEncrypted, authTag, aesEncrypted])
        .toString('base64'),
    };
  }
ibnux commented 4 years ago

for checkpoint_required

Short solution is to create new account

i search on the internet to fix challenge, it need to Temporarily disable account, i do this for old account, and maybe wait 3days or 1 week, before enabling again.

eldark commented 4 years ago

for checkpoint_required i search on the internet to fix challenge, it need to Temporarily disable account, i do this for old account, and maybe wait 3days or 1 week, before enabling again.

Did this work?

ibnux commented 4 years ago

@eldark Nope, it doesnt work, today i activate my account, try to login, and get checkpoint_url again.

i think we need to scrape checkpoint_url too

pgrimaud commented 4 years ago

i think we need to scrape checkpoint_url too

Sure, I will work on it this next week-end

eldark commented 4 years ago

I created #676 for that purpose. Please, do tell if you need help. And thanks for replying!

aldrin431 commented 4 years ago

@eldark Nope, it doesnt work, today i activate my account, try to login, and get checkpoint_url again.

i think we need to scrape checkpoint_url too

I followed your recommendation and it's fine for me.

Steps

  1. Create a new email account
  2. Create an Instagram account (never wrote a phone in my configuration)
  3. Just I'm following 3 accounts
  4. Try to login
jangrewe commented 4 years ago

Has anybody of you tried what i mentioned before, commenting out the throwing of the exception? It kinda looks like it doesn't make sense to have that there... and it's working perfectly fine for me since then, with the existing account.

amaelftah commented 4 years ago

am still getting same exception after pulling the latest package version. and here's a debug from $userArray

am using getAccount('some Instagram username') without login

would appreciate if someone can help

Screenshot from 2020-06-02 12-28-27

aldrin431 commented 4 years ago

am still getting same exception after pulling the latest package version. and here's a debug from $userArray

am using getAccount('some Instagram username') without login

would appreciate if someone can help

Screenshot from 2020-06-02 12-28-27

Yesterday I had the same problem but with another code, I used getAccount of this scraper with login and it's working.

I think now we should make all request with login.

amaelftah commented 4 years ago

@aldrin431 not sure if it a login problem . as on my local machine i use it without login and it works but on dev & production server it doesn't work

also there's #685 related to this

eldark commented 4 years ago

I believe it's a rate limiting problem. Instagram's rate limiting, that is.

aldrin431 commented 4 years ago

@aldrin431 not sure if it a login problem . as on my local machine i use it without login and it works but on dev & production server it doesn't work

also there's #685 related to this

Yes I have the same problem, when I work in my localhost I work without login and in production I make the login.

zaden1 commented 4 years ago

I have been getting the 200 error as well. I tried to comment out line 385 and I even though I no longer get the error I also can not access any images. This is what I get on sentry

Response code is 200. Body: <!DOCTYPE html>

Login β€’ Instagram
  • asolopovas commented 4 years ago

    +1

    csan-dev commented 4 years ago

    +1

    filippodicostanzo commented 4 years ago

    +1

    roninit72 commented 4 years ago

    +1

    pckz commented 4 years ago

    same for me. please someone say something :/

    hamzadiaz commented 4 years ago

    If you get these errors try using a new Instagram account and login with: $instagram->login(true); $instagram->saveSession();

    Hope it helps.

    Mehrdad-Dadkhah commented 4 years ago

    +1

    ibnux commented 4 years ago

    i found a solution, i change User Agent to another, and it works

    https://github.com/postaddictme/instagram-php-scraper/blob/c9df4728c2b8a38e46bedea0c458bb6d9073f1f5/src/InstagramScraper/Instagram.php#L50

    i change to Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:77.0) Gecko/20100101 Firefox/77.0

    jangrewe commented 4 years ago

    @ibnux: Read the docs, there's a better way to do this.

    $instagram->setUserAgent($userAgent);

    I thought it was kinda well-known by now that you have to set the user-agent string to whatever string is used by the client you use to register/login/unblock logins/etc. when you do it manually. ;-)

    ibnux commented 4 years ago

    @jangrewe hehe sorry i never thought UA important, after getting this error again, i try to change it, and it works,

    just debugging thing, sometimes try to hardcode πŸ˜…

    thanks for mention the function πŸ˜ƒ

    stale[bot] commented 4 years ago

    This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.