search

NicklasWallgren / PokemonGoAPI-PHP

Pokemon Go API PHP library
BSD 2-Clause "Simplified" License
130 stars 51 forks source link

Incorrect wire format for field 3 #160

Closed zunfix closed 7 years ago

zunfix commented 7 years ago

Hi,

I have now this error 👍

Fatal error: Uncaught Exception: Incorrect wire format for field 3, expected: 0 got: 2 in C:\WTServer\WWW\pogo\vendor\nicklasw\pogoprotos-php\src\POGOProtos\Data\PokedexEntry.php:56 Stack trace:

0 C:\WTServer\WWW\pogo\vendor\nicklasw\pogoprotos-php\src\protocolbuffers.inc.php(84): POGOProtos\Data\PokedexEntry->read(Resource id #165, 13)

1 C:\WTServer\WWW\pogo\vendor\nicklasw\pogoprotos-php\src\POGOProtos\Data\PokedexEntry.php(25): ProtobufMessage->__construct(Resource id #165, 13)

2 C:\WTServer\WWW\pogo\vendor\nicklasw\pogoprotos-php\src\POGOProtos\Inventory\InventoryItemData.php(70): POGOProtos\Data\PokedexEntry->__construct(Resource id #165, 13)

3 C:\WTServer\WWW\pogo\vendor\nicklasw\pogoprotos-php\src\protocolbuffers.inc.php(84): POGOProtos\Inventory\InventoryItemData->read(Resource id #165, -12)

4 C:\WTServer\WWW\pogo\vendor\nicklasw\pogoprotos-php\src\POGOProtos\Inventory\InventoryItemData.php(30): ProtobufMessage->__construct(Resource id #165, -12)

5 C:\WTServer\WWW\pogo\vendor\nicklasw\pogoprotos-php\src\POGO in C:\WTServer\WWW\pogo\vendor\nicklasw\pogoprotos-php\src\POGOProtos\Data\PokedexEntry.php on line 56

I think this is related to update : 2016. 11. 6 : Pokémon GO updated to version 0.45.0 for Android and 1.15.0 for iOS. Probably new fields for daily bonus.

Xavier.

sedziwoj commented 7 years ago

It be possible to make this script not crash then they add new field? I read this protocol should works this way.

xmillies commented 7 years ago

I try to comment the exception without any success. => another exception 'Incorrect wire format' => and error on response (if I remember).

I try to add log without success. In vendor vendor\nicklasw\pkm-go-api\src\Handlers\RequestHandler.php I have added : in line 280 :

file_put_contents (date('Y-m-d_H-i-s.u').' Response.raw',$response->getBody()->getContents());

and use protoc but I see only first answer.

DrDelay commented 7 years ago

@sedziwoj is right, protobuf is supposed to allow adding new fields without breaking existing implementations.

Unfortunately, the PHP protobuf plugin we are using does not work this way, it just breaks and those Exceptions are being raised. There is a long (closed) issue about a similar problem: #142

The only way to fix the problem with this library is to identify and add the new fields, maybe it is sufficient that someone (@NicklasWallgren) updates the protobuf PHP classes in NicklasWallgren/pogoprotos-php by syncing and recompiling the protos from the upstream (AeonLucid/POGOProtos), it got quite a few updates in the last 3 days.

DrDelay commented 7 years ago

@xmillies if you want to dump the response you should modify the unmarshall method this way:

$content = (string) $response->getBody()->getContents();
file_put_contents(microtime(true).'.bin', $content);
// Unmarshall the response
$responseEnvelop->read($content);

Reason is that you can only call Psr\Http\Message\StreamInterface::getContents once, as the stream does not get rewinded after reading it. So the read called after your dump gets no more data (reading the remainder of a stream that has its pointer already at the end) and the program terminates.

DrDelay commented 7 years ago

@zunfix I'd appreciate if you could dump the protobuf response that causes the error and attach it here, as I'm not able to reproduce this requesting the Pokémon of my account.

sedziwoj commented 7 years ago

1478728794.3604.bin.txt decode.txt

I have same error:

PHP Fatal error: Uncaught Exception: Incorrect wire format for field 3, expected: 0 got: 2 in /home/krzysiek/pgo2/vendor/nicklasw/pogoprotos-php/src/POGOProtos/Data/PokedexEntry.php:56

I add as you ask, and decode using

protoc --decode_raw

zunfix commented 7 years ago

Maybe a bit late ...

Here the latest protbuf response.

Raw : 1478739423.2167.bin.txt

Decoded : 1478739423.2167.txt

NicklasWallgren commented 7 years ago

Updated the protos files, still experience some issues:

Fatal error: Uncaught Exception: Incorrect wire format for field 3, expected: 0 got: 2 in /home/vagrant/pkm-go-api/source/vendor/nicklasw/pogoprotos-php/src/POGOProtos/Data/Player/PlayerStats.php on line 73

Exception: Incorrect wire format for field 3, expected: 0 got: 2 in /home/vagrant/pkm-go-api/source/vendor/nicklasw/pogoprotos-php/src/POGOProtos/Data/Player/PlayerStats.php on line 73
NicklasWallgren commented 7 years ago

Most of the issues has been resolved (manually patched). Have another go.

https://github.com/AeonLucid/POGOProtos/issues/251

zunfix commented 7 years ago

After composer update

Loading composer repositories with package information Updating dependencies (including require-dev)

  • Removing nicklasw/pogoprotos-php (dev-master 14e2e7f)
  • Installing nicklasw/pogoprotos-php (dev-master ba880fd) Cloning ba880fd84f0a6a3bd84a7e2ca72e0c03dc4b697d

I get the followinfg trace :

Fatal error: Uncaught Exception: new \POGOProtos\Data\Player\PlayerStats did not read the full length in C:\WTServer\WWW\pogo\vendor\nicklasw\pogoprotos-php\src\POGOProtos\Inventory\InventoryItemData.php:83 Stack trace:

DrDelay commented 7 years ago

I am now getting this reading the response dumps provided here aswell as with my own account: new \POGOProtos\Data\Player\PlayerStats did not read the full length

With jaspervdm/pogoprotos-php there are no problems once again. I already tried to migrate this API to it once, but there are quite some things to change in the code :crying_cat_face:.

DrDelay commented 7 years ago

I don't think that we are missing a field, it fails for me after parsing this POGOProtos.Data.Player.PlayerStats:

1: 2
2: 1920
4: 3000
5: 0x3f9737c7
6: 3
7: 3
8: 3
10: 2
11: 4
22: "\000\002\000\001\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"

Using jaspervdm/pogoprotos-php to parse it I get:

#level: 2
#experience: 1920
#prev_level_xp: null
#next_level_xp: 3000
#km_walked: 1.1813896894455
#pokemons_encountered: 3
#unique_pokedex_entries: 3
#pokemons_captured: 3
#evolutions: null
#poke_stop_visits: 2
#pokeballs_thrown: 4
#eggs_hatched: null
#big_magikarp_caught: null
#battle_attack_won: null
#battle_attack_total: null
#battle_defended_won: null
#battle_training_won: null
#battle_training_total: null
#prestige_raised_total: null
#prestige_dropped_total: null
#pokemon_deployed: null
#pokemon_caught_by_type: Protobuf\ScalarCollection {#19
  flag::STD_PROP_LIST: false
  flag::ARRAY_AS_PROPS: false
  storage: array:19 [
    0 => 0
    1 => 2
    2 => 0
    3 => 1
    4 => 0
    5 => 0
    6 => 0
    7 => 0
    8 => 0
    9 => 0
    10 => 0
    11 => 0
    12 => 0
    13 => 0
    14 => 0
    15 => 0
    16 => 0
    17 => 0
    18 => 0
  ]
}
#small_rattata_caught: null

I added some debugging to our protos, and get this:

Initial Len passed to PlayerStats: 45
Processing PlayerStats field 1
Processing PlayerStats field 2
Processing PlayerStats field 4
Processing PlayerStats field 5
Processing PlayerStats field 6
Processing PlayerStats field 7
Processing PlayerStats field 8
Processing PlayerStats field 10
Processing PlayerStats field 11
Processing PlayerStats field 22
Wire type is 2
Reading array pos 0 ... value 0
Reading array pos 1 ... value 2
Reading array pos 2 ... value 0
Reading array pos 3 ... value 1
Reading array pos 4 ... value 0
Reading array pos 5 ... value 0
Reading array pos 6 ... value 0
Reading array pos 7 ... value 0
Reading array pos 8 ... value 0
Reading array pos 9 ... value 0
Reading array pos 10 ... value 0
Reading array pos 11 ... value 0
Reading array pos 12 ... value 0
Reading array pos 13 ... value 0
Reading array pos 14 ... value 0
Reading array pos 15 ... value 0
Reading array pos 16 ... value 0
Reading array pos 17 ... value 0
Reading array pos 18 ... value 0
Remaining length after field22: 0
Processing PlayerStats field 3
Len before fail is -54

It is actually correctly reading the array, what was my first guess of the failure-point, but after it tries to read a field 3 that doesn't even exist in the raw message, this reduces the $len (what was correctly 0), and leads to the Exception (if ($len !== 0) throw new \Exception('new \POGOProtos\Data\Player\PlayerStats did not read the full length');).

So, I suspect there is nothing wrong with the protos, but rather with the PHP implementation we are using. It mistakenly tries to read a non-existent field 3 after the last one, field 22. I don't know enough about protobuf, maybe there is a closing delimiter after this array-value that it is not reading and later on interprets it as another field?

xmillies commented 7 years ago

So where should we fix ? In PokemonGoAPI-PHP or in dependancies ?

baskettcase commented 7 years ago

Im also interested in this. Im just curious about my IVs and this is a show stopper.

NicklasWallgren commented 7 years ago

Can anyone provide me with a valid account? :) Send it to nicklas.wallgren@gmail.com

xmillies commented 7 years ago

I have send one.

NicklasWallgren commented 7 years ago

Thanks @xmillies.

I have re-compiled the pogo protos files using the official protocol buffer compiler. We still need to update this library to support the new files. https://github.com/NicklasWallgren/pogoprotos-php/tree/development

voxx commented 7 years ago

@NicklasWallgren @DrDelay Any chance either of you plan on taking a crack at revamping the API to use the new protos files? I'm still using this api for managing captchas for map purposes, but would love to use it again for managing inventory stuff.

Here's 0.51 btw.

NicklasWallgren commented 7 years ago

@voxx @DrDelay

I have got it working with the official protobuf library. Still waiting on a few bug fixed thought (https://github.com/google/protobuf/pull/2516) (https://github.com/google/protobuf/pull/2504) (https://github.com/google/protobuf/issues/2532).

Will probably release something in a few days :)

The profile data: NicklasW\PkmGoApi\Api\Player\Data\Profile\ProfileData Object
(
    [creationTime:protected] => 1479841849094
    [username:protected] => gulligtuss8
    [team:protected] => 0
    [tutorialState:protected] => NicklasW\PkmGoApi\Api\Player\Data\Profile\TutorialState Object
        (
            [state:protected] => Google\Protobuf\Internal\RepeatedField Object
                (
                    [container:Google\Protobuf\Internal\RepeatedField:private] => Array
                        (
                            [0] => 0
                            [1] => 1
                            [2] => 3
                            [3] => 4
                            [4] => 7
                        )

                    [type:Google\Protobuf\Internal\RepeatedField:private] => 14
                    [klass:Google\Protobuf\Internal\RepeatedField:private] => POGOProtos\Enums\TutorialState
                )

        )

    [avatar:protected] => NicklasW\PkmGoApi\Api\Player\Data\Profile\Avatar Object
        (
            [skin:protected] => 0
            [hair:protected] => 1
            [shirt:protected] => 0
            [pants:protected] => 2
            [hat:protected] => 3
            [shoes:protected] => 0
            [gender:protected] => 0
            [eyes:protected] => 4
            [backpack:protected] => 0
        )

    [pokemonStorage:protected] => 250
    [itemStorage:protected] => 350
    [dailyBonus:protected] => NicklasW\PkmGoApi\Api\Player\Data\Profile\DailyBonus Object
        (
            [nextCollectedTimestampMs:protected] => 
            [nextDefenderBonusCollectTimestampMs:protected] => 
        )

    [badge:protected] => NicklasW\PkmGoApi\Api\Player\Data\Profile\Badge Object
        (
            [badgeType:protected] => 
            [level:protected] => 0
            [nextEquipChangeAllowedTimestampMs:protected] => 
        )

    [contactSettings:protected] => NicklasW\PkmGoApi\Api\Player\Data\Profile\ContactSettings Object
        (
            [sendMarketingEmails:protected] => 
            [sendPushNotifications:protected] => 
        )

    [currencies:protected] => NicklasW\PkmGoApi\Api\Player\Data\Profile\Currencies Object
        (
            [currencies:protected] => Array
                (
                    [POKECOIN] => NicklasW\PkmGoApi\Api\Player\Data\Profile\Currency Object
                        (
                            [name:protected] => POKECOIN
                            [amount:protected] => 0
                        )

                    [STARDUST] => NicklasW\PkmGoApi\Api\Player\Data\Profile\Currency Object
                        (
                            [name:protected] => STARDUST
                            [amount:protected] => 1100
                        )

                )

        )

)
voxx commented 7 years ago

@NicklasWallgren

That's great news! Looking forward to the next release so I can start working on replacing the existing integration's I'm running. Let me know if you need any help testing!

-voxx

NicklasWallgren commented 7 years ago

@voxx I just released a first update, have look. Most of the examples should work, I haven't looked into the other request types yet.

NicklasWallgren commented 7 years ago

Can we close this issue?

xmillies commented 7 years ago

$playerStats = $inventory->getStats(); gives now an error :

Fatal error: Uncaught Error: Call to a member function getOptions() on null in C:\WTServer\WWW\pogo\vendor\google\protobuf\php\src\Google\Protobuf\descriptor.php:468 Stack trace: #0 C:\WTServer\WWW\pogo\vendor\google\protobuf\php\src\Google\Protobuf\Internal\Message.php(77): Google\Protobuf\Internal\FieldDescriptor->isMap() #1 C:\WTServer\WWW\pogo\vendor\nicklasw\pkm-go-api\src\Handlers\RequestHandler.php(221): Google\Protobuf\Internal\Message->__construct() #2 C:\WTServer\WWW\pogo\vendor\nicklasw\pkm-go-api\src\Handlers\RequestHandler.php(122): NicklasW\PkmGoApi\Handlers\RequestHandler->build(Object(NicklasW\PkmGoApi\Requests\GetInventoryRequest)) #3 C:\WTServer\WWW\pogo\vendor\nicklasw\pkm-go-api\src\Services\Request\InventoryRequestService.php(24): NicklasW\PkmGoApi\Handlers\RequestHandler->handle(Object(NicklasW\PkmGoApi\Requests\GetInventoryRequest)) #4 C:\WTServer\WWW\pogo\vendor\nicklasw\pkm-go-api\src\Api\Player\Inventory.php(59): NicklasW\PkmGoApi\Services\Request\InventoryRequestService->getInventory() #5 C:\WTS in C:\WTServer\WWW\pogo\vendor\google\protobuf\php\src\Google\Protobuf\descriptor.php on line 468
xmillies commented 7 years ago

$pokeBank = $inventory->getPokeBank(); gives same kind of error

xmillies commented 7 years ago

I tried just with the sample : RetrieveJournalExample.php to be sure that my code doesn't interfere .... but I get the " Call to a member function getOptions() on null ".

So we should close this case and open a new one. I did : #163

NicklasWallgren commented 7 years ago

This issue has been resolved. Thanks for the help