search

roxlukas / lmeve

LMeve - industry manager and contribution tracker for EVE Online
https://pozniak.pl/wp/
Other
51 stars 13 forks source link

Too Many Items #85

Closed erikterr closed 5 years ago

erikterr commented 5 years ago

o7, we receive that error on our page: ESI error on route /v1/corporations/xxxxxx/assets/names/: {"error":"too many items for 'item_ids', 'item_ids' is required"}

roxlukas commented 5 years ago

Most likely a bug. LMeve will have to check how many itemids it wants to resolve in one transaction, much like using x-pages to fetch large outputs. I will ask you about more details when I get back home.

erikterr commented 5 years ago

no problem, I'm also at work :P

On Tue, Feb 26, 2019 at 3:20 PM Łukasz Poźniak notifications@github.com wrote:

Most likely a bug. LMeve will have to check how many itemids it wants to resolve in one transaction, much like using x-pages to ferch large outputs. I will ask you about more details when I get back home.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/roxlukas/lmeve/issues/85#issuecomment-467455968, or mute the thread https://github.com/notifications/unsubscribe-auth/Aaes3aK2-h5I4qWUk1dOkbK2xOGkZeghks5vRUKrgaJpZM4bSRE9 .

-- Terrenzani Erik From Gmail

roxlukas commented 5 years ago

I have added additional logging in <lmeve>/bin/Assets.php

Please update the file, remove <lmeve>/var/poller.lock if it exists, and let the poller try again.

Then please look for the following line in <lmeve>/var/poller.txt

Getting xxxx names from ESI...

Let me know how many items it wants to query.

erikterr commented 5 years ago

Here the error from command line: [INFORM] Assets: Getting 1039 names from ESI... [INFORM] Route: New Route->request(POST) [INFORM] Route: constructURL()='https://esi.evetech.net/v1/corporations/xxx/assets/names/?datasource=tranquility [INFORM] Route: constructCacheFilename()='/opt/lmeve/bin/../var/7_96354865f2b14b1a7c293eddd03a87df.json [WARNING] Route: Too many errors when accessing Route /v1/corporations/xxx/assets/names/, skipping this route. PHP Warning: count(): Parameter must be an array or an object that implements Countable in /opt/lmeve/bin/Assets.php on line 89 PHP Warning: Invalid argument supplied for foreach() in /opt/lmeve/bin/Assets.php on line 90 [INFORM] Killmails: Updating Killmails...

Seem to be it fails during a parse of a contract: 25.02.2019 17:10:50 [CLI]: [WARNING] /v1/corporations/: ERROR 404: {"error":"Contract not found!"} After some errors it give "Too many errors when accessing Route /v1/corporations/, skipping this route."

roxlukas commented 5 years ago

That explains it.

LMeve tries to resolve 1039 names, but ESI only allows batches of 1000:

post_corporations_corporation_id_assets_names_item_ids[ maxItems: 1000 minItems: 1 title: post_corporations_corporation_id_assets_names_item_ids uniqueItems: true

item_ids array integer($int64) title: post_corporations_corporation_id_assets_names_item_id

item_id integer ]

The other errors are related to name resolution failing and LMeve trying to prevent causing too many errors with ESI, by shutting down all routes that returned mor ethan 10 errors.

roxlukas commented 5 years ago

The plan for a fix is simple, iterate name resolution in batches of 1000, exactly the same way as x-pages support is currently implemented.

erikterr commented 5 years ago

ok how to do? I'm not too much php skilled :)

roxlukas commented 5 years ago

Don't worry, I'll write a patch for Assets.php and let you know when it's ready to update :-) I just wanted you to know the fix is coming :-)

erikterr commented 5 years ago

Rgr tnx a lot. If no needed I'll remove txt log

roxlukas commented 5 years ago

Sure, you can remove it

roxlukas commented 5 years ago

Fixed in becbbe1 - please check if it fixed your problem

erikterr commented 5 years ago

Still receive same error:

[INFORM] Assets: Getting 1080 Asset names from ESI... [INFORM] Assets: Getting page 0 [INFORM] Route: New Route->request(POST) [INFORM] Route: constructURL()=' https://esi.evetech.net/v1/corporations/xxx/assets/names/?datasource=tranquility [INFORM] Route: constructCacheFilename()='/opt/lmeve/bin/../var/7_96354865f2b14b1a7c293eddd03a87df.json [WARNING] Route: Too many errors when accessing Route /v1/corporations/98297004/assets/names/, skipping this route. PHP Warning: array_merge(): Argument #1 is not an array in /opt/lmeve/bin/Assets.php on line 86 [INFORM] Assets: Getting page 1 [INFORM] Route: New Route->request(POST) [INFORM] Route: constructURL()=' https://esi.evetech.net/v1/corporations/xxx/assets/names/?datasource=tranquility [INFORM] Route: constructCacheFilename()='/opt/lmeve/bin/../var/7_96354865f2b14b1a7c293eddd03a87df.json [WARNING] Route: Too many errors when accessing Route /v1/corporations/xxx/assets/names/, skipping this route. PHP Warning: array_merge(): Argument #1 is not an array in /opt/lmeve/bin/Assets.php on line 86 PHP Warning: count(): Parameter must be an array or an object that implements Countable in /opt/lmeve/bin/Assets.php on line 96 [INFORM] Killmails: Updating Killmails... [INFORM] Killmails: Getting Killmails... [INFORM] Route: New Route->request(GET) [INFORM] Route: constructURL()=' https://esi.evetech.net/v1/corporations/xxx/killmails/recent/?datasource=tranquility [INFORM] Route: constructCacheFilename()='/opt/lmeve/bin/../var/7_81e166842e6c610effa4e0bd2b9bd9a5.json [WARNING] Route: Too many errors when accessing Route /v1/corporations/, skipping this route. bool(false) [INFORM] Killmails: Route /v1/corporations/xxx/killmails/recent/?datasource=tranquility is still cached, skipping...

On Fri, Mar 1, 2019 at 11:13 AM Łukasz Poźniak notifications@github.com wrote:

Fixed in becbbe1 https://github.com/roxlukas/lmeve/commit/becbbe11a62f031efdddc241c5cacee635da454e

  • please check if it fixed your problem

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/roxlukas/lmeve/issues/85#issuecomment-468614821, or mute the thread https://github.com/notifications/unsubscribe-auth/Aaes3dGPOOn36sapsOOxM0HhrDcT63Zzks5vSP1RgaJpZM4bSRE9 .

-- Terrenzani Erik From Gmail

erikterr commented 5 years ago

rectify, after another run (the first took 20sec for run, that one took 400!) seem to be is working

erikterr commented 5 years ago

Confirm problem solved, you can close the bug