search

pyfa-org / Pyfa

Python fitting assistant, cross-platform fitting tool for EVE Online
GNU General Public License v3.0
1.61k stars 406 forks source link

Better handling of prices #1342

Closed blitzmann closed 6 years ago

blitzmann commented 6 years ago

Need to take a look at exactly the response we are getting back from the EVE Central api (which is currently down). We should fall back to eve-marketdata if central fails. This should prevent all the emails I'm getting about not being able to fetch prices >.<. Should also take a look at fixing #1334 up while in there.

minlexx commented 6 years ago

@blitzmann did you think about getting prices directly from ESI? It has an api call for fetching all market orders in a region, optionally specifying type ID of the item. I've recently did this for my own eve related project and I think it works :)

Class for getting prices is here - https://github.com/minlexx/whdbx_web/blob/master/classes/eve_price_resolver.py#L222 It uses helper from https://github.com/minlexx/whdbx_web/blob/master/classes/esi_calls.py#L410

I think this code should be portable to py2 and can be added to pyfa without much efforts... I might even try to prepare a pull request some time in the future if you won't do it yourself first.. :)

P.S. need to find where is my pyfa & py2 development environment was ...

blitzmann commented 6 years ago

@blitzmann did you think about getting prices directly from ESI? It has an api call for fetching all market orders in a region, optionally specifying type ID of the item. I've recently did this for my own eve related project and I think it works :)

I did look at it briefly, and it is something I want to do eventually. I have a couple projects that I want to get done first that are more pressing though, especially since we have functioning price sources to begin with.

I did take a brief look and found two endpoints: the average price across the universe, and the orders per region.

The average price for the universe might be interesting, but would require a huge revamping of how prices get fetched (instead of fetching them individually when needed, we would have to fetch them in the background on startup), and I'm not sure of the performance impact of updating all the prices at once.

The one for the region - also may require a bit more tweaking since pyfa gets prices based on system, not region. We could always get the orders for a region and limit the results based on system (if station locationID is in Jita), but that's additional API calls. Or, just simply specify in the preferences what kind of fetch the source does (for example, eve-marketdata (system), ESI (region)). The bigger problem that I noticed for the regional orders is that you can't specify a collection of type IDs, Rather you have to get them one at a time. At least that's the way it looks in the swagger interface, didn't actually test it.

For now the two third-party sources work (well, two if evecentral ever comes back up). If you're interested in trying to tackle this for ESI, I'm open to PRs - will need to evaluate the performance impact of getting all orders for the region and calculating a good price point for each type we send in. The price sources are kind of plugin based - you create a class and register it with the Price service - it's __init__ is supplied a list of types to get prices for, and from there we can do whatever we need to for that source. I don't think we would need to handle the ESI cache timeout ourselves since the pyfa price objects already handle that, and developing / maintaining another caching system will be a bit too much methinks. :)

EDIT: Actually, IIRC a conversation I had with eve-marketdata's dev team, I believe that eve-marketdata even uses ESI as its datasource, and provides APIs to others that are more useful to devs (such as selecting by system, specifying multiple types, etc). So, in a way, we already use ESI :) Food for thought.