Determine strategy for game data

[kalilistic]

Determine how we want to retrieve game data (xivapi, kobold) and if/how we should centrally package this.

It probably should be, but I don't think we discussed that earlier, so I erred on the side of potential duplication for now. We should discuss what we want to do here as a team.

_Originally posted by @karashiiro in https://github.com/the-convocation/venat/pull/44#discussion_r873203649_

[karashiiro]

Summarizing my responses to the PR reviews, I think this should be determined on a dependency-by-dependency basis, with things like XIVAPI (kobold when) being more likely to be extracted into a module than things like Universalis, which have more narrow use cases. Additionally, we should set things up to minimize breaking changes to modules if we decide to start using a new dependency for something (kobold), with a strategy in place for a gradual migration if the APIs aren't compatible.

[kalilistic]

I'll update this issue to focus on XIVAPI or alternatives.

[kalilistic]

Adding a critical priority label as I've run into the need for this too with the lodestone module.

[MKhayle]

I don't know if that's relevant to current case scenario but if at some point we're going for some PHP environment for any reason (dashboard?) there is this project ongoing to fill a postgres database with the game's data, but we theoretically won't need everything. Else, we can, of course, go for XIVAPI.

[ArcaneDisgea]

We can use xivapi for basically anything that's not lodestone. Only caveat is when game patches we'd have to wait on xivapi to be updated.

[karashiiro]

I think we should just use XIVAPI and Cafemaker for now, and migrate to Ironworks later. If we want to standardize game data access across plugins, we'll need some sort of common interface that can support both. Populating a SQL database with game tables is an interesting idea, though. SQL is a natural fit for relational data like this.

[karashiiro]

Actually, why hasn't anyone made a tool to pull all of the game tables into a few SQLite databases (one for each spoken language) for use in any programming language? In retrospect, that seems obvious. SQLite (or insert major database here) is also an engine that has years of development behind it, so I'd trust it to be more reliable and efficient than anything we've developed now.

[ArcaneDisgea]

Is that something we want to take ownership of and create? If so we prolly want to add a todo to the project board and/or migrate this from an issue to a discussion.

[kalilistic]

Makes sense. Off of Kara's recommendation, I'd say two issues: 1) make generic xiv api utility(?) class (to do now) 2) update/create ironworks or database (to do later)

[avafloww]

I'd also be willing to make Thaliak auto-generate these databases for each applicable game version, if a tool to generate said databases from a XIV install existed. We could then have the bot automatically download XIV database updates as needed.