search

project-topaz / topaz

Server emulator for FFXI
Other
160 stars 224 forks source link

[discussion] Craft Recipe Information Sources #1182

Open ibm2431 opened 4 years ago

ibm2431 commented 4 years ago

Long story short (for now) - because some people are trashy I'm doing some work on recipes.

Major part of this is building a database of trustworthy crafting information. Absent a DAT scraper, I've been using information scraped from FFXIAH and BGWiki, procedurally melding it together via tools, and cross-checking the sources against each other.

I've generated the following lists of instances in which they don't agree: disagreements.zip (Apologies for the zip; the lists are HTML-ified for handy hyperlinks to the sources, and GitHub doesn't like straight attachment of HTML files.)

BGWiki and FFXIAH can disagree as to craft skill levels, result quantities, and even ingredient quantities.

Crafting level or skill disagreements:

"FFXIAH used to scrape the client, we can trust it"

Result quantity / HQ disagreements:

"Maybe just the results were changed by retail?"

Ingredient quantity disagreements:

We'll have to come up with a set of rules of which information to take when. It's not always one or the other.

BGWiki's information is more recent (with more recipes), and seems to always win in result quantity matchups.

BGWiki might suffer from ingredient quantity typos, but it's not always clear when. But ingredient quantities can be easily verified on retail. Example: I can check Lord's Cuirass myself next free period. Or people with access to retail can ask crafter friends to test.

My main concern is crafting skill level disagreements. Even if BGWiki is certain about a craft's level (they usually indicate when they're unsure), they can disagree with FFXIAH. And there are some situations in which FFXIAH is claiming a subcraft level which doesn't make sense: FFXIAH Austere Sabots - BGWIKI Austere Sabots Why would there be a subscraft level of 1? How would a BGWiki editor even know there was a subcraft if it was 1? This makes me believe someone was prevented from crafting the boots due to the subcraft truly being 11.

Finally, these aren't in a HTML list because I needed to manually remove them from XML (and there were only a handful), but FFXIAH even has some "dead recipes":

Why are there empty results? How might a DAT scraper have generated that recipe entry?

tl;dr: What should be trusted when?

I'd like as few "judgment calls" as possible. The ideal is that there's a set of rules we follow, and if anyone comes around in the future wondering why some recipe's information doesn't match one or the other, we can point them to this thread.

edit: Updated zip with ffxiah-only.html, which are recipes that have a result (ignoring quantity) and ingredient list with no corresponding recipe found on bgwiki.

ibm2431 commented 4 years ago

(But before someone says it: I don't consider FFXIclopedia as a source.)

zach2good commented 4 years ago

As a general rule, I think people are more likely to start a conversation (read: complain) when things are too hard, rather than when things are too easy. This conversation will result in things getting fixed, or more in-depth research being done.

What is more likely to make someone complain: A recipe being too low level to reach a certain skill level on, or it being too high so they can't start skilling up on it when they're supposed to?

neuromancerxi commented 4 years ago

I don't know where FFXIAH gets its recipes from, but they also do not seem to have stopped maintaining after a point. I'd speculate that some of them may have been culled from the .dats themselves originally (before recipes were moved server side). While FFXIAH isn't perfect, there is still data that can be gleaned there from it's user base.

Since recipe caps and sub skills are often inferred based on user experience, I'd trust BGwiki as a definitive source generally, but there have to be instances where we're willing to diverge if we can confirm otherwise. Adding comments in the PR and/or SQL would be helpful here.

I don't see a path where there can be perfect data unless some figures out how to automate collecting the data directly from retail.

So as a general path, my opinion is:

  1. Base the recipe on data available from BGWiki
  2. Inaccuracies/disputes should source another reliable community resource or sources (good) OR provide data directly from retail to support (best).
  3. In events where its clear that the data from BGWiki is wildly inaccurate, have a fail back to a final Judgement call.
ibm2431 commented 4 years ago

Since recipe caps and sub skills are often inferred based on user experience, I'd trust BGwiki as a definitive source generally

Agreed with this. People usually don't edit a page unless they have reason to believe it's wrong.

I was scrolling through the list again and saw another example I want to bring up: FFXIAH - Igqira Lappas BGWiki - Igqira Lappas

If FFXIAH was scraping, it's done so since when the Senbaak Nagan was added to the game. Missing the subcraft on Igqira of all things is very, very strange.

neuromancerxi commented 4 years ago

I'm not sure who's at the helm at FFXIAH these days. You might be able to reach out to Scragg to see if he can shed any light on how it's done or was done if you want to scratch that itch.

TeoTwawki commented 4 years ago

How would a BGWiki editor even know there was a subcraft if it was 1?

You would occasional get a skillup. and they skillups would cease once skilled up. But I think they example was clearly a mistake and was meant to say 11.

"FFXIAH used to scrape the client, we can trust it"

Unproven, there are a lot of indicators* but we don't know with certainty exactly how ffxi got its recipes.

*Like seeing the unused hidden duplicates in their lists and unreleased recipes with common SE placeholder values. Two fiddy five anyone?

TeoTwawki commented 4 years ago

I'm not sure who's at the helm at FFXIAH these days. You might be able to reach out to Scragg to see if he can shed any light on how it's done or was done if you want to scratch that itch.

I haven't been able to get hold of Scragg or Jaerik in 3+ yrs. I don't think he Scragg that inbox anymore. Best bet is probably to see if Rooks can tell us or contact someone who knows on our behalf.

ibm2431 commented 4 years ago

I've updated the zip with ffxiah-only.html, which are recipes that have a result and ingredient list with no corresponding recipe from bgwiki's main craft lists.

While some of the items in the new HTML file are result quantity mismatches, a lot of them are "FFXIAH Only" desynths which have no corresponding entry for the source item on BGWiki. Many don't have any crafting level for the desynth (see above Legionaire's Knuckles). But there are some FFXIAH recipes which do (FFXIAH Shall Shell Desynth), but still aren't listed on the source item for BGWiki (Shall Shell)

The Witch Nougat example has caught my attention. Both sources have different ingredients and results. Anyone with Cooking main on retail could see which is which (maybe try both?) - it looks like an easy recipe (may have to farm chips and almonds).

Finally, a nil error while trying to run the script did bring this fun recipe to light: FFXIAH Recipe 3821 Indeed there is, Scraag.

ibm2431 commented 4 years ago

In instances in which there are multiple item IDs with the same name, which do we take?

https://www.ffxiah.com/item/4399/bluetail https://www.ffxiah.com/item/5798/bluetail

https://www.ffxiah.com/item/4472/crayfish https://www.ffxiah.com/item/5785/crayfish

https://www.ffxiah.com/item/4428/dark-bass https://www.ffxiah.com/item/5786/dark-bass

The one being sold on FFXIAH? Is this always the lower ID? Which ones are we using?

This seems to be a problem unique to fish of all kinds.

edit: PUP attachments too. https://www.ffxiah.com/item/2268/mana-conserver https://www.ffxiah.com/item/8675/mana-conserver

And Quick Draw cards: https://www.ffxiah.com/item/2183/dark-card https://www.ffxiah.com/item/9771/dark-card

I'm going with the lowest ID, but this might be something to remember.

ibm2431 commented 4 years ago

Maybe not always the lower item ID.

https://www.ffxiah.com/item/16818/san-dorian-sword https://www.ffxiah.com/item/17678/san-dorian-sword

(The +2 version from Recipe 1578 is ID 17679, so I'm going with 17678 on this one.)

But at least my scripts have been catching these for manual verification!

TeoTwawki commented 4 years ago

that 2nd sandorian they never added to game. But works in private servers since we gave it a model and mods. there are a lot of unused items like that, I think SE has the duplicate fish for a diff reason.

neuromancerxi commented 4 years ago

We speculated that the secondary set of fish IDs might come from MMM since there's a whole fishing system in there. There is some empirical evidence of this in the following:

http://www.ffxiah.com/item/5809

Note that the next item ID in the block (http://www.ffxiah.com/item/5810) also has a reference to MMM in the comments (albeit from the same reporter).

ibm2431 commented 4 years ago

Finishing up the master XML, but for the record, I am thoroughly convinced that FFXIAH's recipes weren't from scraping client DATs, but from tracking synth results with Guildwork.

Four reasons: 1) Incorrect yields on many recipes (the x99 result being marked as the NQ when it should be x33), and in a couple cases, FFXIAH having the +1 of an item as the NQ (for a cooking recipe, not an "equipment upgrade"). This leads me to believe FFXIAH based its NQ information off of what came out of a Guildwork-running player doing the recipe. If it only ever sees the x99 result...

2) Zero key item requirements. At first I thought maybe they didn't know how to read KI requirements, but thinking about it from the "Guildwork tracking" perspective, how would they determine which key item is required for a recipe if they're using Guildwork data? You could get possessed key items on synth, but what are you going to do, slowly do process of elimination until you narrow it down to the one key item everyone has when they perform the synth? "Well, everyone who has ever done this synth has an Adventurer's Certificate, so it must be required for the recipe!" It'd be an impossible problem for them to solve with a "from Guildwork" approach, which makes me suspect it's the reason FFXIAH doesn't have KIs.

3) FFXIAH not having precise levels for all crafts in a recipe has been gnawing at me. It stating values of 255 for all crafts could have an argument that it was scraped before the recipe was "real". But there are cases when it has a value for the main craft, but "unknown" for one or both subcrafts. It wouldn't make sense for it to have been scraped in that case - scraped skill levels would be either all or nothing. Since it's instead "some", I think that instead FFXIAH/Guildwork had been tracking skillups and compiling that data for a recipe's "known" levels.

4) When the data stopped, as Teo previously commented on, as being around when Guildwork went bjorked.

That said, the FFXIAH data has still been useful. I've been using it to cross-verify against BGWiki, and will have a list of recipes in which the sources disagree on crystal/ingredients. And even if FFXIAH's skill levels weren't scraped from DATs, it still used an automated process across multiple players to compile craft levels and put the number online. I trust that a little more than a sole editor doing calculations and manually inputting their result on wiki page.

ibm2431 commented 4 years ago

Another small snag - there are times when FFXIAH and BGWiki disagree on the skill level for an item that's now one of the "Craft Kits" that retail has.

FFXIAH States 9, but Bonecraft Kit 10 - FFXIAH Recipe 453 - BGWiki Bone Arrowheads

FFXIAH States 28, but Bonecraft Kit 30 - FFXIAH Recipe 484 - BGWiki Beetle Mask

Three possibilities:

  1. "Maybe FFXIAH got the level wrong?"
  2. "Maybe the skill level was changed on the item after kits were introduced?"
  3. "Maybe SE just picked items for those kits that weren't the level stated and BGWiki is assuming equality based on the name?"

Retail now gets skillups past the original "skill cap" of the recipe, so it may be hard to tell that something in "Bonecraft Kit 30" is actually level 28.

edit: Beetle Mask was changed on BGWiki to level 30 by an editor when the kits were added. Before, it was level 28 and matched FFXIAH.

TeoTwawki commented 4 years ago

It stating values of 255 for all crafts could have

As I mentioned on discord could also just be that the record from guildwork got garbled in transit either on its way from the client or after the fact moving from gw to ffxiah to. But I was hopeful their "recipe id" wasn't just made up. :(

jarmengaud commented 4 years ago

It's not that difficult to find out, you just spam Beetle mask until you cap (the probability of skillup is high enough so you can confidently determine that you reached the cap).