barncastle
commented
4 years ago My first remark is that I'd like to get a definition for every build we possibly can even if it is just meta dumps before moving on to v2. This will 1. give us a good base for version 2 so we can focus on the spec more than the data and 2. should increase the rate of adoption. This should also be a very achievable considering we have the tools now.
Adoption: This is quite a hard one to tackle but I'd say the biggest targets that need to be brought across are the server cores themselves. Most tools out there, especially in the modding community, are using Trinity, AzerothCore or (C)Mangos as their main source of reference and rightly so because they are the literal heart of the community. As such most column names, formats and readers are being built in-line with what these projects' implementations. If we could approach these projects and get their input and adoption this would trickle down and increase adoption rates. This also applies to all major/well known projects such as simc and Stoneharry's spell editor.
In regards to the modding community, the biggest issues are favouritism and project longevity. Most modding is happening on WotLK/Vanilla cores as either they are nostalgic favourites and/or their project has spanned several years. At the moment we aren't catering for them however we must do so if we want adoption to increase - ideally we should focus on getting end-of-expansion definitions 100% correct. The unfortunate side-effect is that most modders were raised using abandonware such as MyDBCEditor and WDBX and continue to propagate it to newcomers. As these tools are abandoned we can't target them directly but we could maybe maintain a set of profiles, DBD branded, that these tools support to introduce people into the DBD ecosphere until another tool (like DBMonster or DBClientFiles.Net) that supports DBD comes in to replace them?
In regards to parsers in other languages - this will happen organically once the format is adopted by more people, so I don't feel this is something we need to focus on just yet - something for v3 maybe?
Version 2 changes:
-
My biggest OCD gripe at the moment is the lack of standardisation of the layout of the definitions themselves. This is only minor but I'd like to adhere to a standard that orders all definitions by newest to oldest and also uses build ranges as much as possible. These 60+ build lists are really off putting to say the least and merging branches has been a nightmare because everything keeps getting re-ordered. Ideally the automated dump would just increment the final build of the range as we're capturing every build that goes out.
-
I'm completely onboard with having an index file for file identification.
-
The big topic and focus of v2 (imo) is of course enums and flags. At the moment we're using comments as a quick fix for wow.tools however this isn't ideal. Looking at the original discussion the general consensus was to keep DBD as clean as possible so I'd like to suggest having an additional associated file (even
<dbdname>.dat) that stores a range of additional information including this data. How this is formatted is up for debate I'd prefer to have a hex - name pair that we commonly use on the wiki ideally but open for anything else. I'm not sure if Blizz reuse enums or whatnot so versioning might be something we need to look into but we could just reuse the build/build-range/layout spec we've already adopted? -
I'd also like to throw in storing metadata-only data too such as the
Flagsfor WDB3 and the default values for WDB6 common fields and potentially TACT keys (* see the end) since sections aren't always decrypted. The metadata data should really go in maindbdas this is a parsing requirement however TACT keys should probably go into the counterpart data file. -
Something @bloerwald brought up too is adding another keyword for what is (poorly) named
secondary indicesakaflags & 0x2- the relationship column for pre-WDC which definitely needs recording! -
For #40: I think we should approach this with symbolic notation as per point 1 here. Namely for the sake of being as true to format as possible and not misleading people with invalid cardinalities when the layouthash doesn't change.
-
For #51: I think the best choice is to move foreign keys into the definitions themselves like we have for the bitwidths. It does increase the amount of text but it gives much more control over how we handle this data.
Implementation/migration Personally I'd work across two branches instead of having two folders as this is the point of git. Also, if we went the route of having an additional data counterpart file we could even work on the same repo with minimal fuss.
* I've been thinking for a while that maybe we should create a gist or repo that replicate's the wow.tools tactkey page that doesn't hit the site.
Shauren
Marlamin
billy1arm
Last month, DBD turned 2 years old. 🎉 It's been quite the ride so far! We're still in the middle of things and will hopefully soon (update: now merged) merge in the pre-wod branch @barncastle has been hard at work on as well as to continue working through the backlog of 6.x/7.x versions we're missing.
Adoption
Adoption still isn't at the point we initially wanted it to be. Why is this? Is it the lack of interest in WoW tool development or is the format too annoying to deal with? Not enough libraries out there to deal with DBD? What can we do to improve this?
Version 2 changes
Throughout this time I'm sure some of you have grown to miss/dislike some things in the current format and with Shadowlands Beta slowly coming to an end changes-wise, I thought this would be a good time to start discussing a possible v2 of the format. Hopefully we can get to a good proposal that we can spend some time on implementing when things calm down in a few months.
A good starting point might be some of the things that were left in the initial format discussion, such as adding enums and flags to the format. I currently deal with these by manually keeping them up to date in the WoW.tools repo, would be cool if we can somehow integrate these into the format if enough people agree. The same goes for flags. No idea how we should handle conditional enums/flags, though.
One thing I'd like us see add (and have discussed previously) is adding a file that combines the tablehash, filedataid and name of DB2s as a way to deal with unnamed DB2s. This idea is based on @MMOSimca's
TableHash.cfg, but with theFileDataIDadded too for if the file is unnamed and to possibly help readers with loading them from CASC.Also, there are still some open issues such as #40 and #51 that we should tackle here if they need format changes.
Implementation/migration
I'd suggest starting off with a new directory, maybe named "v2" instead of definitions. How to handle DBDv1 going forward is up for debate, but the easiest route when we do switch to primarily v2 will probably be to have some GitHub action run DBDefsConverter which will output v2 -> v1 or something.
Disagree with something? Did I miss something? Post below. Issue will remain open for quite some time until we come to a general consensus like last time.