[IDEA] Converting dbc to sql and reading from there [$170 awarded]

[Yehonal]

Description: The idea behind it is to use together to (or just replacing) the system of dbc reader, an sql reader that just replace information of DBCs. For some limited cases it has been already done.

It allows following enhancements: 1) Fixing bugs related to the fact that dbc are a subset of wow db designed for client data caching/validation. Not specific for server. And could contains wrong information in some cases.

2) Allowing customizations that in most cases don't need a client change (areatriggers for example)

3) Allowing AzerothCore to be a bit more unrelated from wow game client since you don't have to extract such information anymore. Moving forward the MMO Framework project.

Who is interested in such project please contact me and we can work together for a PR (i don't have so much time to do this alone)

Current behaviour: Core actually reads some data information from dbc

Expected behaviour: We should convert dbc to sql and read information from DB.

Branch(es): master

--- The **[$170 bounty](https://www.bountysource.com/issues/48667762-idea-converting-dbc-to-sql-and-reading-from-there?utm_campaign=plugin&utm_content=tracker%2F40032087&utm_medium=issues&utm_source=github)** on this issue has been claimed at [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F40032087&utm_medium=issues&utm_source=github).

[Yehonal]

About this project, we've to remember that the core already offer the possibility to load from database without create any query but just adding a parameter LoadDBC function:

https://github.com/azerothcore/azerothcore-wotlk/blob/c9b7c75e1e67111b0d37471cc61b09b83b0ae977/src/server/game/DataStores/DBCStores.cpp#L203

So we've to do 3 things essentially:

1) create the sql table 2) define the custom format in DBCfmt.h (ex: https://github.com/azerothcore/azerothcore-wotlk/blob/c9b7c75e1e67111b0d37471cc61b09b83b0ae977/src/server/game/DataStores/DBCfmt.h#L11) 3) Pass this custom format to LoadDBC function (ex: https://github.com/azerothcore/azerothcore-wotlk/blob/c9b7c75e1e67111b0d37471cc61b09b83b0ae977/src/server/game/DataStores/DBCStores.cpp#L259)

And it should do the job

Then we've just to adapt the code allowing to load sql only and not requiring dbc files

[BarbzYHOOL]

There is a video here about this kind of system http://www.ac-web.org/forums/showthread.php?214767-SmartSpellScripting-Develop-any-spell-without-C-Restarts-or-Compiles!

@Yehonal

[Kaev]

Do you really want to stay with the ugly DBC way?

I mean who knows what this does when you never worked with it? https://github.com/azerothcore/azerothcore-wotlk/blob/master/src/server/game/DataStores/DBCfmt.h#L10

I would properly port the DBCs to classes, load and cache the values at startup and implement reload commands for it. This would already allow what Barbz posted in most ways.

I can do this over the next one or two weeks if you're fine with what i wrote.

[Yehonal]

Kaev i did not understand what you want to do exactly, however we've to remove dbc dependency because they are related to W0W client. Our project is going to become a generic MMO server this is why we have to remove those client data dependencies. In fact we already offer a webclient that is capable to login and chat on azerothcore: http://www.azerothcore.org/demo

Moreover it allows us to customize some data as you know. However, since dbc are a set of data, we should use the sql way without any hardcoded data in code.

Try to explain better what you have in mind :)

[Kaev]

I prefer to implement them as structs/classes and load them like e.g. ItemTemplate than adding the random string "niiiiiii" to the sourcecode. :P

[Yehonal]

random? lol it indicates the value type of each field.

However yes i prefere it too and if you are able to do it succesfully would be really nice! (but there are tons of DBCs, it's why i preferred to reuse already existing method...i'm lazy )

[Yehonal]

It's very old issue but I would like to know if you've considered to complete it @Kaev

I've seen that you've been very good to convert items

would be nice to create a dbc converter using c++ and our dbc definitions inside azerothcore, what do you think? if you need I can help you with it

[Helias]

If there is anyone interested in this issue I can help him to convert the DBC into SQL and with the implementation.

[Kaev]

It's very old issue but I would like to know if you've considered to complete it @Kaev

I've seen that you've been very good to convert items

would be nice to create a dbc converter using c++ and our dbc definitions inside azerothcore, what do you think? if you need I can help you with it

I 've checked DBCfmt.h and converted all available DBCs to SQL files. You can find these here: https://github.com/Kaev/AzerothcoreDBCToSQL These are all in english but i don't think we need any translations for these because the client already has them. I currently don't have the time to write the C++ classes, loaders, reload commands etc. I guess someone could create issues for each dbc and flag them with good first issue. It's not hard, just alot of work. Maybe my other item and area trigger to SQL PRs could be used as an example for new contributors. :)

[FrancescoBorzi]

@Kaev excellent, I left a comment with a small suggestion

[Yehonal]

Amazing @Kaev! Can you also share the code that you've used to convert dbc to sql using DBCfmt.h definitions? I'll appreciate it if you do it.

However, I've noticed we've such code inside our code: https://github.com/azerothcore/azerothcore-wotlk/blob/master/src/server/game/DataStores/DBCStores.cpp#L208

That is used to load some tables from our database to use some custom data where DBCs miss data.

We could rewrite it a bit allowing us to use this method by default and eventually allow an user to use DBC for legacy reasons (for example servers that are using modded DBCs and they have to convert them first)

[Kaev]

Amazing @Kaev! Can you also share the code that you've used to convert dbc to sql using DBCfmt.h definitions? I appreciate if you'll do it.

However, I've noticed we've such code inside our code: https://github.com/azerothcore/azerothcore-wotlk/blob/master/src/server/game/DataStores/DBCStores.cpp#L208

That is used to load some tables from our database to use some custom data where DBCs miss data.

We could rewrite it a bit allowing us to use this method by default and eventually allow an user to use DBC for legacy reasons (for example servers that are using modded DBCs and they have to convert them first)

The last time i used a self written C# tool that doesn't exist anymore. This time i just used a slightly modified version of the WDBX editor. It already contains the function to export DBCs as SQL files. https://github.com/WowDevTools/WDBXEditor

[Yehonal]

Job offer for this issue here: https://github.com/azerothcore/forum/issues/98

[Kitzunu]

The question I got for this project is when we convert all dbc to SQL, should we really do it in acore_world or should we have a specific database just for the DBC's? Like acore_dbc?

[Yehonal]

No, we don't need another database and we don't need to fill those table by default either

Those tables must be there allowing people to create their set of data or just customize the dbc. The cases are essentially the following:

1) tables empty, dbc installed (classic way) 2) tables with some custom data + dbc installed (to easily customize dbc that are not needed to be installed in the client too) 3) no dbc installed and personal data inside those tables (you're using azerothcore for your MMO project)

[grabthar]

@Kitzunu I think your idea good. Having dbc tables separated in a different database with all those dbc. The world have to much tables, and they are all related to client data? What i mean here is they are not like itemtemplate you can't add some entries without modify the client, but i could be wrong. Maybe a acore_client is more apropriate for tables wich require client modification?