Configuration, Setup AoWoW

Sarjuuk / aowow

Database viewer for TrinityCore based on aowow by @LordJZ, based on the JS-Engine of Wowhead

204 stars 217 forks source link

Configuration, Setup AoWoW #427

Closed virgo77 closed 4 months ago

virgo77 commented 4 months ago

Hi @Sarjuuk :) me again ^^

So, I don't know if you remember; but earlier this year I opened a topic about difficulties I went through to install this DB system. I think I'm in good progress as I can launch the setup. I finally know how your aowow script is letting us installing it.

If you are ok, I would like to explain all the steps I've done, and the "warns" and "errors" I'm going through.

I installed a fresh Debian with MariaDB, nginx, php-fpm and some dependencies
Under Windows, with MPQEditor, I extracted all the MPQ files to ./aowow/setup/mpqdata and 3 locales to ./aowow/setup/mpqdata/enus (engb / frfr)
I did a git clone of AOWOW to /var/www/html
I moved mpqdata to /var/www/html/aowow/setup
And sudo chown -R www-data:www-data /var/www/
In root, I created the AoWoW SQL user, and gave it the sufficient permissions
Then, with AoWoW user, I imported databases (aowow_db // tcore_world) with SQL files (db_structure.sql // world.sql)
I launched sudo php aowow --db to configure the DB Access. The tools seems to accept as I got a green OK in return
I executed sudo php aowow --configure and change the _SITEHOST and _STATICHOST to reflect my website
- SITE_HOST = 192.168.1.239/aowow
- STATIC_HOST = 192.168.1.239/aowow/static
- and the locales = EN / FR only When I finished and validate, the script semt to crash as I returned back to the shell. I had to launch it again to be sure my informations were still present
Finally, sudo php aowow -s

The scripts seems to work and populate the DBs. But I got a bunch of warns and some error I don't know how to fix. You can see the files I added to this post

For examples:

you say on your repo that it is highly recommended to change these values:
- Calculate.Creature.Zone.Area.Data = 1
- Calculate.Gameoject.Zone.Area.Data = 1 These values are in the worldserver.conf.dist. And as I don't run any WoW Server in this debian, how can I make these changes? And your script does not support AzerothCore, I don't want to run a TrinityCore just for these 2 values

The script can't reach the "Sound..." folder. I got a lot of warns like these:

[WARN]  [soundfiles]  - did not find file: setup/mpqdata/[enus,frfr]/Sound/Creature/Satyr/SatyrAttackA.wav_
[WARN]  [soundfiles]  - did not find file: setup/mpqdata/[enus,frfr]/Sound/Creature/Satyr/SatyrAttackB.wav_
[WARN]  [soundfiles]  - did not find file: setup/mpqdata/[enus,frfr]/Sound/Creature/Satyr/SatyrAttackC.wav_
[WARN]  [soundfiles]  - did not find file: setup/mpqdata/[enus,frfr]/Sound/Creature/Satyr/SatyrWoundA.wav_
...

(you can find a little export of what's going on, at the end)

NOTES: I tried with uppercases (enUS / frFR)... same errors. I tried to copy the mpqdata/Sound to enUS and frFR... same errors.

So, what is that? [enus,frfr] How should the folders be named? enUS frFR? or enus frfr? The 'Sound' folder is, basically in ./aowow/setup/mpqdata/Sound, even for the 'Items' folder, which is NOT in locales! So how to correctly export these sounds?

I got an error about 53 images referenced by DBC but not in the mpqData directory. Is there an error in the script? Is the name of mpqData bad? Is it related with point '2'?

If you can help :)

Thanks a lot

Regards, V. warns - errors.txt

virgo77 commented 4 months ago

Some pics...

NOTE! I'm trying to document all the steps I followed, and hope I will success to make it work. If it is, I will submit it to you, so you can tell me if it suits you. Maybe, I will create a topic on your GITHUB with this tuto :)

virgo77 commented 4 months ago

I tested the website, it seems to work though! website

But I don't have the popup when hover an item. I don't know if it's an issue or if it is because the install is not complete (I stopped it because of all the warns).

Sarjuuk commented 4 months ago

Just commenting the things that look "off":

2. Under Windows, with MPQEditor, I extracted all the MPQ files to **./aowow/setup/mpqdata** and 3 locales to **./aowow/setup/mpqdata/enus** (engb / frfr)  
Do you now have aowow/setup/mpqdata/enus/frfrand aowow/setup/mpqdata/enus/engb that would be wrong .. and weird. Or do you have two english locales? In that case one would be ignored as aowow doesn't differentiate between these two. I'm not even sure if there are differences. Maybe some wording in GlobalStrings.lua.
8. I launched `sudo php aowow --db` to configure the DB Access. The tools seems to accept as I got a green OK in return
not sure you need sudo for that, but eh...
9. I executed `sudo php aowow --configure` and change the _SITE_HOST_ and _STATIC_HOST_ to reflect my website
     When I finished and validate, the script semt to crash as I returned back to the shell. I had to launch it again to be sure my informations were still present
Yeah my fault. Thats fixed now.
1. you say on your repo that it is highly recommended to change these values:
   * _Calculate.Creature.Zone.Area.Data = 1_
   * _Calculate.Gameoject.Zone.Area.Data = 1_
     These values are in the worldserver.conf.dist. And as I don't run any WoW Server in this debian, how can I make these changes?
     And your script does not support AzerothCore, I don't want to run a TrinityCore just for these 2 values
I'm afraid there is no way around that. You dont need that feature. Just be aware that there will be cases where creatures and objects will be spawned in a neighbouring zone. You should be able to recalculate those by clicking on the map pin, after aowow is set up and running.
2. The script can't reach the "_Sound..._" folder. I got a lot of warns like these:
Did you skip the audio conversion step? It isn't in your step by step description.
NOTES: I tried with uppercases (enUS / frFR)... same errors. I tried to copy the mpqdata/Sound to enUS and frFR... same errors.
aowow is case agnostic even under *nix because the paths in the dbc files are case agnostic. So this doesn't matter.
3. I got an error about 53 images referenced by DBC but not in the _mpqData_ directory. Is there an error in the script? Is the name of _mpqData_ bad? Is it related with point '2'?
no, there is just bogus data in the dbc files. As far as i can tell it is not even consistent between locales :/ About 50 missing icons is expected. (And if you update aowow this is no longer considered an error)

From your log:

14:21:46  [WARN]  WARNING -  at /var/www/html/aowow/setup/tools/sqlgen/spawns.ss.php line 144 @ /var/www/html/aowow/includes/database.class.php:92
14:21:46  [WARN]  WARNING - mysqli_query(): (23000/1048): Column 'areaId' cannot be null @ /var/www/html/aowow/includes/libs/DbSimple/Mysqli.php:165
Can you check if areaId in your world.creature, world.gameobject is NULL? Otherwise i don't really see a case how this gets set to null.
===============================================================================
AN ERROR HERE!!!
Awesome! And what error!?
But I don't have the popup when hover an item. I don't know if it's an issue or if it is because the install is not complete (I stopped it because of all the warns).
Except for the mysql-error that i mentioned earlier there are is nothing unexpected or critical in the logs. See the setup process through before using the page.

virgo77 commented 4 months ago

Hi @Sarjuuk! Thanks for your reply

I will try to give you some answers.

Just commenting the things that look "off":
2. Under Windows, with MPQEditor, I extracted all the MPQ files to **./aowow/setup/mpqdata** and 3 locales to **./aowow/setup/mpqdata/enus** (engb / frfr)  
Do you now have aowow/setup/mpqdata/enus/frfrand aowow/setup/mpqdata/enus/engb that would be wrong .. and weird. Or do you have two english locales? In that case one would be ignored as aowow doesn't differentiate between these two. I'm not even sure if there are differences. Maybe some wording in GlobalStrings.lua.

Nope. As you can see in these screenshots, the folder structure is clean:

BUT! you asked if I had 2 english locales... and yes, I have enUS and enGB. I'm wondering that 'enUS' should be the one to use before enGB, so I will delete enGB.

8. I launched `sudo php aowow --db` to configure the DB Access. The tools seems to accept as I got a green OK in return
not sure you need sudo for that, but eh...

This is due because I never use ROOT. I use an account which is in SUDO group. And as I work in the /var/ directory, only root can pass commands without sudo.
9. I executed `sudo php aowow --configure` and change the _SITE_HOST_ and _STATIC_HOST_ to reflect my website
     When I finished and validate, the script semt to crash as I returned back to the shell. I had to launch it again to be sure my informations were still present
Yeah my fault. Thats fixed now.

So I assume I have to do a git pull origin :)
1. you say on your repo that it is highly recommended to change these values:
   * _Calculate.Creature.Zone.Area.Data = 1_
   * _Calculate.Gameoject.Zone.Area.Data = 1_
     These values are in the worldserver.conf.dist. And as I don't run any WoW Server in this debian, how can I make these changes?
     And your script does not support AzerothCore, I don't want to run a TrinityCore just for these 2 values
I'm afraid there is no way around that. You dont need that feature. Just be aware that there will be cases where creatures and objects will be spawned in a neighbouring zone. You should be able to recalculate those by clicking on the map pin, after aowow is set up and running.

OK, noted! :)
2. The script can't reach the "_Sound..._" folder. I got a lot of warns like these:
Did you skip the audio conversion step? It isn't in your step by step description.

Erm... maybe my bad on this case. Just a question: as I use MPQEditor (the Windows Tool), and as the option to convert .WAV to .OGG is enabled, it seems the software did not converted the sound files. So, do you know a tool or a script that can do that 'on the fly'?
NOTES: I tried with uppercases (enUS / frFR)... same errors. I tried to copy the mpqdata/Sound to enUS and frFR... same errors.
aowow is case agnostic even under *nix because the paths in the dbc files are case agnostic. So this doesn't matter.

OK, understood!

3. I got an error about 53 images referenced by DBC but not in the _mpqData_ directory. Is there an error in the script? Is the name of _mpqData_ bad? Is it related with point '2'?
no, there is just bogus data in the dbc files. As far as i can tell it is not even consistent between locales :/ About 50 missing icons is expected. (And if you update aowow this is no longer considered an error)

Yeah, I was thinking about this, but I wanted to be sure :)

From your log:
14:21:46  [WARN]  WARNING -  at /var/www/html/aowow/setup/tools/sqlgen/spawns.ss.php line 144 @ /var/www/html/aowow/includes/database.class.php:92
14:21:46  [WARN]  WARNING - mysqli_query(): (23000/1048): Column 'areaId' cannot be null @ /var/www/html/aowow/includes/libs/DbSimple/Mysqli.php:165
Can you check if areaId in your world.creature, world.gameobject is NULL? Otherwise i don't really see a case how this gets set to null.

In both the tables, all the cells are with '0'. But no 'NULL' found:

===============================================================================
AN ERROR HERE!!!
Awesome! And what error!?

It is the one about the _53 images referenced by DBC but not in the _mpqData_ directory_, just talk earlier. I just put it between the ====== lines ^^
But I don't have the popup when hover an item. I don't know if it's an issue or if it is because the install is not complete (I stopped it because of all the warns).
Except for the mysql-error that i mentioned earlier there are is nothing unexpected or critical in the logs. See the setup process through before using the page.

OK, I will try with some changes, and post a reply here.

Thanks for the time you take. I really appreciate :)

Regards, V.

Sarjuuk commented 4 months ago

So I assume I have to do a `git pull origin` :)
yeah, thats the easist way.
Erm... maybe my bad on this case. Just a question: as I use MPQEditor (the Windows Tool), and as the option to convert .WAV to .OGG is enabled, it seems the software did not converted the sound files. So, do you know a tool or a script that can do that 'on the fly'? 
I don't. I wasn't even aware MPQEditor had this option. But if you already extracted the wav files as ogg it may be enough to just rename the file to have an extension of .wav_ (and .mp3_ for the mp3's) to signify it was converted (see your warns - errors.txt what was expected). No guaranteee a web browser can play them in the end. https://github.com/Sarjuuk/aowow?tab=readme-ov-file#5-reencode-the-audio-files for the full way, but that takes quite a while. Another gotcha is, that aowow only supports one locale for audio files so delete or rename the sound directory you don't want.
In both the tables, all the cells are with '0'. But no 'NULL' found
Thanks .. i'll have to look into that.

virgo77 commented 4 months ago

ah, I think I found a command to rename the .WAV extension to .OGG:

for /R %x in (*.wav) do ren "%x" *.ogg

It is recursive with /R

All WAVE files are now with the OGG VORBIS extension. No conversion anyway. I think it won't be blocking

virgo77 commented 4 months ago

Hi @Sarjuuk :)

Just an information: I tried your convert.sh script and... I got all my .WAV and .MP3 files like this .WAV / .MP3

I saw in your script this information: -f ogg "%_" I assume that this is this option that put all the sound files with an underscore.

And I compared the final size with the original source ones, and it changed obviously.

Should these files stay like this, or should they got the .OGG extension? Can you tell me if this is sufficient to keep AoWoW working, or do I have to change the .WAV/.MP3 extensions to .OGG?

EDIT: I took a look on the 'error' log files I gave on my first post (as I remembered I encountered errors with sound files), and it seems that the AoWoW Setup is using these specific files So, I would think it's a normal way. :)

virgo77 commented 4 months ago

So! some news :)

the setup semt to be able to find the sounds! but I still have a bunch of warnings
but the setup brutaly stops at step "72 / 79", and I went back to shell Here a file with the sounds warns, and the 'crash' progression.txt

during the first steps, I saw these warns:


[WARN]  [sound] Group [12292] Zone-QuelDanasStage02 has invalid sound file Sunwell\SW_FelEnergyWalkUni01 on index 6! Skipping...
[WARN]  [sound] Group [13401] Zap 00 has invalid sound file ID_Forge_Zap04 on index 1! Skipping...

[WARN] [spell] [54292] could not match Glyph of the White Bear with affected spells [WARN] [spell] [55444] could not match Glyph of Lava Lash with affected spells [WARN] [spell] [56216] could not match Glyph of Siphon Life with affected spells [WARN] [spell] [58132] could not match Glyph of the Black Bear with affected spells


If you see what they are, and if a fix exists, I'll take it!

Thanks for your help :)

Regards,
V.

Sarjuuk commented 4 months ago

There are a couple more files missing than on my end but overall close enough.
one of your spell tooltips contains a formula that can't be resolved .. or something. This requires a closer look from you.
the Bear glpyhs are bogus dbc data, Lava Lash and Siphon Life should have a match, but the matching process is pretty janky, so i wouldn't bother.

as for 2.: Spell tooltips can contain pretty elaborate formula constructs that are a pita to resolve. in this concrete case i suspect that $formula resolved to an empty string for some reason. https://github.com/Sarjuuk/aowow/blob/d93b5df5bcf28001a588a4cc2bfac956124b0d9a/includes/types/spell.class.php#L909 I've never had something like that so you'll have to debug this yourself. I'd place:

print_r($this->id);
debug_print_backtrace();

above the eval and with the spell id check how the spell tooltip looks as raw dbc data.

virgo77 commented 4 months ago

Hi @Sarjuuk Thanks for your reply :)

If I have to debug on myself, it will be a little bit... tricky for me, as I'm not a Linux end-user. I'm starting to get by a little on Linux, but I'm far from being able to completely understand the code used in your tools, and how to fix my issues ^^ So, each time I got errors, or lots of warns, I restart from scratch, over and over again... fresh OS install, needed tools, compilation, etc. Different versions, though! (I got time actually, so I'm testing ^^)

I just have some questions BTW. Can you tell me:

what version of Debian/Ubuntu you recommend?
what about MariaDB/SQL?
PHP8.x but, which one is mandatory?
The "World DB" your recommend is really an old one (2021). Is there an another version that can be more compatible? Maybe I'm going through bugs because of some tool versions that are not really the ones you use! ^^
Should a php aowow -u be ran before php aowow -s?

Actually I'm on:

Debian 11 x64 (I reinstalled from scratch. Exit Debian 12 for now)
nginx
php8.3
MariaDB 10.6.18 (latest)

In your last reply, you told me "There are a couple more files missing than on my end but overall close enough". Maybe I use a 'bad' WotLK Client? I downloaded these ones:

enUS = https://wowdl.net/client/World-of-Warcraft-3.3.5a.12340-enUS
frFR = https://wowdl.net/client/World-of-Warcraft-3.3.5a.12340-frFR In fact, their '.\Data' are strictly identical. So, I keep the enUS locale and just put the frFR locale into '.\Data'. Maybe you can give the link where you downloaded the WotLK Client you use?

And finally, I extract with the latest Windows version of MPQEditor

Just now, I'm trying to make a new 'AoWoW Setup'. I did all the rest. This is the final stretch :D

I hope I'm not boring you too much, but this helps me understand how to proceed

Thanks for your time :)

Best regards, V.

virgo77 commented 4 months ago

OK, so! youhouh, the site works! I still had the warns and others, but I could pass 'til step 79 :) Admin Account created little popup when hover the items, it's great Now I will try the website, and find if the items and objects (where I had warns) are missing or bugged

happy happy happy ^^

Not easy to understand and install btw. As I said, I'm trying to edit a tutorial to make it easier :) Hope it will help others!

Please, I will appreciate if you can share your WotLK Client. I'm wondering mine is not really a complete one... Or maybe you use a pre-Cata Client?

Best Regards, V.

Sarjuuk commented 4 months ago

If I have to debug on myself, it will be a little bit... tricky for me, as I'm not a Linux end-user. I'm starting to get by a little on Linux, but I'm far from being able to completely understand the code used in your tools, and how to fix my issues ^^ So, each time I got errors, or lots of warns, I restart from scratch, over and over again... fresh OS install, needed tools, compilation, etc. Different versions, though! (I got time actually, so I'm testing ^^)

If you do that kind of work you should really look into debugging. It's probably a lot easier, than setting up your system from scratch. You do need to have some understanding of the language though. At the very least you could place the debug i gave and report back with the result.

* what version of Debian/Ubuntu you recommend?
* what about MariaDB/SQL?
no recommendations, i never bothered thinking about it.

PHP8.x but, which one is mandatory? soon-ish™ PHP v8.2 .. for now 8.0 is fine.

The "World DB" your recommend is really an old one (2021). Is there an another version that can be more compatible? Compatibility here is really just: "Has the structure changed"?, No, it hasn't. Not in the last 3 years aparently. Use the most recent one.

Maybe I'm going through bugs because of some tool versions that are not really the ones you use! ^^ probably.. Seeing the filenames with a null-byte attached in your screenshots, reading you extracted under WIN but worked under *nix. Its wild..

Should a php aowow -u be ran before php aowow -s? No, update means: "apply sql changes and rerun speciffic build steps". The former should be included in db_structure.sql, the latter gets executed by running setup anyway.

In your last reply, you told me "There are a couple more files missing than on my end but overall close enough". Maybe I use a 'bad' WotLK Client?

I don't even know where i got my client originally, i have used the clients from wowdl in the past and they were fine. You've probably made a minor mistake when extracting the client files. Mixed up the patch order or something. No i'm not sharing my random assortment of extracted files.

OK, so! youhouh, the site works!

So how did you get past the TalentCalc step with the spell tooltip that broke the eval()?

As I said, I'm trying to edit a tutorial to make it easier

The wiki pages here are open. have at it.^^

Sidenote: I found the reason for the areaId IS NULL errors earlier. In the latest TDB release (and maybe earlier) the creature/gameobject tables partially contain precalculated zoneIds/AreaIds .. and are just wrong. Sometimes equal to the WDBVerified column.

virgo77 commented 4 months ago

Hi @Sarjuuk! Thanks for your reply :)

So how did you get past the TalentCalc step with the spell tooltip that broke the eval()?

I sincerly really don't know. Just configured my host as I explained on my previous post:

Debian 11 x64

php8.3 (fpm / gd / mbstring / etc.)

MariaDB 10.6:latest

configured nginx with a 'host name': wowdb.home.lan

configured AoWoW SITE_HOST and STATIC_HOST with same informations SITE_HOST: wowdb.home.lan/aowow STATIC_HOST: wowdb.home.lan/aowow/static

strangely, my nginx was SSL configured, even I enabled FORCE_SSL, it won't pass the checks

Here some screenshots (I customed it, just a little bit!):

I tested some objects and compared my results with the aowow db of TrinityCore Info website and they are identical... for now. Maybe I will have some broken ones. What I can say is that my website works. In the next few weeks, I think I will try to make a new install with other versions of OS and tools, and trying some other game clients if I can found some. Maybe it will succeed... At least, it seems to be fully working!

Now, I have to find out how to set up the smtp config when registering (yes I have a little mail server ^^ (I told you I had time, so I'm working on my homelab)). I saw some php files like 'email' or something. I will search a bit and I think I'm all set! (I would like to create a user account and the email is asked. So I will be able to add screenshots of items, like the real WoWHead :D)

Have a good night! And thanks again for your time 👍

See you ;)

Best Regards, V.

Sarjuuk commented 4 months ago

Don't waste your time with clients of versions other than 3.3.5a r12340. The dbc structure is different and unless (contrary to previous statements) you want to learn php and adapt the structure it will not work. Though it would be interesting if it can handle modded 335a clients.

The email stuff i put there over a decade ago and it should work in theory, but i don't think anyone used it yet.^^

Anyway, hf.