search

ARIA-VALUSPA / AVP

This is the ARIA-VALUSPA Platform, or AVP for short. Use this platform to build your own Virtual Humans with audio-visual input and output, language models for English, French, and German, emotional understanding, and many more. This work was funded by European Union Horizon 2020 research and innovation programme, grant agreement No 645378.
https://aria-agent.eu
32 stars 16 forks source link

age, gender, and emotion not working #6

Closed jonlnichols closed 6 years ago

jonlnichols commented 6 years ago

hi, i downloaded your repo and got it mostly working, after copying the launcher files like suggested in the other issue thread, but the age and emotion detection isn't working

it may be due to the following errors i've noticed in the window start-4-agent.bat was running in.

note, i have this running on my windows 7 laptop, with the ASR being done on a AWS Ubuntu box.

several of warnings like:

[trainer] # WARNING # failed loading trainer from file '../../models/speech/chunks.audio.compare.age.gold.linsvm.trainer' (r:0,c:0) location: source\Trainer.cpp (1240) [classifier] # WARNING # could not load trainer '../../models/speech/chunks.audio.compare.age.gold.linsvm' location: ....\source\Classifier.cpp (192) ... aEs0315 14:58:03.750545 9312 common.cpp:122] Cannot create Cublas handle. Cublas won't be available. sEi0315 14:58:03.760546 9312 common.cpp:129] Cannot create Curand generator. Curand won't be available. ... [trainer] # WARNING # failed loading trainer from file '../../models/speech/close.mfccdd[-f 0.01 -d 0.015 -w 0.04].filler.gold[-c 5].lin[-s 0 -e 0.01 -B 0.1].final.trainer' (r:0,c:0)a location: source\Trainer.cpp (1240) l[classifier] # WARNING # could not load trainer '../../models/speech/close.mfccdd[-f 0.01 -d 0.015 -w 0.04].filler.gold[-c 5].lin[-s 0 -e 0.01 -B 0.1].final' location: ....\source\Classifier.cpp (117)

note, i dont think these are related to the age, gender, and emotion not working, but there were a bunch of these warnings too.

WARNING: alice/body.mesh is an older format ([MeshSerializer_v1.40]); you should upgrade it as soon as possible using the OgreMeshUpgra de tool. WARNING: bone.mesh is an older format ([MeshSerializer_v1.40]); you should upgrade it as soon as possible using the OgreMeshUpgrade too l. Can't assign material bone to SubEntity of Reference-bone-0.9298522953614133 because this Material does not exist. Have you forgotten t o define it in a .material script?

mvalstar commented 6 years ago

Hi, I think the guys from SSI are best placed to help you with this. I haven't seen this problem before.

The Ogre warnings are something we won't be able to help with. We will soon have our Unity3D version up, which means you wouldn't have to use Ogre anymore.

frankenjoe commented 6 years ago

Hi, looks like the model files are not in place. Which repository are we talking about? The models should be in a folder /System/Agent-Input/ssi/models (see e.g. ARIA beta branch).

g-pechorin commented 6 years ago

The model files aren't in "git" (they were too big)

Was the RunMeFirst/get_models.bat script successful during setup? It should have downloaded several (slightly-big) files.

frankenjoe commented 6 years ago

The missing models should be part of the repository (only those > 100mb are downloaded with the script)

bpotard commented 6 years ago

It seems all the small model files were removed in the 3.0 release: https://github.com/ARIA-VALUSPA/AVP/commit/736b6a21abbab1574d5c84691a6bb5da9cb713ac

mvalstar commented 6 years ago

Ok, Peter, can you put them back please, as this breaks the current rule of the 100MB threshold for models.

Do we want to move to a way of working where all models are in a separate location, and fetched with get_models? If not, we may want to rename get_models.bat to get_large_models.bat for clarity.

g-pechorin commented 6 years ago

I've pushed the missing files to master so it works. I've checked it on a PC that never had AVP installed before. The software works.

I don't want to call this a 3.0.1 release yet; I'm not happy with the .gitignore files since the downloaded models are still showing up. I'll try some other approaches this weekend.

For now; master has the missing files and should work.

Peter

mvalstar commented 6 years ago

I presume this works now.