2D-3D landmarks output . how?

[lulu1315]

hello and thank you for the code I'm trying to output the 3D landmarks of an input image and I'm a bit confused . (I'm running the code on KDE Neon - Ubuntu 16.04)

this is the command I'm using th main.lua -detectFaces True -type 3D -outputFormat txt -model models/3D-FAN.t7 -input images/

images contains several .jpg images

first i get this error : Found 5 images 5 images require a face detector Initialising python libs... Initialising detector... processing /shared/foss/2D-and-3D-face-alignment/images/Surprise_363_1.jpg /shared/foss/torch-multi/install/bin/luajit: main.lua:90: Invalid numpy data type 9

according to https://github.com/AaronJackson/vrn/issues/59 I changed the line in face_detection_dlib.lua local detections = py.reval('[np.asarray([d.left(), d.top(), d.right(), d.bottom()]) for i, d in enumerate(dets)]',{dets=dets}) with local detections = py.reval('[np.asarray([d.left(), d.top(), d.right(), d.bottom()],dtype=float) for i, d in enumerate(dets)]',{dets=dets})

now I have this error : /shared/foss/torch-multi/install/bin/luajit: bad argument #2 to '?' (index out of bound at /tmp/luarocks_torch-scm-1-2154/torch7/generic/Tensor.c:965) stack traceback: [C]: at 0x7f0fb5057b70 [C]: in function '__index' main.lua:184: in main chunk [C]: in function 'dofile' ...orch-multi/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk [C]: at 0x00405d50

the first .txt file is written before crashing , it contains the following lines :

379 218 [torch.FloatTensor of size 2] , 379 250 [torch.FloatTensor of size 2]

I tried to replace preds_img with preds_hm in the loop were the .txt file is written in main.lua (line 172 to 187) and now it works ... but I only got 2D landmarks. I'm certainly missing something here ..

thanks in advance luc

[lulu1315]

tried a little bit harder : th main.lua -detectFaces True -type 3D-full -outputFormat txt -model models/2D-FAN-generic.t7 -modelZ models/3D-FAN-depth.t7 -input images/ -device gpu

gives me 3D landmarks in the .txt files

I'm a bit lost on : -which model to use ? -what's the difference between 3D and 3D-full mode ?

[1adrianb]

Hi @lulu1315,

• Regarding the model, it depends on your needs, they will produce slighly different configurations.
• 3D in this case is the 2D projection (x,y) of the 3D points (x,y,z), while the full 3D will actually give you the x,y,z location of the points. You will still have however to pass the right models for this.

Depending on your needs, if its not for evalution/benchmarking purposes you can have a look of our python package: https://github.com/1adrianb/face-alignment (instalable via pip or conda) which should be easier to use.

[lulu1315]

thank you adrian.I will look into it luc