license for using depth image

cleardusk / 3DDFA

The PyTorch improved version of TPAMI 2017 paper: Face Alignment in Full Pose Range: A 3D Total Solution.

MIT License

3.61k stars 654 forks source link

license for using depth image #118

Closed liwei46 closed 4 years ago

liwei46 commented 4 years ago

Dear @cleardusk ,

Thanks for sharing your great work.

I notice that for the project, it is under MIT license.
If I wish to use the depth image, it also involves the 3dmm rendering part. Is this part also under MIT license?

Thanks and Regards,

cleardusk commented 4 years ago

For commercial usage, you may apply for the business license of BFM. The code is under MIT license, but the BFM part is NOT.

tetterl commented 4 years ago

@cleardusk thanks for such a great work. Which part exactly relays on the BFM and would not be withing the MIT license? Aka would it be possible to use your code (e.g. main.py) to get a fitted 3D mesh to an image without the BFM license?

fleskovar commented 4 years ago

If I understood correctly and based on what the original paper says, the dependency with BFM is in the "w_shp_sim.npy" (w_shp) and is only used when generating the vertices in "ddfa.reconstruct_vertex" int he line below, correct?

vertex = p @ (u + w_shp @ alpha_shp + w_exp @ alpha_exp).reshape(3, -1, order='F') + offset

If that is the case, could you confirm that getting rid of the term w_shp @ alpha_shp in the equation above would get rid of all BFM dependencies?

cleardusk commented 4 years ago

@fleskovar You are right. But the dense vertices reconstruction will rely on w_shp and w_exp, is it possible to get rid of using it?

fleskovar commented 4 years ago

@cleardusk I am far from being a reliable source of information here, but here are my thoughts:

If you delete the w_shp @ alpha_shp term, you will still be able to generate the dense vertices. However, your resulting mesh will only have the information coming from the FaceWarehouse correlation. Most likely, all the face you generate will have pretty much the same geometry.
In order to replace the BFM dependency, you will a new dataset that correlates the geometry of the faces to the params generated by your algorithm. Once you have that, you can do PCA on that new geometry and come up with a new alpha_shp tensor.
I think that there is still some dependency to BFM in the u variable (base mesh) from the vertex equation, correct? If that is the case you will need to also replace that.
Do you know if it would be possible to use the 300W dataset to do the steps described above?

cleardusk commented 4 years ago

@fleskovar I thought your meaning is to build a 3D model basis like BFM using another collected datasets of scanned 3d faces. For BFM, the academic license is free but the business license needs a charge.

cleardusk commented 4 years ago

@fleskovar Using private datasets to build a 3D model basis is the most direct and maybe the best solution, many companies are doing it. The whole procedure may be a little complicated and needs lots of resource including the high-resolution scanning devices and manpower costs.