Hot to estimate p,q

[abeacco]

Hi,

Thank you for this amazing work. I am trying to use your code to recover a 3d surface from an image. I have now a first normal map and a depth map, but I want to integrate normals to reconstruct a better depth map. I am using the smooth_integration function like in demo2, but I'm not sure that I am feeding the proper p and q values. As far as I understand p and q are the z derivatives, and they can be taken either from my initial depth map or derived from the normals I have. From the following paper I thought that I could compute p and q as: p = −nx/nz and q = −ny/nz http://www.weizmann.ac.il/math/ronen/sites/math.ronen/files/uploads/basri_jacobs_kemelmacher_-_photometric_stereo_with_general_unknown_lighting.pdf But this doesn't seem to work for me... I'm just wondering if I'm understanding it properly.

Thanks!

[yqueau]

Hello,

You are indeed understanding correctly how to compute (p,q), so I guess it is more related to how your normals are defined, especially if their (xyz) frame matches the one I use in integration.

If you want, just send me one example normal map of yours, I'll have a quick look and send you back the two lines of codes to define (p,q), or a minimal working example.

Best, Yvain

Le mar. 12 févr. 2019 à 12:01, abeacco notifications@github.com a écrit :

Hi,

Thank you for this amazing work. I am trying to use your code to recover a 3d surface from an image. I have now a first normal map and a depth map, but I want to integrate normals to reconstruct a better depth map. I am using the smooth_integration function like in demo2, but I'm not sure that I am feeding the proper p and q values. As far as I understand p and q are the z derivatives, and they can be taken either from my initial depth map or derived from the normals I have. From the following paper I thought that I could compute p and q as: p = −nx/nz and q = −ny/nz

http://www.weizmann.ac.il/math/ronen/sites/math.ronen/files/uploads/basri_jacobs_kemelmacher_-_photometric_stereo_with_general_unknown_lighting.pdf But this doesn't seem to work for me... I'm just wondering if I'm understanding it properly.

Thanks!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/yqueau/normal_integration/issues/1, or mute the thread https://github.com/notifications/unsubscribe-auth/AKB5rPf9QrL-I3luAGO2ig1GCRvtHcscks5vMp7xgaJpZM4a2Iz5 .

--

Yvain Quéau CNRS Researcher GREYC (UMR-CNRS 6072) E-mail: yvain.queau@ensicaen.fr 6, Bd du Marechal Juin Tel: +33 (0)2-31-45-54-52 F-14050 CAEN Cedex https://sites.google.com/view/yvainqueau https://sites.google.com/view/yvainqueau France

[abeacco]

Hi,

Thank you for your quick answer. Yes, then it must be a problem on how I'm computing or defining the normals. Maybe I'm not using the proper space. Here is an image of a normal map I'm using. I'm reading the colours and converting them from [0,255] space to a [-1,1] space.

[yqueau]

Alright,

Indeed the problem was just with the orientation of axes. Below is a minimal working example which will display a nice human body ;)

Best, Yvain

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% clear close all

% Load the integration functions from the git repo addpath(genpath('normal_integration/'));

% Load normal map, and convert it to my 3D-space coordinates N = double(imread('52631957-7d29ca00-2ec0-11e9-8aad-28518923db2a.png')); Nx = (N(:,:,2)/128)-1; Ny = (N(:,:,3)/128)-1; Nz = -((N(:,:,1)/128)-1);

% Get (p,q) from normals p = -Nx./Nz; q = -Ny./Nz;

% Some normal are weird (not pointing towards camera => do harmonic depth completion in that case by setting (p,q) = 0) p(Nz<0.05) = 0; q(Nz<0.05) = 0;

% Get mask from the normals (a bit hacky, better do the mask manually with Gimp...) mask = (Nx~=-1) & (Ny~=-1) & (Nz~=1);

% Set parameters for integration lambda = 1e-6*ones(size(p)); % Uniform field of weights (nrows x ncols) z0 = zeros(size(p)); % Null depth prior (nrows x ncols) solver = 'pcg'; % Solver ('pcg' means conjugate gradient, 'direct' means backslash i.e. sparse Cholesky) precond = 'ichol'; % Preconditioner ('none' means no preconditioning, 'ichol' means incomplete Cholesky, 'CMG' means conjugate combinatorial multigrid -- the latter is fastest, but it need being installed, see README)

% Run integration z = smooth_integration(p,q,mask,lambda,z0,solver,precond);

% Display result figure surfl(z,[-135 30]); view(-35,20) axis ij; shading flat; colormap gray; axis equal; grid off axis off title('Reconstruction with mask','Interpreter','Latex','Fontsize',14) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Le mar. 12 févr. 2019 à 12:21, abeacco notifications@github.com a écrit :

Hi,

Thank you for your quick answer. Yes, then it must be a problem on how I'm computing or defining the normals. Maybe I'm not using the proper space. Here is an image of a normal map I'm using. I'm reading the colours and converting them from [0,255] space to a [-1,1] space. [image: normals_medium] https://user-images.githubusercontent.com/12295499/52631957-7d29ca00-2ec0-11e9-8aad-28518923db2a.png

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/yqueau/normal_integration/issues/1#issuecomment-462721659, or mute the thread https://github.com/notifications/unsubscribe-auth/AKB5rIe7Om3J6Hiy_fgNvoA1yJ-CU9V9ks5vMqOfgaJpZM4a2Iz5 .

[abeacco]

Great!!!

That worked just perfectly!!

Many thanks!!

And congratulations for such a good and usefull work :)

Alejandro Beacco

[yqueau]

My pleasure :)

Le mar. 12 févr. 2019 à 12:42, abeacco notifications@github.com a écrit :

Great!!!

That worked just perfectly!!

Many thanks!!

And congratulations for such a good and usefull work :)

Alejandro Beacco

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/yqueau/normal_integration/issues/1#issuecomment-462728121, or mute the thread https://github.com/notifications/unsubscribe-auth/AKB5rF9OhjJcohD3P17MhcjUygCaASBaks5vMqiygaJpZM4a2Iz5 .

[DataEnthusiastSathya]

Alright, Indeed the problem was just with the orientation of axes. Below is a minimal working example which will display a nice human body ;) Best, Yvain %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% clear close all % Load the integration functions from the git repo addpath(genpath('normal_integration/')); % Load normal map, and convert it to my 3D-space coordinates N = double(imread('52631957-7d29ca00-2ec0-11e9-8aad-28518923db2a.png')); Nx = (N(:,:,2)/128)-1; Ny = (N(:,:,3)/128)-1; Nz = -((N(:,:,1)/128)-1); % Get (p,q) from normals p = -Nx./Nz; q = -Ny./Nz; % Some normal are weird (not pointing towards camera => do harmonic depth completion in that case by setting (p,q) = 0) p(Nz<0.05) = 0; q(Nz<0.05) = 0; % Get mask from the normals (a bit hacky, better do the mask manually with Gimp...) mask = (Nx~=-1) & (Ny~=-1) & (Nz~=1); % Set parameters for integration lambda = 1e-6*ones(size(p)); % Uniform field of weights (nrows x ncols) z0 = zeros(size(p)); % Null depth prior (nrows x ncols) solver = 'pcg'; % Solver ('pcg' means conjugate gradient, 'direct' means backslash i.e. sparse Cholesky) precond = 'ichol'; % Preconditioner ('none' means no preconditioning, 'ichol' means incomplete Cholesky, 'CMG' means conjugate combinatorial multigrid -- the latter is fastest, but it need being installed, see README) % Run integration z = smooth_integration(p,q,mask,lambda,z0,solver,precond); % Display result figure surfl(z,[-135 30]); view(-35,20) axis ij; shading flat; colormap gray; axis equal; grid off axis off title('Reconstruction with mask','Interpreter','Latex','Fontsize',14) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Le mar. 12 févr. 2019 à 12:21, abeacco notifications@github.com a écrit : … Hi, Thank you for your quick answer. Yes, then it must be a problem on how I'm computing or defining the normals. Maybe I'm not using the proper space. Here is an image of a normal map I'm using. I'm reading the colours and converting them from [0,255] space to a [-1,1] space. [image: normals_medium] https://user-images.githubusercontent.com/12295499/52631957-7d29ca00-2ec0-11e9-8aad-28518923db2a.png — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/AKB5rIe7Om3J6Hiy_fgNvoA1yJ-CU9V9ks5vMqOfgaJpZM4a2Iz5 .

Hi @yqueau ..thanks for your amazing support. I have normal files like the below one from DILIGENT data set. Kindly let me know the rescaling I should be doing.

https://sites.google.com/site/photometricstereodata/