[REVIEW]: RayTracer.jl: A General Purpose Differentiable Renderer

whedon commented 4 years ago

Submitting author: !--author-handle-->@avik-pal@matbesancon<!--end-editor-- Reviewers: @snoeyink, @dourouc05 Archive:

Status

Status badge code:

HTML: <a href="https://submissions.juliacon.org/papers/ebd92f2a47780826004bd88a6fd1f717"><img src="https://submissions.juliacon.org/papers/ebd92f2a47780826004bd88a6fd1f717/status.svg"></a>
Markdown: [![status](https://submissions.juliacon.org/papers/ebd92f2a47780826004bd88a6fd1f717/status.svg)](https://submissions.juliacon.org/papers/ebd92f2a47780826004bd88a6fd1f717)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@snoeyink & @dourouc05, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

Make sure you're logged in to your GitHub account
Be sure to accept the invite at this URL: https://github.com/JuliaCon/proceedings-reviews/invitations

The reviewer guidelines are available here: https://proceedings.juliacon.org/guide/reviewers. Any questions/concerns please let @matbesancon know.

✨ Please try and complete your review in the next two weeks ✨

Review checklist for @snoeyink

Conflict of interest

[x] As the reviewer I confirm that I have read the JuliaCon conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

[x] I confirm that I read and will adhere to the JuliaCon code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Authorship: Has the submitting author (@avik-pal) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Paper format

[x] Authors: Does the paper.tex file include a list of authors with their affiliations?
[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?
[x] Page limit: Is the page limit for full papers respected by the submitted document?

Content

[x] Context: is the scientific context motivating the work correctly presented?
[x] Methodology: is the approach taken in the work justified, presented with enough details and reference to reproduce it?
[x] Results: are the results presented and compared to approaches with similar goals?

Review checklist for @dourouc05

Conflict of interest

[x] As the reviewer I confirm that I have read the JuliaCon conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

[x] I confirm that I read and will adhere to the JuliaCon code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Authorship: Has the submitting author (@avik-pal) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Paper format

[x] Authors: Does the paper.tex file include a list of authors with their affiliations?
[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?
[x] Page limit: Is the page limit for full papers respected by the submitted document?

Content

[x] Context: is the scientific context motivating the work correctly presented?
[x] Methodology: is the approach taken in the work justified, presented with enough details and reference to reproduce it?
[x] Results: are the results presented and compared to approaches with similar goals?

whedon commented 4 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @snoeyink, @dourouc05 it looks like you're currently assigned to review this paper :tada:.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/JuliaCon/proceedings-review) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

Set yourself as 'Not watching' https://github.com/JuliaCon/proceedings-review:

watching

You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

whedon commented 4 years ago

Attempting PDF compilation. Reticulating splines etc...

whedon commented 4 years ago

PDF failed to compile for issue #37 with the following error:

Can't find any papers to compile :-(

matbesancon commented 4 years ago

Comment from the pre-review from @snoeyink This paper introduces a raytracer built in Julia that allows the Zygote Automatic Differentiation tool to compute gradients of parameters. One can then optimize parameter values for scene understanding or reconstruction.

The examples shown are toy experiments of obtaining camera or light parameters to match a rendered image. They demonstrate that the combination works, which is good.

I would like to see some discussion of more interesting parameters, such as determining material properties from images. And more on limitations: determining geometry seems out of reach because there are so many discrete choices: the problem is not differentiable. How fast is the gradient computation after AD? (How fast is the AD?) How do these grow with scene complexity?

This paper shows the hallmarks of being written in haste as the package is being developed. I have many comments and suggestions in pdf uploaded to the package repository. avik-pal/RayTracer.jl#9

The abstract discusses training times for ML, which is not in the paper, so here is a rewrite: In this paper we present RayTracer.jl, a renderer in Julia that is fully differentiable via Zygote.jl. This means that RayTracer not only renders 2D images from 3D scene parameters, but it can be used to optimize for model parameters that generate a target image in a Differentiable Programming (DP) pipeline. We interface our renderer with the deep learning library Flux for use in combination with neural networks. We demonstrate the use of this differentiable renderer in rendering tasks and in solving inverse graphics problems.

matbesancon commented 4 years ago

@whedon generate pdf from branch juliacon-paper

whedon commented 4 years ago

Attempting PDF compilation from custom branch juliacon-paper. Reticulating splines etc...

whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

snoeyink commented 4 years ago

added comments in avik-pal/RayTracer.jl#9 on latest version, which is an improvement.

avik-pal commented 4 years ago

@whedon generate pdf from branch review-updates

whedon commented 4 years ago

Attempting PDF compilation from custom branch review-updates
. Reticulating splines etc...

whedon commented 4 years ago

PDF failed to compile for issue #37 with the following error:

error: pathspec 'review-updates ' did not match any file(s) known to git. Can't find any papers to compile :-(

avik-pal commented 4 years ago

@whedon generate pdf from branch review_updates

whedon commented 4 years ago

Attempting PDF compilation from custom branch review_updates. Reticulating splines etc...

whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

matbesancon commented 4 years ago

ping @snoeyink and @dourouc05 :)

matbesancon commented 4 years ago

bump

avik-pal commented 4 years ago

@whedon generate pdf from branch juliacon-paper

matbesancon commented 4 years ago

@whedon generate pdf from branch juliacon-paper

whedon commented 4 years ago

Attempting PDF compilation from custom branch juliacon-paper. Reticulating splines etc...

whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

matbesancon commented 4 years ago

@avik-pal no idea what happened, whedon was napping

matbesancon commented 4 years ago

ping @snoeyink and @dourouc05 :)

matbesancon commented 4 years ago

@snoeyink, @dourouc05 I think we are almost done here, could we finish this review?

dourouc05 commented 4 years ago

I did not see that @avik-pal had taken my comments into account... Green light for me!

matbesancon commented 4 years ago

@snoeyink, @dourouc05 still missing some check boxes. If they are considered good, tick them so that we can wrap this up

avik-pal commented 4 years ago

Bump.

@matbesancon is there anything I need to do?

dourouc05 commented 4 years ago

@snoeyink still has a few check boxes to tick, I think.

matbesancon commented 4 years ago

@whedon check references

whedon commented 4 years ago

Attempting to check references...

matbesancon commented 4 years ago

@whedon check references from branch juliacon-paper

whedon commented 4 years ago


OK DOIs

- 10.1145/15922.15916 is OK
- 10.1145/1468075.1468082 is OK
- 10.1145/54852.378457 is OK
- 10.21105/joss.00615 is OK

MISSING DOIs

- https://doi.org/10.1137/141000671 may be missing for title: Julia: A fresh approach to numerical computing
- https://doi.org/10.1145/3272127.3275109 may be missing for title: Differentiable Monte Carlo Ray Tracing through Edge Sampling
- https://doi.org/10.1007/s12532-018-0139-4 may be missing for title: CasADi – A software framework for nonlinear optimization and optimal control

INVALID DOIs

- None

whedon commented 4 years ago

Attempting to check references... from custom branch juliacon-paper

matbesancon commented 4 years ago

@avik-pal can you check the DOIs above?

whedon commented 4 years ago


OK DOIs

- 10.1145/15922.15916 is OK
- 10.1145/1468075.1468082 is OK
- 10.1145/54852.378457 is OK
- 10.21105/joss.00615 is OK

MISSING DOIs

- https://doi.org/10.1137/141000671 may be missing for title: Julia: A fresh approach to numerical computing
- https://doi.org/10.1145/3272127.3275109 may be missing for title: Differentiable Monte Carlo Ray Tracing through Edge Sampling
- https://doi.org/10.1007/s12532-018-0139-4 may be missing for title: CasADi – A software framework for nonlinear optimization and optimal control

INVALID DOIs

- None

avik-pal commented 4 years ago

@whedon check references

whedon commented 4 years ago

Attempting to check references...

whedon commented 4 years ago


OK DOIs

- 10.1145/15922.15916 is OK
- 10.1145/1468075.1468082 is OK
- 10.1145/54852.378457 is OK
- 10.21105/joss.00615 is OK

MISSING DOIs

- https://doi.org/10.1137/141000671 may be missing for title: Julia: A fresh approach to numerical computing
- https://doi.org/10.1145/3272127.3275109 may be missing for title: Differentiable Monte Carlo Ray Tracing through Edge Sampling
- https://doi.org/10.1007/s12532-018-0139-4 may be missing for title: CasADi – A software framework for nonlinear optimization and optimal control

INVALID DOIs

- None

avik-pal commented 4 years ago

@whedon check references

whedon commented 4 years ago

Attempting to check references...

whedon commented 4 years ago


OK DOIs

- 10.1145/15922.15916 is OK
- 10.1145/1468075.1468082 is OK
- 10.1145/54852.378457 is OK
- 10.21105/joss.00615 is OK

MISSING DOIs

- https://doi.org/10.1137/141000671 may be missing for title: Julia: A fresh approach to numerical computing
- https://doi.org/10.1145/3272127.3275109 may be missing for title: Differentiable Monte Carlo Ray Tracing through Edge Sampling
- https://doi.org/10.1007/s12532-018-0139-4 may be missing for title: CasADi – A software framework for nonlinear optimization and optimal control

INVALID DOIs

- None

avik-pal commented 4 years ago

@matbesancon I am not sure what is the issue here I added the DOIs in the latest commit f03c297

matbesancon commented 4 years ago

Sometimes it just doesn't pick it up, if you put it it's fine.

On Tue, Oct 22, 2019, 18:06 Avik Pal notifications@github.com wrote:

@matbesancon https://github.com/matbesancon I am not sure what is the issue here I added the DOIs in the latest commit f03c297 https://github.com/avik-pal/RayTracer.jl/commit/f03c2974a87f5f05e34c2d8f53f736e5a6bd5d4f

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/JuliaCon/proceedings-review/issues/37?email_source=notifications&email_token=AB2FDMRJTGEIO67AVJFPY7DQP4QGVA5CNFSM4IDWUZZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEB6JLUY#issuecomment-545035731, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB2FDMRQCCW7TDQ4NJWJSYTQP4QGVANCNFSM4IDWUZZQ .

avik-pal commented 4 years ago

@whedon check references

whedon commented 4 years ago

Attempting to check references...

whedon commented 4 years ago


OK DOIs

- 10.1145/15922.15916 is OK
- 10.1145/1468075.1468082 is OK
- 10.1145/54852.378457 is OK
- 10.21105/joss.00615 is OK

MISSING DOIs

- https://doi.org/10.1137/141000671 may be missing for title: Julia: A fresh approach to numerical computing
- https://doi.org/10.1145/3272127.3275109 may be missing for title: Differentiable Monte Carlo Ray Tracing through Edge Sampling
- https://doi.org/10.1007/s12532-018-0139-4 may be missing for title: CasADi – A software framework for nonlinear optimization and optimal control

INVALID DOIs

- None

avik-pal commented 4 years ago

@whedon check references

whedon commented 4 years ago

Attempting to check references...

whedon commented 4 years ago


OK DOIs

- 10.1137/141000671 is OK
- 10.1145/3272127.3275109 is OK
- 10.1145/15922.15916 is OK
- 10.1007/s12532-018-0139-4 is OK
- 10.1145/1468075.1468082 is OK
- 10.1145/54852.378457 is OK
- 10.21105/joss.00615 is OK

MISSING DOIs

- None

INVALID DOIs

- None

whedon commented 4 years ago

Attempting to check references...

whedon commented 4 years ago


OK DOIs

- 10.1137/141000671 is OK
- 10.1145/3272127.3275109 is OK
- 10.1145/15922.15916 is OK
- 10.1007/s12532-018-0139-4 is OK
- 10.1145/1468075.1468082 is OK
- 10.1145/54852.378457 is OK
- 10.21105/joss.00615 is OK

MISSING DOIs

- None

INVALID DOIs

- None

JuliaCon / proceedings-review