janpaul123 / paperprograms

Run Javascript on pieces of paper!
https://paperprograms.org
MIT License
484 stars 55 forks source link

Program data includes camera picture of program #38

Closed maryrosecook closed 6 years ago

maryrosecook commented 6 years ago
shaunlebron commented 6 years ago

For context, this is mary's demo video.

Thanks for doing this! I looked at this for a bit, and the method of cropping the ROI (region of interest) in OpenCV is restricted to an axis-aligned bounding box, so I'm guessing the picture of a rotated paper will always look something like this:

It might be easiest to provide users with a general utility function instead—to take a region from the camera image of the whole projection surface, and draw it to a target canvas context (paper or supporter):

drawFromCamera(
  ctx,       // canvas context to draw to
  srcRegion, // source coordinates from camera image of whole projection surface
  dstRegion  // destination coordinates to draw to
)
// NOTE: a "region" can be an array of 3 or 4 vertices (triangle or quad for now)

For example, the demo video could be achieved with something like:

const srcIds = [...]; // source animation papers
const dstId = ...; // draw animation to this paper

while (true) {
  for (let srcId of srcIds) {
    drawFromCamera(supporterCtx, papers[srcId].quadPoints, papers[dstId].quadPoints);
    supporterCtx.commit();
    await delay(250);
  }
}

For implementation, the hard work has already been done by patternPath, found in Three.js's canvas renderer. It seems to retrofit the simple canvas API with this arbitrary texture filling function that we need.

I'll try this out in an isolated environment to confirm if the implementation works as expected.

maryrosecook commented 6 years ago

Sounds great! Definitely seems like a better approach. Thanks a lot!

shaunlebron commented 6 years ago

Progress here. It looks like transforming triangles separately doesn't look good. So I'll try to copy the transform method used for each paper context to do proper perspective correction.

texture-transpose

maryrosecook commented 6 years ago

Wow cool!

On Apr 10, 2018, at 3:15 AM, Shaun Lebron notifications@github.com wrote:

Progress here. It looks like transforming triangles separately doesn't look good. So I'll try to copy the transform method used for each paper context to do proper perspective correction.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

shaunlebron commented 6 years ago

Works properly now. Will try to plug it in this weekend.

correct-transform

janpaul123 commented 6 years ago

Very cool! Thanks both for your contributions. Looking forward to the final PR! 😄

maryrosecook commented 6 years ago

Super cool!

On 10 Apr 2018, at 4:16 pm, Shaun Lebron notifications@github.com wrote:

Works properly now. Will try to plug it in this weekend.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

shaunlebron commented 6 years ago

Won't be able to get to it this weekend, so I left some breadcrumbs here: https://github.com/janpaul123/paperprograms/issues/41

janpaul123 commented 6 years ago

Superseded by #54. Thanks so much, very cool feature. 💯

maryrosecook commented 6 years ago

So excited this feature is in PaperPrograms. Thanks so much for making it general purpose and high quality, @shaunlebron and @janpaul123 !