Create a Learn page for p5.Framebuffer

davepagurek commented 1 year ago

Increasing Access

p5 is soon adding support for a new type of object, p5.Framebuffer: https://github.com/processing/p5.js/pull/6072 It isn't obvious to a non-expert when one should use it vs, for example, a p5.Graphics, and it's also worth showcasing some of its functionality that one might not know they want until they hear about it (e.g. floating point textures.) These topics are a little too big to put in the docs for its methods, so a dedicated learn page seems like the best place.

Most appropriate sub-area of p5.js?

Learn

Feature request details

Ideally this would go over:

How is a p5.Framebuffer different from p5.Graphics?
- Why might you want to use floating point textures?
- What kinds of things can you do when you have access to depth information?
- How much faster is it to draw to a framebuffer instead of a p5.Graphics?
- Using a framebuffer texture as data in a shader
How framebuffers relate to the canvas they're created on
- When to use a framebuffer on a p5.Graphics
- How do autosizing and fixed sizes work
- How to use cameras with framebuffers

Some resources to maybe pull from:

This sketch uses 120 framebuffers to make a "rolling shutter" effect with the webcam, which is able to run quickly even on phones
This sketch uses the depth buffer to add focal blur
This sketch implements a particle system with the data stored in a texture

ShivangMishra commented 1 year ago

I'll work on this over this weekend. @davepagurek other than the code in #6072, the readme of original p5.framebuffer library, and the examples you mentioned, would you like to suggest any other resources I could utilize for our learn page?

davepagurek commented 1 year ago

Thanks @ShivangMishra! If you want to have any interactive examples, the PR isn't merged yet but here's a build of p5 off of the PR that you can work with for now: https://gist.github.com/davepagurek/2a9118d03c4d81db86037ad37aa56639

openFrameworks, a library similar to Processing for C++, has these docs on its version of this, ofFbo: https://openframeworks.cc/documentation/gl/ofFbo/

I also linked some more examples in https://github.com/processing/p5.js-website/issues/1341 (I also just edited it now to add some more) that might be helpful if you're looking for things one can do with framebuffers.

ShivangMishra commented 1 year ago

@davepagurek , @aferriss ... so I've started working on the Learn page. Here's a rough outline - https://github.com/ShivangMishra/p5-fbo-learn-temp/blob/main/learn-framebuffer.md . I've taken most of this text from the readme and comments of @davepagurek and @aferriss on the previous fbo libraries. Please have a look and give suggestions, I'll work on it in my temporary repo until we have a final draft. While you review it, I'm working on the code examples.

I need a little help on these points : -When to use a framebuffer on a p5.Graphics
(How can we use framebuffer on a p5.Graphics ?)

-How to use cameras with framebuffers (I found defaultCamera() and createCamera() functions relevant to this point, but not sure what else I should cover)

-How much faster is it to draw to a framebuffer instead of a p5.Graphics? (Any suggestions on how I should compare the performances on some benchmark?)

Thanks!

davepagurek commented 1 year ago

Thanks for your work so far @ShivangMishra! I haven't had a chance to take a look just yet but I'll get back to you in the next few days 🙂

davepagurek commented 1 year ago

When to use a framebuffer on a p5.Graphics

I think generally you'd only use this if you need to have a 2D main canvas with a little bit of 3D stuff on it, so you'd already be using a createGraphics(w, h, WEBGL). You'd then add a framebuffer if, in that 3D bit, you need float textures, or need to render something in layers.

But otherwise, if the main canvas is WebGL, you can probably just use framebuffers over WebGL graphics.

How can we use framebuffer on a p5.Graphics?

All the methods to make framebuffers on the main canvas (e.g. createFramebuffer()) can also be called on a graphics object (e.g. graphics.createFramebuffer()). You can then use that framebuffer when drawing to that graphic.

How much faster is it to draw to a framebuffer instead of a p5.Graphics? (Any suggestions on how I should compare the performances on some benchmark?)

I made a little test a while ago animating some letters coming in by either (1) drawing each to a createGraphics and then drawing that to the main canvas, or (2) drawing each to a framebuffer and then to the main canvas. There are some little issues like letter spacing I didn't fix but mostly the test is meant to show how fast you can draw canvases to a WebGL context vs drawing framebuffers. On my 2015 Macbook this is a difference between 20fps and like, <1fps. It's not the best test because if you're drawing text by letter there's no reason to use a framebuffer OR a graphics here, but maybe there's something similar that makes a bit more sense but tests the same thing?

ShivangMishra commented 1 year ago

Thanks for your work so far @ShivangMishra! I haven't had a chance to take a look just yet but I'll get back to you in the next few days 🙂

Hi @davepagurek did you get some time to review my draft? While waiting for your reply, I too got busy with some other projects. Would you like me to continue working for making the Learn Page?

davepagurek commented 1 year ago

Oh I'm sorry, I didn't realize that I missed half of your question! Here are some notes:

Since we currently don't provide a way to access the stencil buffer, we can probably omit mentioning it for now. (Maybe later if we implement masking with this we can add it? https://github.com/processing/p5.js/issues/3998 :eyes:)
Maybe we should explain how drawing to a framebuffer is a bit different than drawing to a p5.Graphics? Maybe we could show two versions of some simple code to draw a sphere to a texture using framebuffers and using graphics. The difference would be:
- With a p5.Graphics instance, let's say it's called g, all the calls to draw to it are done by prefixing them with g. (e.g. g.sphere())
- With a framebuffer, all the calls are done on whatever context the framebuffer was made on, and you draw to the framebuffer by putting draw calls between begin() and end() calls (e.g. fbo.begin(); sphere(); fbo.end())
In the depth section, we'll probably need to explain a bit about the range of the depth values. I'm guessing we'll need to say:
- Depth gets squished into the range of 0 to 1
- 0 is as close to the camera as the camera allows; 1 is as far away as the camera allows before it gets clipped
- You can change what becomes 0 and what becomes 1 by editing the near and far params, respectively, in perspective()
- Maybe a good example of this would be a sketch that draws a bunch of spheres or something and visualizes the depth, and some sliders to control the near/far parameters so you can see how they affect the depth numbers? (Let me know if I can help provide a sample sketch for something like this!)

davepagurek commented 1 year ago

Also if you feel like making a PR that has your .md file in it (you can mark it as a draft while we're still working on the text), I can add comments on different lines and be notified when you make changes 🙂

davepagurek commented 1 year ago

Hi again! We're going to start getting the ball rolling on a new p5 release soon. Would you be interested in continuing working on this in the near future? If not, that's totally fine too, we can update the website after the p5 release, so there's no rush! We probably want to have something available for when the new release comes out, so I could also make a small first version of a learn page for when we launch the new version, and then build on top of that after, if that's OK with you?

processing / p5.js-website-legacy