Filmic Color Management :: New feature

[birawaich]

STATUS Lately, the Filmic Blender color management has gotten much attention and is now being more and more used among the blender community. See filmic-blender for more information.

IDEA The idea is now to add a feature to allow setting materials prepared for filmic blender. This only affects all materials using the emission node, since filmic blender color management needs "more light" than sRGB color management.

HOW I am currently planning on creating a Minecraft Cinematic. Thus I will find the different necessary values. Depending if I understand the code I can add it myself or I will just add comments ;) (Thank you!)

EXAMPLES Bellow are some examples I made playing around: only mesh lights (lava, redstone lamp, little glowstone only mesh lights (torches in cave) One notices the fireflies... I couldn't get rid of them, both render use 256 samples. Results with using lamps: mesh lights vs lamps -- torches (only lights added, all torches have light emitting mesh) mesh lights and plane lamps (only redstone lamps were done, lava, glowstone is still only a mesh light, also no fill light has been added) this image shows the fireflies issue for torches... notice the sample count So adding lamps is necessary. Keep in mind though that I am a blender newbie, so maybe there are way better results possible. I might write a script to add lamps automatically. (Just based on the project I want to do (cinematic) which would require me to add too many lamps by hand. I will share the code if I get it. No promises though. It might be a nice feature to add to the add-on as well).

TECHNICAL In the above renderers I set the materials up in the following manner: example redstone lamp material The color ramp node makes sure only the bright areas glow. This is then plugged into an add shader so the non glowing areas are not black. The top part is generated by the add-on.

-> What do you guys think?

Note The map seen is the Hermitcraft Season IV vanilla map. It is a part of spawn below the iron titan built by Tango. Map (under the map tab - if you want to look at it in MC)

[birawaich]

UPDATE I started writing the lamp script. It is far from perfect, since I lack the skills to select all connected vertices and so determine the correct coordinates for the lamp. So I exported all torches and then worked with groups. It simply duplicates (same data block) predefined lamps. The results are not perfect, but for vast maps it should work out nicely. I'm not sure if you want to embed it in the addon, but I will definetly share the code (as soon as it is finished and tested.) Materials: Is it correct that I can "simply" add to the materialsCycles() function in materials.py? If so I will add materials like the above presented + displacement and maybe some more fancy individual nodes. This will all be dependent on booleans (whether or not to use the standart material settings or the ones I will eventually write). These boolean values can then be linked to buttons in the UI. How does this sound? Below an example of progress so far (256 samples): torch and redstone torch with displacement (ground) and shiny quarz

[TheDuckCow]

Hey, looking really nice! Could I understand a little how you're accomplishing the above - are you placing lamps above the torches, or just duplicating the geo and then moving upwards (and hiding direct view from camera)? If the later, there is some node setup that can be used to make it work without duplicate data block and only modify the original material using light paths. And yes, and I'm happy to configure it into master. If you'd like, make a fork of MCprep and then create a function with your code. Then, call this function inside the cyclesMaterial function call as you correctly guessed.

I like that shiny quartz, looks good.

[birawaich]

Thanks :)

I'm placing lamps. To determine the location I am scanning the material slots of the objects (then set the object center based on geometry, increase the z coordinate a bit, and place the duplicated lamp). I will upload the code onto my GitHub, as soon as I did all light sources (or at least the ones that should receive a lamp and not only rely on mesh emission). Sounds good with the function call. I'll make a fork as soon as I figured out the material properties I actually want to use ;) I have to note that I might not have too much time for development. I hope to be done in summer, maybe it will take me till fall. Always depends on how many evenings I have nothing to do. So just don't except me being done anytime soon. Also I still have to learn the Blender Python API. (and Python).

I will post further updates here.

[birawaich]

UPDATE I continued coding the addLamp script. Now it is in a state that allows for "easy" appending of new Lamp types and the "ordinary" point and area lights are supported (torch, glowstone, etc). See pictures below ;)

Whilst playing around I discovered the MeshSwap option, or rather realized that it is also adding lights which are already in the obj. I removed the lights added by the MeshSwap and only used "my" lamps: see below:

No MeshSwap: only lamps created by script in dev MeshSwap and created lamps via script combined behind the scenes

NOTES ON LAMPS

• You can see that the area lamps of the blocks only shine outwards. The rest is done via Mesh-Emission of the material. This creates some apparent darker surfaces. Would you recommend letting it as it is (when light bounces of other objects it should only become better) or multply the lamp count by 2 through adding the same lamp (or maybe a less powerful one) shining towards the obj?

NOTES ON FURTHER DEVELOPEMENT

• Since the MeshSwap does not (yet) have all possible emitting geometries with lamps, something is still needed. These are some ideas I have for further development:
  1. Continue as is, making lamp script, and than manually remove the lamps on the torches when MeshSwapping
  2. Only focuse on creating rigs for the MeshSwap and somehow adjust the lamp values based on the render engine and wheter or not "Filmic Blender" is used.
  3. Create a function in the MeshSwap which stops any light to be imported (only runs if lamps were already created).
  4. Create a function in the MeshSwap that adds the lights when mesh-swapping. (only if a certain tickmark is ticked). I.e. intergrate the light script into the mesh swap class.
• Also I have seen that you did something really smart with "Facebook" (MeshSwap) to figure out where the torches should be placed. I couldn't do something like this and imported "individual block" objects with the help of MCprep instead and used them to figure out the position. While this option works, it is definitely not resource efficient or nice. Maybe you can help me understand and using your "facebook" method (I hope this is the right code xD) at a later stage.

NOTES ON MESHSWAP

• The redstone lamps seem to be mispositioned. The blend file might simply be to much edited or there might be a slight bug in there.
• Would "prepping all materials of the imported meshes" be helpful to the user, or more an unwanted action? I haven't used the addon during production, so I am not sure, but it might be a nice feature.

QUESTIONS

• What direction should developement take?
• If I want to run the "combine material" function i.e. located in one of the MCprep classes, how do I access it? Or how would I check, if MCprep is installed?

what further developement includes

• [ ] creating of master lamps if none are present
• [ ] automatically importing .ob and add to groups, and afterwards remove them
• [ ] better error handling --> I'll upload the script to my GitHub or maybe to a forked MCprep (I still have to figure out how GitHub works) once it can run in any blender environement with MCprep installed and the .obj somewhere accessible (i.e. the user does not have to create lamps, groups etc)

PS: ... and once the lights are figured out I will start my animation project which will include creating the "createMaterialsForCyclesAndFilmicBlender() Function ;)

ChangeLOG: 17-05-02: added option 4 in the "NOTES ON FURTHER DEVELOPEMENT" section

[TheDuckCow]

Hey sorry for the delayed response, busy with a few things at once. I like the progress you're making and want to help push you in the best direction.

Only focuse on creating rigs for the MeshSwap and somehow adjust the lamp values based on the render engine and wheter or not "Filmic Blender" is used.

I like this direction the most, but it would and should work at the same time as your script that also adds lights to materials... so long as those materials aren't about to be meshswapped. That's because I can image there are times where the user wants to meshswap a torch into a full 3d model/with particles, and then other times when they want to keep the original 2D-texture-based model but still place a lamp above it. I think it should be made as a part of the MeshSwap operator, which after all we've talked about would look something like this:

meshswap_lamps = bpy.props.EnumProperty(
        name="Lamps",
        items= [('group', 'With groups', 'Repalce light emitting blocks group instances, containing 3D blocks and lamps'),
                ('material', 'By material', "Add lamps above light emitting blocks, withotu modifying original light-emitting blocks"),
                ('none', 'Skip lights', "Don't add any lights, skip meshswapping for light-emitting blocks")],
        description="Set how lights are added (if any, based on selected materials)"
        )

And the UI of meshswap accordingly (most of this UI is a new proposal.. but focus on the HOW TO ADD LIGHTS section)

So, to explain this:

• Group: does it as it normally does, replaces e.g. redstone lamp or torch with a group which contains a light. If filmic is enabled, updates the values for those lamps accordingly (see thoughts on ID properties below)
• By material: This would be your script which places lamps above materials. It would skip meshswapping the blocks themselves for light-emitting meshes and put those lamps instead; if filmic is enabled, use those applied values you have saved.
• None: This skips adding lamps, but still does the meshswapping. ie this would add the 3D redstone lamp object instead of a group with a light, still doing the 3D meshswapping but just not bringing in lights. This could be done by deleting the lamps brought in from the groups.

After quite some thought, I think this works as it provides all the options without hopefully giving the user too many options. Because of the way lamps are added in groups in the source file, it's easy for an individual to modify positions/values slightly the way they want to fir their neds. Plus I like the idea of being able to adjust the lamps in the MCprep meshswap file because it allows people to create their own assets in a custom way and still leverage it, as opposed to more hard coding. But it's still very nice to be able to just add lamps without the geometry/groups as you have already implemented.

So, if you want to take a pass at first finishing your existing script to add lamps above materials (irrespective of MeshSwap), and then also updating the way existing lamps are setup (e.g. location/placement/count) in the meshwap file, go for that, as well as adding for new lamp sources I don't have there yet. It would be good to keep them using internal materials and then convert to cycles using our update function.

What direction should development take?

My recommendation:

1. Work in a fork around getting more of the lights working the rest of the way. As for checking that filmic is installed/available, see how it's done in part here (note the "filmic_is_installed" function is not how you want to check against it). When you get to a point where you think your changes can be committed, I can help you submitting a pull request into the MCprep dev branch.
2. Finish working on whatever to make your lamp-adding above materials work and be production ready (and functionally working independent to whatever meshswap does)
3. If you have time :) add/update the assets in the meshswap file to have the ID properties for filmic values and add the missing blocks (once there, I can help hook up with meshswapping).
4. Then, I can largely assist with the integration of the different settings. That includes the part of code which adds meshswapping of light blocks without adding the lights (one of the settings proposed above)

If I want to run the "combine material" function i.e. located in one of the MCprep classes, how do I access it? Or how would I check, if MCprep is installed?

You can run the operator by calling bpy.ops.mcprep.combine_materials(selection_only=True), you can see if MCprep is installed by for example seeing if any MCprep operators exist via hasattr(bpy.ops,'mcprep')

Note: I see there is currently a bug with the combine_materials operator - well, technically it's just that combine materials is working too well. Even if selection_only == True, it seems to consolidate all materials with the same base as any existing materials (and not just among the materials on the selected objects). I'll be working on a fix.

The redstone lamps seem to be mispositioned. The blend file might simply be to much edited or there might be a slight bug in there.

Hmm I find they work for me (using jmc2obj more), maybe there is an issue with the Mineways method. The two coordinate systems are off from each other by 0.5,0.5,0.5.

Would "prepping all materials of the imported meshes" be helpful to the user, or more an unwanted action? I haven't used the addon during production, so I am not sure, but it might be a nice feature.

It would be nice to have it in the popup menu that appears before MeshSwap runs, and by default be off. See the mockup screenshot above as I have added it there.

[TheDuckCow]

(Btw I just pushed the UI change into the dev branch, for reference)

[TheDuckCow]

Ask one other recent development that validates this work effort, appears filmic_blender just got committed to trunk! https://twitter.com/plasmasolutions/status/861246943602855936

[birawaich]

This is awesome news! What do you estimate when Filmic Blender is in "vanilla Blender"?

Thank you for the detailed answer! I will respond as soon as I got time to think your answer through and completely understand it. (Which means this text will be "edited" as soon as I get around. Life's busy here too ;)

[TheDuckCow]

Sounds great. Actually you can just make a new comment so I don't miss the notification. Hard to say, but I'd expect it's being built into 2.8 which will be a long time off. Not sure how long you've been using blender, but the transition from 2.7 to 2.8 will be akin to 2.49 to 2.5 in terms of UI and function overhaul.

[TheDuckCow]

Hey just wanted to give you the heads up I marked this as a milestone feature for MCprep 3.1.0+, no rush as that's a ways off but looking forward to hearing your responses when you have time

[villager-uncle]

villager's IK didn't work,fix it please:)

[birawaich]

Hi, just wanted to give you a short update why I am not getting anything done:

The thing is that I am in the army now.. So I don't have time anymore :( I hope things will get better but right now all development (as well as ordinary messing about with blender) is on halt. I cannot imagine ever stopping to use blender, so in 2018 I should someday get my stuff done... One questions: How do I program elements of an add-on the best? Up until now I have always just worked on one script which I ran for debugging. I was just wandering if there is a smart way to develop parts of an add-on without continuously removing and adding the edited add-on again. If you find time it would be nice if you reply. But honestly, I don't need the information anytime soon. Otherwise I'll figure it out someday next year ;)

[TheDuckCow]

Thanks for the update, sounds good - and good luck in the army! If you'd like, feel free to fork the repository with the code or otherwise share what you have thus far. I won't work on it but it may be good to see what is there so far so that in 2018 when you return, it will have a place to slide back into. As I mentioned, with blender 2.8 going to support filmic more natively, it will be an important thing to be able to support.

[TheDuckCow]

Hey @birawaich glad to hear you're back! I'm sure you've had quite a few adventures, hope all has been well. I'm in the middle of a hefty update to MCprep, but in context to the above conversation, I wanted to let you know that filmic will be the default color management system in blender 2.8 onwards, set to release later this year (but, I of course want to maintain backwards compatibility and such).

On the note of light intensity, I came to realize that even before we talk about changes in light intensity with Filmic, the exisitng emitting defaults for cycles as it is are not really appropriate. I am going to be implementing the use of a light falloff node which improves this greatly, you can see a sample of this setup here:

Note how the camera only sees strength-1 emissions so textures don't look blown out, but the rest of the scene perceives a much brighter light. This may not be strictly PBR, but looks much better and also ends up with less noise generally. Stay tuned for a related push to the dev branch

<edit: I should say, less noise than the previous version of just using emit planes; using lamps generally will still be less noisy>

[ghost]

is this thread still on?

I've been looking forward Filmic in the past but the colors just looked bad, rn I'm trying using texture maps with the principled shader (and a bit of extrusions) and it looked like this

I'm ok with the result but I believe it can get better

[TheDuckCow]

Hey, yup this is still an open topic @Zophiekat. In fact, Filmic has actually been made the default for blender 2.8 (still in alpha) so it will be something everyone should get quite used to. Typically scenes made with cycles pre-filmic won't translate nicely to filmic without some extra work, so MCprep ultimately will help with this.

https://github.com/TheDuckCow/MCprep/tree/dev (if you have trouble installing, then just hang on for a couple days.. going to push out an official call for testing with a pre-packaged zip file).

As for the principled shader, yes... MCprep is verrry close to a big update which includes using the Pinrcipled shader. In fact, if you're willing, why not give the current dev branch a try. (And let me know if anything is broken).

Liking the look of your crafting table! I presume you have multiple materials for the different materials? <edit: I see now you said you had texture maps, so nvm> I might make a similar update to the crafting table that comes with MeshSwapping.

[ghost]

@TheDuckCow The crafting table has only 4 materials, Top, Front, Side, Planks. this is a special case but most of the blocks I'll make will have 1 material, I could combine all crafting table textures into a single image so I would have 1 material only

[StandingPadAnimations]

I think that Blender's current way of calculating brightness(watts) is not really that good. Lumens are the industry standard and can make adding lights easier(since people could google the brightness rather then eyeballing it). I have created a node for lumens and mesh lights and I'll try working on a Python script to automatically replace emission nodes with it(and maybe add a lumens slider for EEVEE).

[zNightlord]

Passing by this and I don't know is this still considered as an open topic since AgX started to become a thing.

[StandingPadAnimations]

Based on the original request of this issue, I'll close this as completed