Blender Plugin Improvements.

[mifth]

Hello guys!

I did some small but very vital improvements to the blender addon:

• Possibility to pack All Linked objects and groups from other blend files
• Possibility to make all textures with relative paths
• Possibility to pack all Textures into the blend file. So that all will be in one blend file.

Here you can see these checkboxes: http://i.imgur.com/TRqNupg.png

Also i did some design changes:

• we make changes mentioned above
• save temporary file and apply all jobs to the temporary file
• After exporting to CGRU - open orginal file again. In this way all goes perfectly.

Here is my diff for "cgru/plugins/blender/render_afanasy.py":
https://dl.dropboxusercontent.com/u/26887202/blender/afanasy/render_afanasy.diff

Could you review it and commit? Thanks.

[timurhai]

Hi! Great! But better to use https://github.com/CGRU/cgru/pulls to merge code (not issues). Can you do so?

[mifth]

Hello! I have never did it. But i'll try to do it now! Give me a bit time. :)

[mifth]

Ok, i did the pull request.

[mifth]

Thank you a lot for merging! That's my first contribution. :)

I have some questions about another blender plugin improvements: 1) When i delete a job from a server - the temporary file will not be deleted. but it should be deleted as it's set in the plugin: https://github.com/CGRU/cgru/blob/master/plugins/blender/render_afanasy.py#L366 Is it a bug? Can it be because of the String is created a bit incorrectly somehow?

2) Question about the code for "images" here: https://github.com/CGRU/cgru/blob/master/plugins/blender/render_afanasy.py#L342 What is it for? It takes all images to upload to the server?

3) Question about 'BLENDER_RENDER' and 'CYCLES'. They are set as String line. Are you ok if i make it as Dropdown list? Like this: http://i.imgur.com/26puvKx.png We can also add other renderers like 'YAFARAY', 'LUXRENDER', 'V-RAY', 'OCTANE'. Just for future.

[timurhai]

1) Check job post command, if it is correct - it is not a bug, may be some permissions problem or system job problem. Check system job, it creates "system" tasks to delete temp files. ( to see jobs from all users use admin password - default is IDDQD.

2) block.setFiles([images]) needed to open folder, preview frame and generate a thumbnails for a job and it tasks.

3) I am OK.

ps I am using blender, but very rare. I am grad that some more experienced person helping with this plug-in!

[mifth]

1) i'll try to test. 2) ok 3) cool! I'll make pull request soon.

Thank you! Man, you do great job. I really like the project. I hope to be helpful in blender things. Please, don't close this issue. I'll put more ideas here to dicuss.

[mifth]

Hello again.

About 1) issue. Temporary files still exist. Even if a job was deleted. Here is me job: http://pastebin.com/pWkBdeX4 The file was: /home/mifth/Desktop/123/test.blend.0808-181056-496.blend

[timurhai]

Hi. Check system job post_commands task log.

[mifth]

I tried to restart and reset it but it does not help http://pastebin.com/8VMH1knE

[mifth]

Oh!!! I restarted server. And it works perfectly! It deletes files! Sorry.

[timurhai]

Fri 08 Aug 18:29.54 : Task[0]: Finished error on "mifth": mifth: "test.blend.0808-170754-582": deletefiles "/home/mifth/Desktop/123/test.blend.0808-170754-582.blend" This command can finish with an error only in 2 cases:

1. File does not exist.
2. Permissions denied.

[mifth]

Is it possible to add a check if a file exists? Just not to stop other commands.

I did a pull request for DropDown list https://github.com/CGRU/cgru/pull/98 Now it looks cool :) http://i.imgur.com/IanNUuK.png

[mifth]

For the dropdownlist we can also add other renderers: V-RAY - http://www.chaosgroup.com/en/2/vray_blender.html OCTANE - https://render.otoy.com/shop/blender_plugin.php YAFARAY - http://www.yafaray.org/ MITSUBA - http://www.mitsuba-renderer.org/plugins.html LUXRENDER - http://www.luxrender.net/en_GB/index INDIGO - http://www.indigorenderer.com/blender/

We can just list these renderers. Possibly, people can add theit\r setup somehow through CGRU python.

What do you think?

[timurhai]

Yes, we can. But there is no field to specify some custom engine manually. Is there a way to edit some string parameters with this pull down menu?

[mifth]

You mean to edit DropDown menu in blender? In blender we can't edit it. Only if we add them into the script. Or we will have possibility to change the list of renders somehow.

[mifth]

Also we can make a list like this: http://i.imgur.com/N0zAcmH.png In this way we can add a custom rennderer.

[timurhai]

And this list will be stored in a scene?

[mifth]

Yes, it will be stored in a scene file. We can have default - Cycles and BLENDER-INTERNAL. But other Names can be added by users.

Or we can store it in the addon. Like this example http://i.imgur.com/Xdm0ydN.png (Exchange Folder) And this list will be always the same in all blend files.

[timurhai]

I think that in add-on will be better. Do you?

[mifth]

Let me test. i guess it should be fine.

[mifth]

Hello. I did pull request for custom renderers improvements. To save custom renderers - a user should save default scene (Ctrl+U) or in the FileMenu(as in video). Here is demonstration http://youtu.be/RCWD1sff-Io Here is Pull Request https://github.com/CGRU/cgru/pull/99

[mifth]

The only question is - should you change some thing for Afanasy? Take a look at lines 327-332 https://github.com/CGRU/cgru/blob/925c4b4152f3d6774ba53d2f3c4486a1a8b97358/plugins/blender/addons/render_afanasy.py#L327 Afanasy set Only official renderers. But on the server i see Custom renderer http://i.imgur.com/dUSooJY.png

[timurhai]

Hi. Great work with custom renders! We can put video in documentation (video itself, or a link). Have you some best quality? You can add you as an author in a plugin description!

That lines in a code, set parser (python class) to parse progress from task process output. There is no other parsers in cgru for now.

Parser, as any other python class can use multiply inheritance. Good example is Houdini. There is a hbatch parser, that parses frame, and mantra parser that parses frame progress. https://github.com/CGRU/cgru/blob/master/afanasy/python/parsers/hbatch.py https://github.com/CGRU/cgru/blob/master/afanasy/python/parsers/mantra.py They can be used separately. But when we use mantra directly from houdini (hbatch generates scene and renders it with mantra) we should have a parser that parses and frame number and frame progress. In this case hbatch_mantra parser can just inherit both parsers to reuse code. https://github.com/CGRU/cgru/blob/master/afanasy/python/parsers/hbatch_mantra.py

Also parser can parse rendering image for thumbnails and activity to inform user about what is going on, for example, loading scene, generating shadows and so on.

[timurhai]

But parser existence will not affect will be a scene rendered or not. You you can't render with VRAY from blender check task service and where your farm has such service. ( you can double click in afwatch task and render and paste info here )

[mifth]

Thank you. :) The video is 720 quality. it seems youtube has caching issues. no problem to put it to docs.

about parser: it runs blender and makes thumbnails? This is its general purpose.

i have some ideas about big refactoring of the addon. What if we put the addon to the left panel in 3d viewport? In this case there will no need to change renderers in blender. a user can set his favourite renderer and export to afanasy. what do you think about it?

[timurhai]

Hi. Parser just parse info from output, this info needed for frame(s) progress, thumbnails, activity. Also parser can find out an error and ask render to stop task process. Or that task is already done and no matter that process not finished. Or just warning to inform user that something is not good and he should check output. https://github.com/CGRU/cgru/blob/master/afanasy/python/parsers/parser.py#L108

It will be good to have an ability not to switch engines! But i did not got where you want to place it?

[mifth]

Hi.

Thanks about the parser info. We can put the addon here at the left panel: http://i.imgur.com/CKoctZ2.png We can create CGRU or Afanasy tab and put all stuff there. In this way, the only thing which Afanasy Client needs - is just open a blend file and render it. :) And no matter which renderer. Blender, V-Ray, Cycles, Octane etc... The only thing is the Afanasy Client should have all installed software(engines) to render. We can leave renderers types so that Afanasy Client knows what renderers are used in the blend file.

[timurhai]

Yes. This way will be definitely better!

[mifth]

Great! I‘ll try to refactor it. how would you prefer to call the tab? Cgru or Afanasy? In future we can add other cgru tools there.

[timurhai]

Yes, it should be CGRU. ( at least i want to add there open RULES item )

[mifth]

Hello! I refactored blender plugin as we talked before. Sorry for the delay. Here is my pull request https://github.com/CGRU/cgru/pull/102 Here is video test: http://www.youtube.com/watch?v=7sEwL-e7iDk

Benefits of new plugin:

• No need to set renderer. Any renderer works if a machine has such an engine.
• It works in ObjectMode.
• CGRU Tab for all CGRU tools

Now you need to use "cgru/plugins/blender/addons/cgru_tools/" folder instead of "cgru/plugins/blender/addons/render_afanasy.py". You can link "cgru/plugins/blender/addons/cgru_tools/" as you did it with render_afanasy.py

The only thing I did not solve: I commented the parser for the blender. Lines 352-356 https://github.com/mifth/cgru/commit/b0523eee7a136c1b7c2c7ff256455de81313d92b#diff-b7b565eef2e5366b4073be51d3be9634R352 I think we need to make a unified blender parser for all blender renderers.

Also, i commented all old code so that you could check what was changed. If all works ok - I can remove commented code.

Could you review my pull request?

[timurhai]

I have comments/questions, but very busy for now )

[mifth]

No problem. You have added me to CGRU. Thank you! :) This is thegreates honor for me.

[predat]

mifth,

Thank you for you addon. It's more user friendly then before...

For parsing cycles rendering log output, we need to change keypart variable from 'blender_cycles.py' parser class to: keypart = "Path Tracing Tile "

after that you can uncomment the parser line in you addon.

[mifth]

@predat hello. No problem. :) Sorry, what do you mean? You mean lines 352-356? https://github.com/CGRU/cgru/blob/f3376ce46726af32b0d4811bbb6700ab8d43aa55/plugins/blender/addons/cgru_tools/afanasy_tools.py#L352

[predat]

yes, it is.

[mifth]

As i wrote in this post https://github.com/CGRU/cgru/issues/95#issuecomment-54064944 we need to make a unified blender parser which will suit to all renderers. Right now we have blender_internal.py and blender_cycles.py. We need to make only one parser which will be used for the blender. I think. Is it possible to do?

If no, i can uncomment these lines sure. But if you will use VRay, for example? Which parser will suit you? That's why it would be better to have one unified parser.

[predat]

I don't think you can know which engine you are use from parser point of view. That's why there is multiple parser for blender. Same for Maya or Houdini... Correct me timurhai if i'm wrong.

[mifth]

You can know which engine is used now. http://youtu.be/1IVhujJamus

[predat]

In your addon, yes but from parser ? How do you send engine type to the parser class ?

[mifth]

Afanasy gets the list of engines. http://i.imgur.com/YPaOsrn.png I guess the parser can get this string?

And what is the difference between BlenderInternal parser and Cycles parser? Why we cannot get the same output?

[timurhai]

Hi. Here can be several solutions:

1. Write separate "blender" parser and other parsers as "blenderinternal", "cycles", "povray" and so on. This will help to reuse the same code. "blender" parser should parse only frame (each task can consist of several frames). "cycles" parser should parse only one frame progress. And write parsers such as "blender_cycles", "blender_internal", "blender_povray" each should just inherit "blender" and "cycles" parser. Good example is houdini-mantra: https://github.com/CGRU/cgru/blob/master/afanasy/python/parsers/hbatch_mantra.py Benefits of such approach is that you can use "mantra" parser for stand-alone mantra, and in any other 3d soft that can have mantra render engine (for example prman or vray exists in lots of softs). This way we must know render engine during submission.
2. Write one universal "blender" parser for all engines. Submission script should send engine type (in "custom_data", for example, you can send anything). Or parser can get render engine type directly from output. Each render engine usually has some keyword. This way we have less files but we can't reuse code for stand-alone "cycles" or "povray". But whether blender render engines has stand-alone mode? Such engine as mantra, vray, prman has: 3d soft can generate files in a special format for them to render, than launch separated program to render this files. And also, does some other 3d soft uses cycles or other blender engines.

[timurhai]

An old plugin file should be removed, i think. Do you use something from that file?

Also, it is good for parser to parse output image(s). This needed for thumbnails and preview. We can get images path from render settings, but parsing images is better. Sometimes soft use layers, takes and so on, and in this case render engine add some suffix output files names, such as "_occlusion" or "_backgroung" and you can't imagine what suffix can add render engine on rendering stage.

• i do not know whether blender engines do so for now, but later they can.

[timurhai]

Also! Best way to render is not to run program (blender) directly, but ask program to launch some custom render script. Nuke, Houdini, SoftImage do so in CGRU. This way makes render pipeline very flexible. Example: "program -s some_script [parameters]" program launches our python script and give [parameters] to it. In this script we read parameters, that tell us to open a scene, and render frame(s) N in a file(s) F. Benefits: We construct task command with parameters for our own script. We can pass any parameter and do something in a script. In a script we can:

• change paths for multi-OS farm (we simple detect OS in python and do changes),
• tile-render control
• enable\disable simulations, re-simulate,
• check rendered image(s)
• render an image in a local temp folder and move to some shared location on success (this can speed up network)
• help parser, as we can just print frame number and image path in own format
• lots of things can be imagined here, i just tell you what other soft with CGRU can do.

[mifth]

Hello guys. I'll answer in an hour.

[mifth]

Hello again.

About the parser: I'm looking at blender_render.py and blender_cycles.py and they have only one line difference. I think we can make unified blender parser. We can override some methods if we need for different renderers. What do you think?

Question: What if I want to render with 2 engines at the same time? For example like this: http://i.imgur.com/krDBWV7.png Here is test file: https://dl.dropboxusercontent.com/u/26887202/blender/afanasy/test_engines1.blend For example, I can render the main scene with Cycles but render particles with Internal. Blender can do it. But output will be only one.

Do we really need to stick to engines? Can we get thumbnails by different way?

About old plugin - yes we can remove it fully. I'll do it.

[timurhai]

We can use one unique parser. And later, if cycles appear in some other soft, we can split it and use class inheritance to reuse code.

[mifth]

Ok, cool. Then i set blender_cycles.py enabled in the addon. And blender_render.py will be deprecated. Right?

Forgot to say - Cycles and BlenderInternal are only in blender.

[timurhai]

So, may be call it just blender, without cycles?

[mifth]

Ok, cleaned up the blender plugin and set blender_cycles.py parser always. https://github.com/CGRU/cgru/commit/91816665873dc96876ad12d152af11a92e5d06ab

If i rename blender_cycles.py then i get no thumbnails. Dunno know why.