search

erich666 / Mineways

Exports models from Minecraft for 3D printing or rendering
http://mineways.com
Other
399 stars 59 forks source link

USD Export - USD Preview Surface Support #70

Closed Vochsel closed 2 years ago

Vochsel commented 3 years ago

Hey there,

Noticed that USD support is limited to mdl material's for Omniverse. The "standard" for interchange between many USD based DCC's currently is still limited to the USD Preview Surface (https://graphics.pixar.com/usd/docs/UsdPreviewSurface-Proposal.html). It's not as comprehensive as MDL, however would suit most of the situations for a basic Minecraft world.

Is support for this planned? Happy to jump in and take a look if a PR would be considered.

Also noticed that currently there's no instancing used at all for USD export. This would lie outside of the scope of this issue, but if PR's or discussion was open, I'd be keen to help!

Cheers, Ben (@Vochsel)

Vochsel commented 3 years ago

Implemented the basics of this, however the biggest 1:1 issue will be that UsdUVTexture's don't support image filter-types/interpolation metadata. i.e. textures will be blurry not crisp nearest-neighbour/point sampled.

This isn't as much of an issue for higher resolution packs, however could be a blocker for wider consumption...

erich666 commented 3 years ago

Ben,

Thanks for this great note. I know extremely little about the USD Preview Surface, other than (I believe...) that they can co-exist with MDL descriptions of the same materials.

If you have some code that is ready to roll, I'd certainly love to fold it in as an option and provide it to users. The main thing would be understanding how I could test such code (I'd certainly learn a lot from seeing what it does).

UsdUVTexture's don't support image filter-types/interpolation metadata

Right, and Omniverse by default has the same limitation, which is why the "Custom material" option outputs slightly modified MDL files that give the blocky look. But, if you don't mind the smoother look, which is reasonable for 64x64 or larger texture sets, then things are fine.

Also noticed that currently there's no instancing used at all for USD export.

How do you see instancing being used? I think you mean that if someone wants to export individual blocks, the output blocks could be instanced instead of lumped into a single mesh. You can then animate the individual blocks, etc. This is a feature supported in OBJ export, but one that I haven't tackled for USD export because I want to see if anyone's interested in USD export (you are, so that's one :) ) and see if people want such a thing. I suspect you could do a poor-man's version of instancing by exporting USD after USD after USD using scripting to export one block each time and then pulling in each USD file individually, perhaps scripting import in some way. Messy, though.

Anyway, I'm game to provide some other USD export option, as long as something like Blender can consume it. The work shouldn't be much (especially if you've already done most of it :) ). So, let me know; feel free to write me direct at erich@acm.org.

Vochsel commented 3 years ago

Awesome, thanks for that debrief Eric!

they can co-exist with MDL descriptions of the same materials.

Yep that is correct, my current additions add the preview surface as the default surface material, but if anything ingests mdl then it will use that. It might be useful selecting from a list of material types to export, but for now I'd assume both are fine.

The main thing would be understanding how I could test such code

Sure thing, are there tests for the mdl code? I can add some to the UsdPreviewSurface code too. The preview surface material is a fair bit more limited/concise so imagine the tests would be smaller.

Right, and Omniverse by default has the same limitation

Ah gotcha, interesting, I know other USD Hydra Render Delegates (Cycles, Renderman, Karma) allow image texture shader nodes to set interpolation, it's a shame that USD doesn't by default. I've posted on their interest group, might get some traction (https://groups.google.com/g/usd-interest/c/uQHn5ODbvZY) but agree that this functionality would be useful regardless.

How do you see instancing being used?

Yep that example is a perfect one. Being able to manipulate each block after the fact would be great for edits after the fact, and FX done on the per block level. USD has a concept of a Point Instancer, which allows large amounts of instancing based on points (position. rotation, scale) and any number of prototypes (geometry). It could be implemented a number of ways, two of which would be:

  1. All blocks are stored in the one Point Instancer primitive, and each block is a unique prototype
  2. Each type of block is a Point Instancer primitive, and only one prototype (the block used)

I suspect one would be the most "USD" of approaches, but I haven't dug into the exporter code enough yet.

Yeah these would all be importable into Blender (once upstream blender has USD Import support). I actually wrote a version of the importer: https://github.com/tangent-opensource/blender/wiki/ta-usd-import which is open sourced. A few other companies are looking at taking that and submitting upstream to the Blender Foundation, but time will tell.

Houdini has great USD support, here's a screenshot of a test of the USD Preview Surface code in Houdini, rendered with Cycles https://twitter.com/Vochsel/status/1343784894682562560/photo/1

Anywho, I'll tidy the exporter and submit a PR (probably in a week or so).

Cheers, Ben

erich666 commented 3 years ago

Ben, this sounds great, I look forward to the PR. I don't have Houdini, though (nor do most users - I'd guess 95%+ of Mineways users are on Blender). So I'm happy to fold it in as a beta feature, expecting Blender will have this support someday realsoonnow.

As far as using the Mineways USD(A) exporter that currently exists, give it a try (but only if you have an NVIDIA RTX card, currently): http://www.realtimerendering.com/erich/minecraft/public/mineways/mineways.html#ovcreate

Omniverse Create is free, and easy to use for simple stuff. I like that it does interactive ray tracing with progressive refinement, all built in.

Vochsel commented 3 years ago

Hey Eric, Submitted the PR, happy to make it a beta feature if you point me to how best you'd like that integrated (opt-in, etc)

Blender might get support soon, it should support UsdPreviewSurface when it does....

I've used Omniverse, actually beta tested it. It has support for UsdPreviewSurface as a fallback, so you should be able to test there too (might have to disable the mdl material though).

FYI Houdini Apprentice is free and has full (export limited) USD functionality through Solaris. It's super simple. (https://www.sidefx.com/products/houdini-apprentice/)

erich666 commented 3 years ago

I've installed Houdini, ran Mineways with your code, and imported the file into Houdini using File | Import | Geometry...

It appeared - cool! But, no materials. What's the trick to getting materials to appear? And, please, feel free to write me at erich@acm.org - much more direct.

image

erich666 commented 3 years ago

@Vochsel Could you please give me a bit of guidance on the issue above (displaying in Houdini)?

Also, I seem to have found a problem with normals export, at least as far as Omniverse is concerned. This code you added:

` // Normals

        if (gModel.tileList[CATEGORY_NORMALS][swatchLoc]) {
            strcpy_s(outputString, 256, "        def Shader \"cutout_texture\"\n");

`

is causing problems in Omniverse Create when I load something that has normals. For example, export a simple grass block, but with the JG-RTX textures applied (part of Mineways), and when I load into Omniverse Create I get an error about:

token inputs:sourceColorSpace = "raw"

being illegal as a color space. What I'm concerned about is that having a normal map is applying a cutout texture - that seems wrong.

Vochsel commented 3 years ago

Hi eric!

Sorry, only just catching up on this now. I've emailed directly about the Houdini viewport/materials steps. Might require some back and fourths.

In terms of that colorspace question, that's interesting. I can try test those steps to reproduce, but unfortunately won't be set up to do so till mid next week.

I went off the official documentation for Pixar's latest UsdPreviewSurface implementation (https://graphics.pixar.com/usd/docs/UsdPreviewSurface-Proposal.html#UsdPreviewSurfaceProposal-TextureReader). Which lists sourceColorSpace as a legal attribute with 3 legal tokens (raw, sRGB, auto). This would be definitely supported as of USD-20.11. But maybe the version of USD that Omniverse is built with doesn't have it? (Actually yeah it was only added in 2.3 https://graphics.pixar.com/usd/docs/UsdPreviewSurface-Proposal.html#UsdPreviewSurfaceProposal-Version2.3 which was added about 5 months ago)

What is the actual error that gets reported?

If it does indeed not respect it in the current version of Omniverse, my expectation would be that it treats it "automatically", whilst not ideal, it might work out that the behaviour is fine.

Vochsel commented 3 years ago

Actually, just digging into the code, I see the error you pointed out.

            // Normals

            if (gModel.tileList[CATEGORY_NORMALS][swatchLoc]) {
                strcpy_s(outputString, 256, "        def Shader \"cutout_texture\"\n");
                WERROR_MODEL(PortaWrite(gModelFile, outputString, strlen(outputString)));
                strcpy_s(outputString, 256, "        {\n");
                WERROR_MODEL(PortaWrite(gModelFile, outputString, strlen(outputString)));
                strcpy_s(outputString, 256, "            uniform token info:id = \"UsdUVTexture\"\n");
                WERROR_MODEL(PortaWrite(gModelFile, outputString, strlen(outputString)));
                sprintf_s(outputString, 256, "            asset inputs:file = @%s/%s%s.png@ (\n", texturePath, mtlName, gCatStrSuffixes[CATEGORY_NORMALS]);
                WERROR_MODEL(PortaWrite(gModelFile, outputString, strlen(outputString)));
                strcpy_s(outputString, 256,  "            token inputs:sourceColorSpace = \"raw\"\n");
                WERROR_MODEL(PortaWrite(gModelFile, outputString, strlen(outputString)));
                sprintf_s(outputString, 256, "            float2 inputs:st.connect = </Looks/%s/uv_reader.outputs:result>\n", mtlName);
                WERROR_MODEL(PortaWrite(gModelFile, outputString, strlen(outputString)));
                strcpy_s(outputString, 256, "            float outputs:rgb\n");
                WERROR_MODEL(PortaWrite(gModelFile, outputString, strlen(outputString)));
                strcpy_s(outputString, 256, "        }\n");
                WERROR_MODEL(PortaWrite(gModelFile, outputString, strlen(outputString)));
            }

As far as I can see, the error is in name only. I'm naming the UsdUvTexture prim cutout_texture instead of normal_texture. This should be fixed. In terms of the actual attributes. My previous comment still stands. If Omniverse doesnt know about sourceColorSpace, then it will still error. Does that filepath attribute look correct for a normal texture to you?

If so, then the only error is in name.

erich666 commented 3 years ago

OK, I've put in the normal_texture fix, but still get the error in Omniverse Create:

2021-01-13 15:39:45  [Error] [omni.usd] Runtime Error (secondary thread): in textFileFormatYyerror at line 3110 of pxr/usd/sdf/textFileFormat.yy -- syntax error at 'inputs:sourceColorSpace' in </Looks/grass_block_top/normal_texture.inputs:file> on line 188 in file C:\Users\Eric\Documents\_documents\MinecraftOmniverse\USD conversions\test\test.usda
2021-01-13 15:39:45  [Warning] [omni.client.plugin]  29848: usd_plugin: syntax error at 'inputs:sourceColorSpace' in </Looks/grass_block_top/normal_texture.inputs:file> on line 188 in file C:\Users\Eric\Documents\_documents\MinecraftOmniverse\USD conversions\test\test.usda
2021-01-13 15:39:45  [Error] [omni.usd] Runtime Error (secondary thread): in Open at line 867 of E:\w\ca6c508eae419cf8\USD\pxr\usd\usd\stage.cpp -- Failed to open layer @C:/Users/Eric/Documents/_documents/MinecraftOmniverse/USD conversions/test/test.usda@
2021-01-13 15:39:45  [Warning] [omni.client.plugin]  29848: usd_plugin: Failed to open layer @C:/Users/Eric/Documents/_documents/MinecraftOmniverse/USD conversions/test/test.usda@
2021-01-13 15:39:45  [Error] [omni.usd] Failed to open : C:/Users/Eric/Documents/_documents/MinecraftOmniverse/USD conversions/test/test.usda
erich666 commented 3 years ago

This code is now active in the 8.02 release of Mineways. I hope to poke at it with Houdini and see how it goes - I haven't really tested it thoroughly. I'll keep this issue open, in case I find any problems I can't fix myself. If you have any further improvements, great, just let me know the fixes or submit PRs.

erich666 commented 3 years ago

To anyone interested in this feature, here are some further notes on UsdPreview and Houdini.

First, to use this in Houdini, you need to switch to a USD-friendly mode. Under the Build menu, pick Solaris. Right click in the lower right part of the app, in the big window labeled "Stage," and type "File" (really, just "fi" will get you there). Pick it. Left-click in the "Stage" area, you should see a rounded trapezoid labeled "file1." In the upper right, scroll down the dialog there and find the "File" line, just under "Enable." Put the location of the USDA file there. This will load the file.

You'll likely see black in the viewport. With the mouse there, hold down Alt and scroll the mouse scroll wheel downward. You should eventually get outside of your model. It'll look dark. In the lower left, Scene Graph Tree, there are yellow on/off buttons to the far right for each object. Click the ones for the DomeLight and Sun to toggle these off. You now should see your model more reasonably.

Holding Alt and the various mouse buttons move the camera. In the upper right of the viewport is "persp". Click that and you can choose "Karma (beta) Persp" to render the scene with a higher-quality (but slow) path tracer. You likely want to add lights to the scene to get shadows, etc.

At this point you're on your own. Best of luck! And if you figure out better settings for any materials, let me know - I'm barely a beginner with Houdini. Currently:

erich666 commented 2 years ago

I consider this one now implemented - thanks again for your help. I worked out the intricacies of opacityThreshold and what it means (if non-zero, opacity is a mask cutoff value, else it's true opacity). In the meantime, Houdini fixed its use of opacity, which had been broken. So, now the feature seems to work pretty well.