Using viewer in another project

[d07RiV]

Hello,

I'm working on a data viewer/map parser for WC3, and I'm looking to add a model viewer. This seems like a perfect candidate, but I'll need to strip some code I don't need, like loading BLP/TGA textures (I already have all textures available as PNG). Is it ok if I include this repository code (modified) in my project?

My repo is at https://github.com/d07RiV/wc3data and website at https://wc3.rivsoft.net/

Also there's an issue with your code when transpiled with Babel - extending native classes isn't supported in Babel 6, and in Babel 7 it needs to be hinted when they're not extended directly. This causes a problem with this class:

export class SceneNode extends nodeMixin(Object) {}

In transpiled code, instantiating SceneNode results in a plain object :(

[d07RiV]

More issues so far:

Model with layers but no geosets (i.e. just ribbons) causes a crash. Need a couple extra checks in bucket.js to prevent this.

Texture colors are flipped. Not sure if it was made with BLP textures in mind (which flip blue and red, compared to normal JPEGs), but it doesn't work when a regular PNG is loaded in via imagetexture handler. Flipping bgra to rgba in shaders (the two parts where textures are read) fixes it.

Edit: seems that was all for issues, it appears to be working fine now.

[d07RiV]

Another issue is that removeInstance doesn't work - it doesn't detach node or remove modelView. I'll submit a pull request with the things I've fixed, not sure what to do about textures though since it will break BLP loading code (color flip probably needs to be in the BLP loader).

Overall it looks amazing, but there are a few issues with some models that I have a hard time figuring out.

• ROC main menu - something's wrong with the birds
• TFT main menu - water animation glitches
• NE expansion campaign menu - what's with her legs, the leaves and water?
• NE classic campaign menu - same deal with legs, and light rays animation
• Undead expansion campaign - Arthas you ok?

Also I'm getting significant framerate drops (from 60 to 10) when the window is large enough. That's really odd, considering fragment shaders are all very simple, and the problem goes away once resolution is low enough.

[flowtsohg]

Hello. I no longer develop this library, and perhaps never will. That being said, here are a few clarifications: 1) The code is licensed as MIT :) 2) It is not Babel compliant. It was for a time, but at some point I decided it's more worthwhile to use new features such as async/await, and basically anything new that Babel doesn't support, or supports in yiffy ways (e.g. you could technically support async/await with Facebook's implementation). 3) If you don't want to load BLP/TGA textures, you don't have to. In fact you don't even need to include those handlers/parsers in your final minified file. You control the path solvers and what files are loaded and how! For example, your path solver could simply replace paths ending with ".blp" or ".tga" to ".png" or something similar. That being said I am not quite sure why you'd want to use PNGs. That's how the viewer used to work years ago before it could handle the file formats by itself. 4) The texture colors are indeed sizzled in the shader, and it's a problem I am aware of, but unfortunately never ended up fixing. The idea was to ultimately have a isSizzled member or something similar for all textures, which will automatically be true e.g. for BLP textures. The sizzling used to work on the client in the past, so this wasn't an issue, but having a simple BGR->RGB conversion (at least at the time when I was writing that particular part) lagged pretty bad on Firefox when loading many textures, so I moved to the shader. And since I never really use random non-BLP images, it was always in the back of my head, but I also didn't have any strong reason to implement it. 5) The menu models are not supported by my code as far as I am aware. I remember only looking at one of them in the past (the orc bonus campaign), but I am going to assume they are all similar. The one I checked had I believe up to 17 bones per vertex for some vertices. My code supports up to 4 bones per vertex, since it's running in a shader and it's a common number to support. Having more won't be reported as an error or cause any issues, but rather anything beyond 4 bones is ignored.

I am not quite sure about the other things you mentioned.

The code should handle pure-SFX models just fine, but it can be I regressed it at some point without noticing. I don't remember if there is a unit test that actually checks a geoset-less model.

Not sure about the framerate issue, however I do remember having similar experiences with some Chrome versions, which seem to have gone away since. Does a profile in devtools show anything interesting for you?

As to removeInstance, I honestly can't remember which one you are talking about.

If you have any further questions I don't mind answering and clarifying. I'll be happy to get pull requests if they seem good and follow the code style.

[flowtsohg]

I checked out your site, and it's pretty cool!

You may want to consider using Scene/ModelInstance.clearEmittedObjects() when you select a different animation for the currently visible model.

You could also opt to using the BLPs directly - whether they are loaded via the viewer, or whether you load them manually and use the BLP parser, at the end you get back an ImageData object which you can load into the browser. It's a bit annoying, because for some reason browsers don't know how to load Image directly from ImageData, but e.g. you can convert it to a data-url with your own code or with imageDataToDataUrl which is found in common/canvas.js. It's not the nicest, but it does allow you to keep the original files.

[d07RiV]

Hey, thanks for the reply. It's mostly working, I'll see if I can add support for more bones per vertex myself, it would be cool to get those working if that is indeed what's causing the issues.

Right now my biggest concern is that rendering is really slow in some situations. It mostly occurs when I have a game or something else running in the background, but when I view a model in a large window, FPS often dips to ~10. If I make it a bit smaller, suddenly everything is fine. JS code does not seem to be taking up much time at all, so there must be some weird issues with full rate or whatnot. I have no idea how to profile this stuff though, the only extension I found for profiling webgl was horribly outdated :(

Images are loaded via webassembly code that transforms everything into PNGs, which is used for many other parts of the site, and it was a lot easier to just plug those into the viewer instead of getting original files and using the viewer's loader (and probably faster than parsing JPEG via javascript anyway).

[flowtsohg]

Unfortunately there are indeed no real ways to debug WebGL as far as I am aware. There is the excellent WebGL Inspector addon for Chrome, however it wasn't updated in a very long time, and doesn't support instanced rendering. Perhaps you can try disabling antialiasing (you can pass WebGL options to the ModelViewer class and it will pass them along). Other than that, you could always use a smaller canvas and let it stretch via CSS.

Once upon a time the viewer actually did animations on the CPU, which is probably what the game does (after all there weren't really proper shaders or shader languages back then). This allowed for any model whatsoever to work, but was incredibly slow, considering it can render whole maps nowadays. If you'd want to do this now, you'd need some pretty major changes, I don't know if it's worth the effort for a few models that should have never been made in the way they have. I am not sure how feasible it is to do this in the shaders, what with dynamic loops not really being possible.

[flowtsohg]

I looked at the models you mentioned. There are other issues as well, not just the bones.

The texture atlasing is bad, but WebGL is really fighting me to the edge of the world in using a simple NPOT texture in the viewer (ironically it never gave me issues with my other projects). The design is also just not very good with texture atlases in general.

The archer legs and other things in that scene are...using billboarding? but shouldn't?

All in all, I just don't want to start redesigning all of this stuff and look again at all of the billboarding math, and create a software-animations rendering path again, and so on and so on. I worked on this library with the complete lack of interest of anyone in the world for too long, it's just out of me.

[d07RiV]

Ah, I feel you. This project might end up the same way, but I'll wait for reforged to see if anything good comes out of it (any idea if they'll be using the same model format?)

[flowtsohg]

Someone told me the new models are essentially the same with more textures (which affect the shaders of course), but I can't say I've seen any of the reforged files. I would assume Retera on the Hive is more updated on the subject.

[d07RiV]

Hey again, I decided to get around to doing some more stuff, adding a map viewer now. It almost works right out of the box, but there's a few issues I noticed.

Cliffs have weird colors, like so:

There seems to be no support for ramps - was it ever planned? Is there any documentation I can find anywhere that would help me implement it (though it's pretty glitchy in the game as well)?

Many maps have this large billboard for some reason, not sure what object that is, will have to investigate. Also, human towers show a bunch of different models overlapping.

No ubersplats on buildings, no shadows, water uses simple moving texture - guess no support for those either :(

Do you happen to know if HiveWE supports all/any of that? If so, I'll have to take a look at their source.

[flowtsohg]

Hello.

The cliff textures in that map are probably overridden and my code doesn't handle it properly.

HiveWE has some (full?) support of ramps I believe.

As to the billboards and human towers, it's likely that they are simply not running the correct animation. The old map viewer implementation looked for the first Stand animation, I think the new one fails to do that.

The water is as it is in the game as far as I know. If you mean that it lacks the dynamic waves, I never looked at implementing them.

Ubersplats were never actually implemented with maps in mind (their implementation is from before map rendering was a thing), and so they don't actually follow the terrain height, but they should exist as simple XY planes.

I had experimental shadows for all units/doodads, but I think I commented it out, don't quite remember.

I haven't been following HiveWE nor working on this library for a long time now, so unfortunately I can't really help you much.

[d07RiV]

I fixed cliffs - the fragment shader was truncating a float value and 1.0 randomly resulted in 0 or 1.

Working on ramps atm, just need to add the actual ramp models now. This stuff is painful -_-

Yes I meant water waves. Maybe I'll check how they're supposed to be done eventually.

I made quite a few changes in my local version, do you care enough for me to make pull requests to this repo?

[flowtsohg]

Of course, I'll be happy to get pull requests.

[d07RiV]

Made ramps, shadows, ubersplats, looking fancy now. Haven't quite figured out how to make cliff lighting not look like ass, though.

https://wc3.rivsoft.net/5db2816a7bdf4922/map

I can merge most of these changes into the original code, but some that require additional unit data (such as animProps) would be hard to fit in because they're coming from my backend, and would require parsing a bunch of Units/*.txt files.

Also, opinion on WebGL2? It seems to simplify a lot of things, replacing texture atlases with arrays and allowing mipmap support (cliffs are mipmapped which makes them look out of place when zoomed out), and I'm also getting no floating point texture support on my phone, while I'm pretty sure it works in webgl2.

[flowtsohg]

That looks great! You are impressing me, I wasn't sure if anyone could actually read my code, let alone extend and improve it.

I too couldn't quite figure how cliff shadows are calculated. My guess was something along a normal that is the mean between the terrain and the cliff faces. I guess eejin never looked further into it since then?

What do you mean by ubersplats, did you make them follow the map's terrain?

Can you elaborate on what you mean by animProps? If you need to read mapped data, the MappedData class (already used in the w3x viewer) supports loading data from both SLK files and the INIs (.txt).

I thought about upgrading to WebGL2, however support for it wasn't amazing when I checked, and ultimately it didn't actually affect too many things, pretty much just texture arrays for the couple of atlases (albeit I'll admit looking at terrain from a distance always bothered me). Not sure in what scenario you can have WebGL2 and not WebGL1 float textures, that's pretty weird.

On a side note, I checked, and the MDX shading code is in fsSimpleModel in the w3x viewer shaders. I don't remember if the hardcoded values there were based on data from the game or were arbitrary. Either way, for u_unshaded you can look at layer flags when rendering a batch.

[d07RiV]

I changed flat terrain normal calculation to only use ground level and ignore layer height, otherwise it got too dark near cliffs - haven't figured out how to make ramps light correctly with this approach though, since they're considered flat now. For cliffs I'm currently interpolating between face normal and the same normal used for regular terrain, based on fract(z), so cluff edges are smooth, but this still causes issues with ramps for the same reason.

I suspect the game might be stitching terrain meshes together, which allows it to calculate proper normals per-vertex.

For ubersplats and unit shadows I just generate a small grid for each of them in a static VBO.

Animprops seem to list tags to look for in animation sequence names, so towers can use the correct one (There's like 8 different models mashed into TowerDefenseTower).

I don't think fsSimpleModel is used anywhere, aside from the commented out doodad code.

There's also an interesting issue in particle emitters, where flat (XY) particles always use unit axes. This made water waves always turn the same way, so I made them orient using velocity XY projection, not quite sure if that's the right way to do it though.

[flowtsohg]

I am still unsure what you mean by ubersplats, since they are supported as a geometry emitter. They just never followed the terrain height, which can be added using similar shader code to the terrain rendering, and allowing them to reference the terrain texture in some way. I was mostly hesitant to add this at the time, since this project began as a simple model viewer, and is still used as one by the Hive, and in that context there is no map or terrain texture to work with.

fsSimpleModel isn't used anywhere, just mentioned it in case you want to add shadows for the generic MdxModel instances. For dynamic lights (e.g. torches), I assume the game does the normal old way of collecting the closest N (where N is likely 8) dynamic light sources. I was going to add a simple grid-based quad tree at one point, both for rendering and logic, but unfortunately never did, so it might be too slow to query light sources.

Particle emitters that don't use the ModelSpace flag should use the world coordinate system, if they don't that's a bug with my code.

[d07RiV]

That won't work, since a simple quad with terrain heights plugged in will not follow the ground if it curves in that area, it needs to use the same grid cells (still won't work with cliffs, but the game does that too). Same code works for unit shadows, selection circles, regions (those need different texturing though).

I haven't looked into dynamic lights, that sounds awful. So I'm supposed to gather light sources from model instances?

About particle emitters, they do use world coordinate system, specifically they always draw XY aligned quads (if not billboarded), instead of using emitter orientation, so no matter how I turn shoreline objects, the waves always face the same way.

[flowtsohg]

I mean using the terrain height texture in the same way it is used when rendering cliff models, by dynamically changing the heights in the shader. This means MdxModel and the particle shader need to have knowledge of the heights texture, which in turn makes the handler less generic, unless you can figure a nice way to support both having this texture and not. A possible way is to simply bind the default black texture so that every height in the shader will come out as 0. But maybe there's a nicer solution that pollutes the logic less.

About lights - that is typically how lights are done in forward-rendering. I am guessing the game stores a list of all light sources to not have to iterate over everything.

So far when testing XY emitters they were always non-directional (like circular explosions), so I never really thought about the orientation on the XY plane, it sounds like you are right, in that my code is wrong.

[d07RiV]

I checked again and indeed my phone supports webgl2 fully but only has a handful of webgl1 extensions. I'll see if I can make the viewer support both.

e: Added basic support, works in both webgl2 and 1 on desktop and actually loads on my phone, but elevation textures seem to all read as zeroes. And of course controls aren't working at all.

[flowtsohg]

Do other float textures (e.g. model animations) work?

I didn't add any touch event support to the existing client. a new client should probably use pointer events.

Just as a side note, I looked at how the game renders terrain. It splits all of the ground to many blocks, which was already known, albeit they are way smaller than I imagined they would be, and at least for the map I tested don't even seem to be square. The cliffs are merged in the same exact blocks, so they are not always physically attached. They don't actually need to be attached though, since the models already supply a normal per vertex. Instead you need to transform each of these normals in the vertex shader, so that they change when the ground angle isn't 0. I guess something like getting the height at the vertices (which already happens), and then add it to the normal's height and renormalize, or something along these lines.

[flowtsohg]

Are you ok? 😯

[flowtsohg]

This has become pretty irrelevant, and unfortunately it doesn't look like there will be activity here.