alariq / mc2

Mech Commander 2 open source engine + OpenGL Linux port
https://alariq.github.io/mc2-website
GNU General Public License v3.0
118 stars 18 forks source link

You need help with Graphics? #5

Open Methuselas opened 6 years ago

Methuselas commented 6 years ago

I used to work on Mechcommander Omnitech. I had issues, though, with the importing of animations, as I work mostly in Maya, not Max. If you can get me Bone support for animations and get rid of awful Targas, using something better for GPUs, I'll update all the graphics for you, since you're actually building an updated 64bit port for Windows/Linux. I actually have some models I built for the game, but never used. Problem is they ran about 5-6k polys which was too much for MCO to use.

Shoot me an email at methuselas@gmail.com if you'd like a graphic artist. :)

-M

alariq commented 6 years ago

Hi, Methuselas

This would be great. I also was thinkng about moving away from TGA's. You are right that high poly models kill performance in MC, because all T&L is done on CPU and in addition game is not multithreaded.

Currently, I at least want to finish all small things and put the game somewhere, so everybody can try it. And if there will be enough demand then I can think about rewriting T&L and other parts of the engine. Because it is quite a big chunk of work, although not that complicated and it will simplify code at the end. After this, bone support can be added. MC2 spports bones, I think it is a matter of correctly naming joints, etc. Have you tried first to import some existing mech animation and analyse it? They are in max format have .ase extension (i.e. anubiswalk.ase, just simple ASCII exported max file).

The problem is I do not have much free time, so we'll see how it goes. In any case, thank you for your interest! I'll keep you updated if I be starting on any of those changes.

Methuselas commented 6 years ago

You should really go talk to Magic about the MCO build. It's only 32bit, but you're already further along with the engine rebuild, you two might be able to share code. His Mech Bay is amazing, as it allows for adding or reducing tonnage, hiring mechwarriors , doing repairs, etc. He stripped the engine of all the excess UI that was unneeded. I posted a link about your build on the forums. You can find them here:

https://www.hard-light.net/forums/index.php?board=210.0

As for animations, trust me, I've picked apart the ASE files over the years. It's not using actual bones, but helpers (or locators in maya) to move the polys. It's a bit of a pita and an antiquated system. I've literally pulled a model, converted it to an FBX in Max, imported it into Maya, deleted all the polygons, added my own, renamed them all to match the original polygon names, parented them to the corresponding joint helper, imported it back into Max, exported it as an ASE and it still crashes the engine. Only thing I can think of, is it's something to do with importing to Maya, but I don't use Max. I'm going to keep messing with it, since my clan died in MWO, so what little free time I have now, is back with MWO.

Last message I read was that Magic had reached the limit of what he could do with the Engine, when I popped back in, so I started looking for a replacement engine. That's when I came across your build. I wanted to test it out, but my wife couldn't build it in VS (she's the software developer, not me. :P)

I really want to rebuild all the models and had planned to, but the limits with tgas, low poly count and no shader support had me migrating off to do other things. If it's not fun to do, you don't end up doing it. I got an itch to play MC2 again and then saw your build. A 64 bit MC2 engine would be outstanding.

alariq commented 6 years ago

I see, have not looked that much at animation code as math was not the main issue when porting the engine. I understand your concerns with animations now.

Building code in VS should be quite straightforward if one has experience with CMake (it is just a matter of setting up paths to 3rdparties like SDL correctly), but I am planning to add some instructions to README file. I am also working on automated generation of all binary resources and packs from source data, so it is easy update game resources. It is currently in "working on my machine" state, but I already can play with my own generated resources and finished the game on Linux.

Methuselas commented 6 years ago

As soon as you can get me a way to build a Windows version, let me know and I can bug test for you, at least on the Windows side. I've been itching to play vanilla MC2, but I miss the video files, so I'll probably dig out my old disc. ;) I've got a bunch of FIT files and CSV files that are more true to tabletop I've been wanting to test out.

I may just break down and watch some intro to max tutorials on Digital Tutors. When I left the Amiga scene, I toyed with Max in 2000, but the UI was so kludgy, I quit. Then, I took a couple of Maya courses and ran with Animation. Snagged me a few jobs in the game industry for a while. Now that I'm a Dad, this is more for fun. My son likes watching me play MWO, so I showed him MCO and now he wants to play it.

Methuselas commented 6 years ago

Hey Alariq,

I've managed a workaround with Max and Maya. I successfully imported a mech, in game this weekend and passed a converted ASE animation to BVH (motionbuilder) back and forth between Max, Maya and the MC2 engine. I'm going to make a rig for Max and Maya, build generic animations to replace the current ones and update the game graphics. I want to do them in batches, so if you'd like copies of them, when I finish (I'm working on vehicles as well), let me know and I'll post you archives on my google drive. I'm working in vanilla MC2, since I need the Encyclopedia to view them.

alariq commented 6 years ago

That is great. Sure I'll be glad to test them. By the way I also ported Mechlopedia (in fact this was the first thing I did : -) )

Methuselas commented 6 years ago

Don't get your hopes up just yet. It was proof of concept. The bones I converted from helper bones still show up in the engine, instead of being invisible. They are moving, however, which is what I wanted. I'm still working the kinks out.

When testing, I jump back and forth between MCO and Vanilla. I use Vanilla's Encyclopedia to to test the fit files to make sure they're working and the mission editor to check the models. MCO supports up to 10k polys now (I wasn't aware of the update) and 512k targas, which is my test bed for graphics.

BTW, concerning the Encyclopedia, are you going to remove the string.txt from the dll file? I know you can just use 7zip to remove it for editing, but an external xml or an xls file would be so much nicer to use.

alariq commented 6 years ago

you mean string.res.cpp/.h. Yes I was thinking about it. But at the beginning for me as a programmer it was much easier to write a simple script which would grab all strings and embed them right in cpp file )

I am not sure what do you mean by 7zip, because it is not packed. All string are stored in plain cpp source file. So if one would like to make some changes it is a matter of changing this file and recompiling mc2res.dll

Methuselas commented 6 years ago

I'm a former game developer (art, side, not code side - I'm a script kiddie, at best), but I like the ability for people to mod with ease. That's why I love XML files, but despise schema files. XD I've always hated the way the encyclopedia looked and functioned. They should have put drop down menus that point to external string files. I used to have a mockup I did of a new encyclopedia that would have supported drop down menus but it never got anywhere.

As for the mec2res.dll, I just always opened the library with 7zip stripped the string.txt, edited it and put it back. I can't remember if I got it to work or not. The reason I asked, was 'cos I always wanted the ability to add things to the encyclopedia on the fly, based on campaigns. All this was beyond my coding expertise, however. I work best in an enclosed environment, with a hard-coded limit on what I could and couldn't do. Neverwinter Nights was a perfect example. Their script engine allowed me to code, but I couldn't break anything too bad. XD

I was talking on the Autodesk forums and I may have found a way to hide the bones from the engine, using a material. I haven't tested it yet, though, 'cos I just found out about it last night. I was always told the engine doesn't support IK, but I know for a fact that the animations are burned in the ase/tgl files, so I really would like to build generic rigs for mechs and humanoid (vehicles are easy - it's 2 helpers, plus point helpers for weapons.), so people can animate their own mechs. There's a lot of wasted disk space from animators being lazy and not animating from default pose, so we're stuck with walk2run and other junk animations which aren't needed.

I don't know if this material thing is feasible, however. It's my understanding that the MC2 engine is hard-coded for a single texture map, which may break the engine, if bones are using a separate material, to make them invisible during renders (and I'm not even sure if the engine classifies as a render.) If it doesn't work on my side, I'll send you a replacement mech from the game, just with bones, instead of bone helpers, to see if you can change the way the engine responds to bones.

alariq commented 6 years ago

for the strings. Easy way (if you have compiler) is to just add/modify sttring in text file and recompile dll. AFAIR, indeed, each mech has only one texture, but even if there would be ability to hide bones by rendering them with "null" material, it does not sound like a good decision. In fact all this part has to be rewritten.

Methuselas commented 6 years ago

Yeah, I see your point. I'm always looking at things from a modding perspective. The more a game has an ability to mod, the more likely you are to attract new developers. A lot of people can't compile and truthfully, I never understood GitHub enough to add commits to the other game project I'm working on.

So, are you planning on re-writing all the animation routines? I mean, I can deal with the animations issue, now that I know what I was doing wrong this entire time. I even setup a VM workstation test bed, just so I could install an old copy of 3DS Max 4.2 to see if I could maximize the limitations in the animation code. If you are going to do this, I would suggest keeping legacy code, to do it the old way, until, at the very least, I get new skeletons and animations done for all the different mechs. (I believe there's only 4 or 5 different helper skeletons in the game.)

Let me know when you're ready and I'll build you a rigged mech to test in game. I do have a simple request, though, if you have the time. Can you have the engine look for "variant" mech models in the mech cvs? Basically, have a "mechname" under the variant section that points to a different model, with another weapon loadout. If it doesn't fine one, it automatically uses the default mech model.

alariq commented 6 years ago

I understand "modding viewpoint" as well. It is just that too much time needed to make everything "the right way". So for me it was a shortcut to write script to autoconvert all strings and embed them in the code. Though compiling is not that scary :) in most cases it is just a matter of one button press. Way too easier than making an anmimation.

I do not think that I'll be able to change animation code in near future. Task number one for me (after sorting out all infrastructure issues, so that anyone can easilly build code and data and run game) - is to rewrite model loading and rendering code, which is quite a big piece of work, but it will make everything look simpler and will allow to strip away a lot of code. From more high level perspective it will remove hard constrain on the amount of vertices/textures in the models and allow for more sophisticated lighting/shading as it will be performed on GPU, not to mention performance boost.

Though, if there will be no interest to the game I might as well loose motivation )

Methuselas commented 6 years ago

Don't lose motivation! :( I've already started doing new models for kit bashing. Besides, you're the closest I'll get to being able to make a setup as close to tabletop, as possible. Also, is the HBSTATZ.csv, where we would find all the stats for ranges for weapons?

I mentioned your build on the old MCO site. I was going to mention it on the MWO forums, but I wasn't sure if you wanted that many inquiries so soon. Like I said, I stumbled upon you accidentally.

alariq commented 6 years ago

I can't find any reference on HBSTATZ.csv in the code/data. Not sure if it is needed

Methuselas commented 6 years ago

I think that's for range values, as well as a few other in-game things, but I'm not sure. BTW, I hope you're still motivated, as I have this completed. I haven't UV mapped it, but it's 10k tris, which is about 10 times what we got in the original game and it's half the size of the MWO model I based it on. ;)

https://i.imgur.com/wrmNNYg.png

alariq commented 6 years ago

Nice! Though, I think MC2 will choke tryint to crunch all the vertices ) . Until I'll move T&L to GPU, this, probably, will not fly, but we can check.

Methuselas commented 6 years ago

I know Magic gave me a polycount of 3-5k in MCO, which worked, but you DID say you wanted 2017 graphics. ;) Believe it or not, this is the low poly mode, I prepped to do a high poly version for normal maps I planned on baking into the diffuse textures. I could probably drop it to 7k, by removing some of the raised edges. It's missing some of the extra polys from the original model, which makes me sad, but this is done in pieces so I can kit bash other mechs that are missing.

I'll build you a test mech with all the animations if you want to check memory usage.

Methuselas commented 6 years ago

Okay, it may be a while before I get you a working rig. I got the thing UV'ed to a single map, but once again, I'm breaking the ASE files for some reason. Even exporting a source ASE file, with no modifications is still crashing the game. It's one of the things that frustrated me with the MCO mod group, as I could never get a proper walk-through on how to export the models in ASE format for the game. I'll still keep building high poly models, though. They're just additional portfolio pieces.

alariq commented 6 years ago

you may send me you ASE example and I might be able to figure out why it crashes the game. I know at least one bug I fixed when loading ASEs

Methuselas commented 6 years ago

https://drive.google.com/open?id=0BwpB_jpXgx3Ia25ZZ2ZxRExwTTQ

Here's a link for two. One is a Catapult, that I linked myself, using a helper skeleton. The second is an Anubis that I exported from Max 7, as a Max file and imported it into 2017. I then exported it back out as an ascii file. Both crashed the Encyclopedia and the Editor, neither loading. Ironically, the helper skeletons work fine in Max.

[EDIT] - Okay, I figured out how I've been screwing up all the time. The catapult works now. However, it's missing the center torso piece in the vanilla MC2 source. On Mechcommander Omnitech, it shows the entire model. The scale is completely off, though, which is funny, considering I scaled it off the original model. I updated the catapult on my google drive to see if you wanted to check it.

Methuselas commented 6 years ago

Merry Christmas!

Here's the Flea/Fireant

https://i.imgur.com/T7nXVSJ.png

Alariq,

Do you know from the code how many LODs MC2 supports? I know, it's at least one.

alariq commented 6 years ago

The problem with your file is that it does not have normals info in it (as well as texture UVs). Loading code in MC2 is quite simple, it does not check for any of those cases and just crashes it this happens :-)

Methuselas commented 6 years ago

Yeah, it's a bit more than that. I figured out the normals issue after reading the ASCII code by hand. Also, if the helper objects aren't set to "bones" for the skeleton and the weapon helpers set to "point", it won't work properly. It really doesn't care about UV info, which is funny. I'll write up a tutorial to add mechs to the game, when I work out all the kinks with using the original animations (the Catapult test above shifts in elevation on the idle animation - I don't think they're zeroed properly.) I need to find out just how many skeletons are used by the engine, so I can redo all the animations. Making generic animations for generic skeletons is the easy part.

Methuselas commented 6 years ago

Is there any chance you could update the readme to cover compiling on Windows? The developer of Mechcommander Omnitech has taken an interest in your build. He might be contacting you about code sharing.

alariq commented 6 years ago

Sure, it is very easy. I'll add some notes

Methuselas commented 6 years ago

https://drive.google.com/open?id=0BwpB_jpXgx3Ia25ZZ2ZxRExwTTQ

Not sure if you want to set these up, but here's all the movies ripped off the original MC2 CD. If you decide to add in the support for bik files, so these will play, I might be persuaded to redo these as CGI elements so they can be fully GPL. They honestly add to the game. It's a shame that Microsoft left them out.

Methuselas commented 6 years ago

[EDIT] - Nevermind. It's some sort of ASCII corruption in Maya. Not sure exactly why, but it's resolved, by going back to an older model.

SeanLang commented 6 years ago

@alariq do you have a current version of this that runs on win 10 64bit? I can not get the source or omnitech to run.

Also what are your goals for said project?

alariq commented 6 years ago

@SeanLang yes windows version runs on win 10 64 bit. you need to build it (ther is a read me instructions) and also build data package from other repository (mc2srcdata)

@Methuselas glad you resolved your issue, wanted to look at it, but was distracted.

Methuselas commented 6 years ago

https://i.imgur.com/mrUj1vE.jpg

Didn't really get it resolved. All of the sudden, my mats no longer work. My workaround is doing all the work in maya, sending it to max, exporting as an obj and then re-importing it back into Max to do the final editing (which has to be done in Max anyway). Otherwise, I get some strange backface culling that I can't figure out. It's a PITA, but as you can see in the above image, even a simple poly cube does it. Not sure why or what happened. I even did a complete reformat thinking it was some sort of corruption issue, as Maya says there's something wrong with my files, but there are no errors in the script editor.

SeanLang commented 6 years ago

@alariq unfortunately I'm not familiar with visual studio/building projects. Any chance you could do a build for win10 64bit? If not I understand!

alariq commented 6 years ago

@SeanLang I'll try to do it next week. I've got new win PC so it will be a good experience to build everything from scratch and ensure it all works

alariq commented 6 years ago

@Methuselas maybe yoiur triangle winding order is not consistenst across a model? CCW vs. CW?

Methuselas commented 6 years ago

No, it's some sort of bug. I have a work around, which is just a single, extra step and I'm brushing up on 3ds Max to see if maybe I can figure out if it's a material problem. Like I said, if I export the model, as an obj, even the original material parameters work fine. As soon as I send it back to Maya or import it into Maya as an obj, I'm missing polygons.

This did, however, work in my favor. As I thought my original files were corrupted, I deleted them and started over, completely. I rebuilt the Locust and shaved off about 1300 tris, so it's sitting at just over 11k, instead of 12.5k. I can reduce it even more in testing, using Poly Cruncher, but not by much, if I want to keep my edges.

Methuselas commented 6 years ago

One more thing, if you want me to build a test model for materials, let me know. I can make a Locust for you, labeled as the Fireant, with normal, spec, diffuse and AO maps. I have it all UV'ed, I just haven't optimized them to maximize texture space, yet.

SeanLang commented 6 years ago

@alariq awesome, I'd love to play this again!

SeanLang commented 6 years ago

@alariq also is 2560x1440 supported? I currently use a nVidia 1080GTX.

djzort commented 6 years ago

A kickstarter campaign might be in order - to fund some dedicated hours of working through issues?

alariq commented 6 years ago

@Methuselas : currently even in my branch which uses hardware vertex transform I do not support lighting, I may hack it though. But I'd wait a bit untill I'll gather and pass all lighting info in pixel shaders. This will be a good model to test

@SeanLang: Haven't tested it on 2560x1440 (only up to full hd), but I do not see why it should not work. The only poroblem right now would be to correctly position all UI elements.

@djzort: Kickstarter campaign sound aewsome, but I am not sure that there will be enough interested people. And the game is not tested enough.

Methuselas commented 6 years ago

I'll do the Urbanmech, instead. I just started modeling it and it should be able to use the original animations, without my having to make new ones, as I've been intending. I need to do animations for the Locust and Fireant. They don't play well with the original animations. I haven't even bothered yet with the Commando, Firestarter, Wolfhound or Raven. They're all not uv'ed yet. I've been focusing on getting models done to kitbash the ones that are missing from MC2.

Yeah, I avoided the branch you've been coding, as I wasn't sure how complete it was. The build with CPU rendering took a bit, but I have it working fine now.

As for a Kickstarter, it might not be a bad idea. You've already got 4 interested parties and probably more. We're just the only ones with GitHub accounts. ;)

alariq commented 6 years ago

@Methuselas I've updated README.txt so it should be easy (ok, not hard )) to build code + data. Sorry, it, probable have taken you some time to figure out all steps to buid the game. But how did you went with data files? I am using different font file format and only recently uploaded scripts for generateng them )

Methuselas commented 6 years ago

LOL! Simple. When I borked the code, I asked my Linux guru coder friend to fix my compile. XD

alariq commented 6 years ago

I see, I hope, with updated README it will be simple enough for a person who does not have a Linux guru friend ))

Methuselas commented 6 years ago

https://i.imgur.com/7YuksWf.png

Let me know if you want me to UV this to test for T&L. I'll just need to know what textures you'll need.

alariq commented 6 years ago

Him @Methuselas I know I am not very active here, but I've got some more time lately to work on shader based lighting, first I'd like to make existing assets work and then we can test your model. Or you can prepare this one to be loaded by existing code (I mean export it and add necesary agl file, etc.), so I can test with it.

Methuselas commented 5 years ago

Him @Methuselas I know I am not very active here, but I've got some more time lately to work on shader based lighting, first I'd like to make existing assets work and then we can test your model. Or you can prepare this one to be loaded by existing code (I mean export it and add necesary agl file, etc.), so I can test with it.

So far, none of my models work well, with the stock 3DMax skeletons. I'm seeing a lot of artifacting. Best for now, to work on what you're doing, before we change all the assets.

Also, I noticed with the latest commit, we have fog in shaders now? Does this mean T&L are shader based?

alariq commented 5 years ago

Him @Methuselas I know I am not very active here, but I've got some more time lately to work on shader based lighting, first I'd like to make existing assets work and then we can test your model. Or you can prepare this one to be loaded by existing code (I mean export it and add necesary agl file, etc.), so I can test with it.

So far, none of my models work well, with the stock 3DMax skeletons. I'm seeing a lot of artifacting. Best for now, to work on what you're doing, before we change all the assets.

Also, I noticed with the latest commit, we have fog in shaders now? Does this mean T&L are shader based?

Currently everything is shader based, but only solid models ( mechs, some building etc.) are lit in shaders and drawn not as bunch of triangles pretransformed on CPU but as BV + IB transformed and lit in shaders. I am planning to get rid of all CPU transformation and lighting code, but it is spread all over the engine and requires correct handling. But eventually will simplify things.

Methuselas commented 1 year ago

Alariq, if you find the time to update, I've got all the mech animations exported from ASE to FBX. Apparently Z is up in MC2, so I'll have to fix them, since I use Y-Up.

But I did get a Mechcommander Online Battlemaster rigged to the MC2 Atlas skeleton. ;) Here's the repaired walkcycle for the Atlas, using the Battlemaster.

https://drive.google.com/file/d/1fiH24H8jkrH-rvdKy1HLRuepAGK-Ntv3/view?usp=sharing

If you can get me FBX and PNG support for models and textures, I'll start updating the original artwork. I use the MechcommanderOmnitech mod, since it allows me to not have to use PAK files and gives me 256x256 texture maps to test and I don't have a windows build of your version. I couldn't compile it and couldn't find someone I could hire that was willing to build it, who wasn't trying to gouge me. -_-

Hope all is well with you.

Methuselas commented 1 year ago

Oh and Alariq, I got all the animations and models ripped from the Battletech Game by HBS. ;) I gotta do cleanup on them all, but if you want me to make you some ASE files of a mech or two, so you can test shaders and animations, let me know.

Methuselas commented 1 year ago

Alariq,

Just an update. I've cleaned up the rigs for the default MC2 Mechs up to "R." I'm missing one: the Men Shen. There's no good model available, which means kitbashing one. Vehicles are another story. The majority don't have models, so I've downloaded STL files to remodel for MC2GL. Textures are also an issue. A lot have really bad ones, especially broken normals. I made a quick and dirty PBR material for the Atlas, at 2k, so you can test shaders, but it's going to cost you in Gfx memory, using TGAs. Another thing on the Atlas shader - I'm pretty sure I burned the normals in DX11, so when I send them to you, let me know if they don't work right, as I'll have to re-burn them to GL.

Also, these models aren't optimized for MC2, so they're pretty poly heavy - between 20k-70k for just the models, without weapons. I don't really want to reduce the polycount, TBH, as I have to do it by hand, to keep the UVs together. I do have some tools that may help, but it's not a priority right now. I'm going to install 3DMax this weekend, to export the mechs to ASE. You should be able to just dump the ASE files in the TGL folder. MC2 automatically converts ASE to tgl it finds, as soon as the game starts loading.

As for animations, Battletech came with 64 anims and all but one can be reused for MC2GL. The missing one is GetUpFront, but I can repurpose one of the falling anims, by reversing it. Now, each mech comes with an ini file. Inside that file are elements for Gestures. Can you get me the corresponding name of each gesture to the anim file? I've never been able to find them, nor do I know if the anims need to match case to the originals, due to hard coding.

I can only test these models and anims using Mechcommander Omnitech. If you could get me a Windows executable, I'd really appreciate it. I can't compile it. Coding isn't my strong suit.

Hope all is well.