Compiling

[BeaconDev]

How do we go about creating a working executable or set of executables from the source? I'd be intrigued to try out the supposed new grass shadows, and then start looking at making changes of my own.

[nitrocaster]

First of all, I would like to use my CS repository style here (docs, version plan, coding conventions). Then apply changes I've already done (+ your changes, if there are ones). Note that we should always have good master branch (though my repo wasn't actually good all the time). 'Good' means that:

• it can be compiled without errors
• it runs without critical errors (at least doesn't add new ones)
• it is compliant with the coding conventions we use

To all contributors: Please, fill in your email addresses so we can contact each other personally.

[BeaconDev]

Fine by me, go ahead and set that up if you like. If it looks good and works efficiently I don't see why we can't use your style.

Completely agree with a stable master branch.

You can contact me on CoPRedux@googlemail.com, or probably better, if you have Steam, at http://steamcommunity.com/id/beacon

[ghost]

In terms of compiling, I have not tried compiling yet, I just wanted to port over my changes from CS first which I know will work fine with COP.

I completely agree. I think we need to agree on which COP repository to use first. There are two that come to mind: 1) The official 1.6.02 leak with fixes for VS2008 (that's the one uploaded here) 2) https://bitbucket.org/redpython/xray-cop/branch/1.6.02

I also think we really should see if we can use VS2013 instead of 2008. Nitrocaster, I know you managed to do that with CS, and I have no idea how to do it at all. Would that be something you'd be interested in doing? You could pick what you want as the main branch then update it to 2013, then we can all work together on changes that follow the same style you use.

My email is matthew.w.swartz@gmail.com

I think what I'm going to do until we get all that done is port over all the changes I did for CS (such as the grass shadows by K.D.), into a separate branch and have only the files changed in there, which would allow for easy merging for features people want or don't want.

[BeaconDev]

What are the differences on the two repos you listed?

[nitrocaster]

For emails: I meant you profile settings

For compiling: It is bad idea to commit code that you didn't try to compile. Things are not supposed to work this way, we need 'good' master branch. When I found CS sources, I spent about two days to make sure it can be compiled.

For reference code: I suggest we use original sources for VS2008. Reason: no new bugs, only old well known ones.

For using VS2013: There's no troubles to use VS2013 - just convert solution and projest files. There may be some problems with virtual directories in projects - that have to be fixed manually (awful routine, but it happens).

[ghost]

Works for me. Just filled in my email. I agree, I shouldn't have suggested on making changes when I have yet to even compile it.

I may not have enough time today (have to work in a few hours), but I'll try to convert the project to VS2013 today then compile it. If it works I'll update things and post instructions the same way you did at your CS repo.

Also, for some reason I had two upload the code in two parts using the Github utility as it would crash after around 7,500 files. Is there an alternative way of committing to Github?

[nitrocaster]

So, I'm going to dig into these sources, and once I get them all compiled I'll push them. Meanwhile I suggest you to learn what's new in xray 1.6 comparing to 1.5 and schedule your tasks in the version plan.

[nitrocaster]

@Swartz27, I understood from your previous post that you ask is I would get COP sources compiled under VS2013, so I'm ok with that. Are you going to do it by yourself?

[ghost]

I was going to but if you're willing to do it I'll let you, then you can push it, and we can use what you've done as a base.

I realized I have very little time today, I'll probably jump back on in roughly 7 hours and see if anyone has had any luck. I'll go over the proposed features list again and tidy it up after I've compared COP to CS source and then put them in order.

[nitrocaster]

Also, for some reason I had two upload the code in two parts using the Github utility as it would crash after around 7,500 files. Is there an alternative way of committing to Github?

Actually, I don't use Github utility for committing. I prefer git add + git commit + git push. If you're interested I use WinCmp (Compare it!) as diff/merge tool. I have simple batch file that allows me to do these things:

• gdiff : run difftool on unstaged changes
• gdiff -c : run difftool on staged changes

[LoNer1]

Good to be hearing ablout the structure and how things are going to unpack from here on.

For my email, its "loner_dude@hotmail.com" for anything modding related. Looking forward to our cooperation, gents!

Date: Thu, 18 Sep 2014 09:07:49 -0700 From: notifications@github.com To: cop@noreply.github.com Subject: [cop] Compiling (#3)

How do we go about creating a working executable or set of executables from the source? I'd be intrigued to try out the supposed new grass shadows, and then start looking at making changes of my own.

— Reply to this email directly or view it on GitHub. =

[nitrocaster]

To @Swartz27 : What are 'cs' and 'patch-1' branches for?

[ghost]

I don't remember creating either branch, though it looks like someone removed "patch-1". The CS branch could have been where I had forked your CS repo but it can't be because there's no CS files there. I'll remove it.

[ghost]

Bangalore, we're waiting on Nitrocaster to update the source here to Visual Studio 2013. I recommend using that once it's out rather than messing with these other revisions.

[ghost]

I suggest to the feature list to implement luajit 2.0.3, for the replace of luajit 1.1.4.: http://luajit.org/download.html

Luajit above version 2.0.0 compiles lua scripts much faster, it could remove microstutter, you can compare here the performance difference: http://luajit.org/performance_x86.html

[BeaconDev]

100% yes

also, nice to see you here Bangalore!

[ghost]

I agree completely, I simply hadn't added the luajit version number in the proposed engine changes, but I had meant the version you speak of.

[nitrocaster]

To @Bangalore1010 : r8384 is kinda experimental version.

[ghost]

@Bangalore1010 : Did you use the vs2008 version from the Github here or did you download it from the post on gameru? Just wondering because I encountered a bug trying to compile the source here, but I'm pretty sure it's the same version. Either way, downloaded it again from gameru. I think the reason I couldn't get it to compile was I don't have SP1 installed for VS2008 :P

[tsf00181]

Do you have a link for the CS source code?

Google translate is playing hell with me on the gameru.net site so I'm having trouble finding it.

On 9/20/14, Bangalore1010 notifications@github.com wrote:

I used this pack with Visual Studio 2008 professional + SP1 + both June 2010 DirectX SDK + include and lib folders from November 2007 SDK (but alone the March 2009 SDK should work fine too) + dev SDK from the stasvn.7z pack: http://s.gameru.net/stalker/STK-HLAM/engine.vc2008.fixed.rar

For clarity's sake, here is my complete engine+sdk folder upload: http://www.mediafire.com/download/2hzqxc2ozvekdsk/COP_src_v1602.7z

---

Reply to this email directly or view it on GitHub: https://github.com/Swartz27/cop/issues/3#issuecomment-56267825

[ghost]

@tsf00181 : http://www.gameru.net/forum/index.php?s=297823ebaced19091f6c36b012189218&showtopic=55777&st=1700&p=1489128&#entry1489128

@Bangalore1010 : Did you encounter an error when compiling zlib that mentioned "_vsnprintf" being redefined? Just wondering, because I fixed it by going into zutil.h and commenting out lines 197 to 202. I'm compiling xrgame (again) because I forgot to include the DirectX includes and libraries with it :p I'm expecting this will work fine though as it's the last file to compile and they've all compiled fine so far.

I'll post an update later as I have to go.

[andrew-boyarshin]

I'd like to work on LuaJIT and LuaBind, when CoP sources will become compilable.

[nitrocaster]

I'd like to suggest to rename this repo to "xray-16", if you don't mind.

[ghost]

I don't mind at all. In fact, nitrocaster, is it possible to transfer ownership of a repo? You have a good system with certain coding conventions, you're the top programmer here, etc. I think you should be in charge with us contributing.

Also, unless you've gotten any farther on compiling it in VS2013, I think maybe we're good with leaving it at VS2008. I downloaded and compiled what Bangalore posted to be extra careful, beside the one modification I had to make which I mentioned, it worked fine (well, the R4 renderer isn't working at the moment. It's telling me my video card doesn't support it. So I need to figure out what's causing that which is probably me using a newer DirectX sdk than the one we're supposed to use).

I'm actually going to delete the source code currently up and re-upload with the structure Bangalore is using once I can verify R4 works correctly. It's cleaner to post it that way. Then I think I'm going to grab the coding convention and commit convention documents you have and post them here as well if you don't mind.

@STALKER2010 : Please do work on those, that would be great. With LUAJit are you going to be using the newest version, as that is what we would like to use.It means re-doing the vanilla LUA scripts too in order to work with the new version though.

[nitrocaster]

is it possible to transfer ownership of a repo?

AFAIK, it's not necessary since I'm a contributor. Anyway, you can do it via support.

For migrating to VS2013 - work in progress, just wait a little.

For R4 renderer: do you have DX11-compatible GPU? If so, that is a problem in shader that I've already fixed (in my xray 1.5 repo).

For this:

I'm actually going to delete the source code currently up and re-upload with the structure Bangalore is using

You uploaded the same code Bangalore use (vs2008.fixed). The only difference is that you missed SDK stuff. I'll take care of it, keep calm and don't delete anything :)

[nitrocaster]

Wow, it's not that I thought. Try to investigate.

[ghost]

My guess is that the SDK isn't up to date for DX11. I'm going to try a later revision of the DirectX SDK and see if that solves it. If that doesn't help I'll compile everything under debug.

Edit The SDK is the last one before it becomes Windows SDK for Win8 which the GSC people obviously didn't use. Time to debug.

[ghost]

Looks like it doesn't want to be debugged: http://pastebin.com/Pr83gBWe

Just noticed something else. Every time it crashes with that error it creates the precompiled shader for the r4 dumb.ps in your gamedata folder. WTF is that all about?

[nitrocaster]

The xrGame.dll is a hell a lot of time to compile.

That's why we should split it to multiple modules (maybe DLL + static libraries).

[nitrocaster]

I've just noticed _GPA_ENABLED define. Maybe it's redundant (since VS2013 has really good DirectX debugging tools). Anyway, I gonna move it from source code to project additional defines.

[nitrocaster]

Just noticed something else. Every time it crashes with that error it creates the precompiled shader for the r4 dumb.ps in your gamedata folder. WTF is that all about?

COP caches compiled shaders in $app_data_root$/shaders_cache. gamedata directory should be read-only.

[nitrocaster]

This have to be debugged (see d3dcompiler output).

[nitrocaster]

This is bad solution - you don't know what causes that problem so you wont be able to fix it when it breaks again. We'll need to get this working with new DX SDK.

I suppose this problem is quite simple to debug, try to fix it by yourself.

[ghost]

I can't fix it, i'm not a programmer, sorry. I'm just a beginner with enquiry, so maybe it would be better to remove me from the collaborators.

[ghost]

@Bangalore1010 : You're wanted here, you may not be a programmer but I added you as a collaborator so you could make engine suggestions. It looks like Alundaio might be back too so maybe you can convince him to do the same? I don't think he wants to do any programming, but maybe he'd do that.

As for the shader problem and the latest DXSDK, it is a problem as when DirectX 12 comes out it would be cool to add it to the source, and we won't be able to if we can't use the newest SDK that now comes with the Windows SDK. I have an idea on what the problem might be (has to do with the shaders, not the code) so let me try it and I'll report back.

[nitrocaster]

Little update. I'm trying to remove all original 3rd party code from the repository (from SDK dir) to reduce its size. Already done with:

• libogg
• libtheora
• libvorbis
• FreeMagic

[ghost]

That's great news :)

I'm going to try and reorganize things on here tonight and tomorrow depending on available time.

I wasn't able to solve the problem through the shaders like I thought I could. The big problem is that they use precompiled shaders for everything, so there's no knowing what settings were used when they compiled them (for instance the DX11 renderer uses R1 lights for campfires for some reason unknown). I did compare files between the two DirectX sdk's (Aug 2009 vs Jun 2010) and noticed that in DX11 a lot of data was moved from a few files to a few others, though it remains the same data. It probably is a code thing and I'll try and debug it tomorrow.

In the meantime, Bangalore was right that Aug 2009 will compile R4 just fine, but obviously that's a temporary fix if we want to implement DX12 way in the future, but I also don't consider it the end of the world if we don't get DX12.

I made two small changes to a seperate direcory I have just for screwing around with the game code. It's a good thing I didn't commit those grass shadows and actor shadows, as the format (for actor shadows anyway) is now different. I fixed it and both features look great in R2-R4, but it's a good thing I tested them.

One thing I think is really important to do is get rid of these goddamn pre-compiled shaders, they're a pain in my ass. I asked team Dezowave what they thought of using them, and they said while developing Lost Alpha they tried adopting COP's precompiled shader system, but when they tested it vs. non-compiled shaders they noticed very little difference in performance, so we wouldn't be losing anything and R4 might look a bit better.

[nitrocaster]

One thing I think is really important to do is get rid of these goddamn pre-compiled shaders, they're a pain in my ass.

Precompiled shaders save loading time. What exact problem do you have?

[ghost]

It may save loading time, but it makes it difficult for people to add new shaders as if the precompiled shaders have data that conflicts it will override it (and since we don't know what settings they used for their precompiled shaders, we can't compile our own. A good example of this is objects/r3/accum_sun_near.ps/ . It has a variety of files in there, all with different file sizes).

On top of that, R4 has had a long standing issue where it uses R1 lighting around things like campfires. I imagine that would be in the precompiled shaders, not the source code.

You're welcome to keep precompiled shaders, but I think I'll remove that feature on my end (and won't commit it here). It's just an experiment to see if it helps with R4 for instance.

[BeaconDev]

I'd definitely chime in and put in some support for non-precompiled shaders, the campfire issue is a real pain in my ass.

[nitrocaster]

This should be investigated and fixed, not thrown away. We'll face lots of issues that have to be fixed too (but cannot be thrown away like this).

[andrew-boyarshin]

@nitrocaster when will CoP sources will be adapted for VS2013? I use it, and I don't want to install VS2008. Please, make it compilable under VS2013 and without virtual drives magic. Thanks.

[nitrocaster]

WIP. Currently I'm fixing project dependencies and include/library paths. Final result is going to be better than we have in CS repo.

[fagoatse]

@Swartz27 The renderers in CoP were gimped in comparison to CS so that the game could be run on older hardware. The guy who wrote the dx11 backed(@sopyer) in actually quite active @ github so you might want to ask him for details. Shader caching definitely improves load times but performance difference is probably negligible. Nvidia does shader caching by default on the driver level for dx10+ games since their 337.50 drivers.

[nitrocaster]

Nvidia does shader caching by default on the driver level for dx10+ games since their 337.50 drivers.

Well, in that case, it actually becomes redundant.

[ghost]

@fagoatse : That's good to know. Even more of a reason to get rid of precompiled shaders (don't most people use SSD's at this point anyway? I imagine the difference is negligible, but I plan on comparing it when I do remove precompiled shaders).

I'd ask sopyer, and thanks for the suggestion, but wouldn't he be mad that the COP code got leaked?

@nitrocaster : I heard from K.D. as I'm having trouble with the render difference of the grass shadows implementation I ported. Anyway, in terms of working with us and also OGSE features, he had this to say: "Unfortunately, I stuck with VS2010 now. May be i should wait a little before start to work with your repo, need just fugure out, how to work with git. When OGSE will be released there will be available source code of all features. And i'll help to work with it." So it sounds like at some point we'll have K.D. onboard, which would be epic :D

[nitrocaster]

don't most people use SSD's at this point anyway?

Precompiled shaders make loading faster because compilation process is slow, essentially if you have tons of various shaders. There're several examples in nvidia d3d sdk that start up pretty long just because of compiling shaders.

[nitrocaster]

I'd ask sopyer, and thanks for the suggestion, but wouldn't he be mad that the COP code got leaked?

I don't think so, he'd rather be ok with that :)

[ghost]

I'll message him then :)

Update on precompiled shaders: I successfully disabled them for the R4 renderer and tested it out. Although I did not measure loading times, they seemed to be identical to when precompiled shaders were used. However the campfire light and shadow issue was not fixed by removing precompiled shaders. I did a winmerge compare of the dx10 vs dx11 renderers, and oddly almost everything is the same, except that DX11 has a couple of tessellation references.

Very odd. Something else is causing the R1 campfire lighting issue. I'll see if sopyer knows.

Also, not strictly related to code (more to shaders) but tess_hm doesn't even work. tess_pn does of course, although the effect is weak. The height map tessellation though looks like it was not ever implemented correctly which is probably why it was never used.

[nitrocaster]

The height map tessellation though looks like it was not ever implemented correctly which is probably why it was never used.

We'll add it to the version plan.

I have a little update: all projects are converted & fixed, and now I'm in the middle of compiling them. LuaJIT and LuaBind have just been compiled.

This is going to be a single huge blob commit (which is considered evil), but there's no any other way.