Using a more recent version of UE?

[MichaelMathieu]

Hi,

I managed to compile UE4Editor, but it crashes when I launch it. The error I get is:

[2016.09.23-19.34.43:062][  0]LogGenericPlatformMisc: FPlatformMisc::RequestExit(1)
Assertion failed: Match(ShaderSource, '(') [File:/home/myrhev/dl/UnrealEngine/Engine/Source/Developer/ShaderFormatOpenGL/Private/OpenGLShaderCompiler.cpp] [Line: 1049] 

[2016.09.23-19.34.43:172][  0]LogLinux:Error: appError called: Assertion failed: Assertion failed: Match(ShaderSource, '(') [File:/home/myrhev/dl/UnrealEngine/Engine/Source/Developer/ShaderFormatOpenGL/Private/OpenGLShaderCompiler.cpp] [Line: 1049] 

The only thread I can find online about this issue is https://answers.unrealengine.com/questions/258959/ue4editor-linux-crash.html . And they suggest to update to version 4.9.

I understand people have managed to make it work with UE 4.10, but I can't find how. That would be awesome if someone who managed to use UE4.10 could write a quick howto, or just explain how they did it. Also, the pastie link in #19 is dead, which doesn't help.

Thank you!

[adamlerer]

@MichaelMathieu I will take a look towards the end of this week and see if I can get UETorch upgraded to a more recent version (hopefully 4.12 or 4.13).

In the meantime, I think @willwhitney may have encountered a similar problem. Will do you remember if you saw this problem? Did it have to do with Linux version or NVIDIA vs. Nouveau driver or something?

[willwhitney]

I'm not sure if this is the same issue, but you should check to make sure your machine is using the Nvidia driver instead of the open-source one. There's an Ubuntu UI for picking your drivers.

[MichaelMathieu]

Thank you for your answers! @adamlerer, it would help me a lot if you manage to upgrade it, I can confirm that the release branch (4.13, I think) compiles and run well.

@willwhitney , as far as I can check (from X logs, in particular), I'm using the nvidia driver correctly. I can run cuda code all right, too, so I don't think there is a problem there. Please tell me if you have any other idea.

[marioyc]

@MichaelMathieu I think I got this error when I was trying to build UE 4.8 in more recent versions of Ubuntu, as it was mentioned in the first answer of that thread.

[thesilencelies]

Got the same error. How's progress going with getting it to work on 4.12 etc? If I have to hack it through myself, any pointers on where to get started?

thanks

[thesilencelies]

ok, got it to work, sort of. I changed Setup.sh in UEtorch to use 4.10 instead of 4.8. I downloaded and applied the patch in http://pastie.org/10885471 . Then I had to modify the buid.sc for both ScriptPlugin/Source/ScriptPlugin and ScriptGeneratorPlugin/Source/ScriptGeneratorPlugin so that they were looking in the right folder (had to remove caps) and for liblua.so not Lua.lib. Then I also had to remove the lines blacklisting linux from ScriptPlugin/ScriptPlugin.uplugin

So now it compiles, and lets me attach things fine. However, the scripts fail to do anything, which seems to be because when it runs 'uetorch.lua' it warns "error loading module 'ffi' from file : undefined symbol: lua_gettop". Any ideas on how to fix this?

thanks

[marioyc]

Could be that you are not using LuaJIT.

[thesilencelies]

I thought that could be a fix, but elsewhere you seem to insist that we use Lua-5.2 I'll try reinstalling torch with luaJIT and recompiling, and see what else catches fire.

[thesilencelies]

With some tweaks (see below) it does compile, though now it says the same symbol is undefined in "libpaths.so". The fix may be easy, I'll have a go at it tomorrow. EDIT: still unable to fix it, and the same issue turns up when I delegate to lua code in my own C++, though only from UE4. I think it's a bug in what they have, so I'll chase it up with them.

additional changes required were - in the build files changing the library to libluajit.so. I also added the directory that has ffi.so in it to the includes, but I don't think that did anything. I also had to do some compatability work on ScriptPlugin/Private/LuaIntegration.cpp, which had a couple of 5.2 only methods (lua_setfuncs and lua_pushglobaltable, which I replaced with lua_register and lua_gettable and appropriate parameters. I fully expect this to break later, but it'll do as a sticking plaster) Additionally, lua_newstate had to be replaced with luaL_newstate cause luajit distinguishes between 32 and 64 bit like that.

[thesilencelies]

so the fix I've come up with involved creating a new module that loaded the dynamic modules in advance so that it actually had their members in memory (the problem was cross dependencies between the modules...) I'm sure something similar/better could be implemented directly here. What I made is based on https://github.com/ue4plugins/VlcMedia/blob/master/Source/VlcMedia/Private/Vlc/Vlc.cpp

[reppolice]

I seem to have the same Assertion Failed, after improvising a bit with these workarounds https://helleniq.blogspot.dk/2016/10/uetorch-and-moving-target-that-is.html

[adamlerer]

I am taking a look today at upgrading UETorch to a new UE version. @MichaelMathieu if you've got some time to look at it with me let me know.

[adamlerer]

Upgraded to the latest UE (4.13) in https://github.com/facebook/UETorch/pull/32.

[adamlerer]

Note about the upgrade: I was unable to get ScriptGeneratorPlugin working with UE 4.13; I ended up just removing the dependency because I was barely using it and I could write C bindings for the tiny bit I used.

ScriptGeneratorPlugin might be useful for some people (it uses the UE reflection API to wrap a whole bunch of UE objects that you can then access from Lua), but I didn't find it very useful in my work. It was easier just to write FFI bindings for the stuff I needed. If anyone has ideas or wants to try to get ScriptGeneratorPlugin building again, ping me and I can explain the build problems. It might just require posting on the UE message boards and getting a bit of help from the developers. cc @thesilencelies

[reppolice]

Not to be an ungrateful git, but getting UE and EUTorch from the repositories afresh gives a build that breaks immediately with a Debug complaint https://gist.github.com/reppolice/13d594f4afe17f0f94bbf42e1ddea53b

[adamlerer]

Sorry, I must have left in a stray Assert. I will post a fix in the next couple hours.

[adamlerer]

Thanks again @reppolice . Fixed in https://github.com/facebook/UETorch/pull/35. Last time I was too lazy, but I now did a full reinstall from scratch and checked that it worked. If you still have problems on your system, feel free to reopen this issue (and report back either way!)

[reppolice]

I didn't get the luxury of a crashing executable this time. Not sure how to troubleshoot this one Meanwhile, you may want to edit the top-level README to include the new UE version.

[951/955] Link libUE4Editor-Matinee.so ERROR: UBT ERROR: Failed to produce item: /UnrealEngine/Engine/Plugins/ScriptPlugin/Binaries/Linux/libUE4Editor-ScriptPlugin.so Total build time: 3356.54 seconds Makefile:183: recipe for target 'UE4Editor' failed make: *\ [UE4Editor] Error 5

[adamlerer]

@reppolice there should be some more info further up in the build log. The initial build log will be very long, try running make again, and it will just rebuild the failing item and give you a clear error log. Could you post that here?

[reppolice]

I cannot post the exact message right now, but it looked like a previous problem of lua vs luajit, with some Lua errors. I saw that one was fixed, could it be back from the dead?

[adamlerer]

I'm not sure which problem you're talking about. Lets wait until you can post the error message.

[reppolice]

Yes, it was seen before:

bash "/UnrealEngine/Engine/Build/BatchFiles/Linux/Build.sh" UE4Editor Linux Development
Building UE4Editor... Using clang version '3.8.0' (string), 3 (major), 8 (minor), 0 (patch) Performing 5 actions (8 in parallel) [1/5] Compile Module.ScriptPlugin.cpp In file included from /UnrealEngine/Engine/Plugins/ScriptPlugin/Intermediate/Build/Linux/B4D820EA/UE4Editor/Development/ScriptPlugin/Module.ScriptPlugin.cpp:3: /UnrealEngine/Engine/Plugins/ScriptPlugin/Source/ScriptPlugin/Private/LuaIntegration.cpp:20:2: error: use of undeclared identifier 'luaL_setfuncs' luaL_setfuncs(LuaState, Lib, 0); ^ /UnrealEngine/Engine/Plugins/ScriptPlugin/Source/ScriptPlugin/Private/LuaIntegration.cpp:545:2: error: use of undeclared identifier 'luaL_setfuncs' luaL_setfuncs(LuaState, PrintOverride, 0); ^ /UnrealEngine/Engine/Plugins/ScriptPlugin/Source/ScriptPlugin/Private/LuaIntegration.cpp:902:2: error: use of undeclared identifier 'lua_pushglobaltable' lua_pushglobaltable(LuaState);

[adamlerer]

@reppolice do you have a pointer to the original issue? I thought the reason for the problem before was that you were using the wrong version of Lua. Missing luaL_setfuncs usually means you're using 5.1 or luajit instead of 5.2.

I added a check in Setup.sh that should check that you're using Lua 5.2 (https://github.com/facebook/UETorch/blob/master/Setup.sh#L31) , but I didn't test it. Can you run lua -v and see what it says?

[reppolice]

I got Lua 5.2.3 . This is my "torch lua" as per the build instructions. The interesting part would have been that I compiled the previous version with the same Lua settings, and that one certainly produced an executable. I do not seem to have any other lua, although I've just installed the Ubuntu lua5.3 just in case anything changed like include files etc, but nothing did. The "torch lua" did not install global includes and such, nor does Ubuntu provide a lua5.3-dev but instead it is a liblua5.3.0-dev, so I am now trying to see if I can pass https://www.lua.org/tests/ with some combination of manual fixes, of which tests I can only pass the basic ones, while "all" fails quite quickly, at strings.lua:99 . I don't know if passing the complete lua test is an issue or not for our build here. Also, what if the "secret" lies in your Torch build, which you may not have touched in a long time, while I did it 2 weeks ago.

[reppolice]

Well, I cannot pass lua5.2 tests on my mac either, so that is not where the problem lies.

[adamlerer]

@reppolice I will try it myself with a clean Torch/Lua52 install. In the meantime, can you tell me the output of the following commands for you?

which lua
$(which lua) -v

Thanks!

[reppolice]

which lua /home/ue/torch/install/bin/lua

$(which lua) -v Lua 5.2.3 Copyright (C) 1994-2013 Lua.org, PUC-Rio

[adamlerer]

If I look in UnrealEngine/Engine/Plugins/ScriptPlugin/Source/Lua/install/include/lauxlib.h, luaL_setfuncs is declared there, and that file is included in LuaIntegration.h. Can you check that luaL_setfuncs is defined in your lauxlib.h? If it is, then there must be another lauxlib.h somewhere else in your system that somehow is getting included... maybe see if you can run a find to find it.

[reppolice]

I am trying to build now on a new but slow VM, it is taking a very long time and I even think there is a temporary github problem. But isn't the "solution" to specify -I includes in the build script? BTW, I thought we were going to pick up the lua under torch, my bad!

[adamlerer]

@reppolice I don't know what you mean "I thought we were going to pick up the lua under torch". Your output from which lua and $(which lua) -v look correct to me, so I wanted you to figure out where UE is picking up lauxlib.h from and make sure luaL_setfuncs is defined there.

Regarding -I, you are correct, if you look in Engine/Plugins/ScriptPlugin/Source/ScriptPlugin/ScriptPlugin.Build.cs you will see I add Engine/Plugins/ScriptPlugin/Lua/install/include to the include path, and that folder is a symlink from the Torch install. So ScriptPlugin should have the right -I. I'm not sure how you can get make to print out the compilation commands it's running though.

I am also having a problem checking out Torch from one of my machines for the last few days :(

[reppolice]

Well, my new VM was not meant to be, some Mono error, so tomorrow back to the Lua troubleshooting

[47/955] Compile Module.UnrealEd.17_of_20.cpp

• Assertion at mini-exceptions.c:1845, condition `tls->signal_stack' not met

Stacktrace:

Native stacktrace:

    mono() [0x49ff2f]
    /lib/x86_64-linux-gnu/libpthread.so.0(+0x113e0) [0x7f8cc61063e0]
    /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38) [0x7f8cc5d61428]
    /lib/x86_64-linux-gnu/libc.so.6(abort+0x16a) [0x7f8cc5d6302a]
    mono() [0x630409]
    mono() [0x63069c]
    mono() [0x630833]
    mono() [0x49fb79]
    mono() [0x423c18]
    mono() [0x423cdd]
    mono() [0x587aa9]
    mono() [0x629a8c]
    /lib/x86_64-linux-gnu/libpthread.so.0(+0x770a) [0x7f8cc60fc70a]
    /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f8cc5e3282d]

[adamlerer]

@reppolice the best thing you can do is figure out the root issue instead of trying different workarounds. The error says "undeclared luaL_setfuncs" so please just check Engine/Plugins/ScriptPlugin/Lua/install/include/lauxlib.h to see if it has luaL_setfuncs declared in there.

If it is declared in there, then make sure that lauxlib.h is actually being included... e.g. insert some invalid code into that file and make sure that compilation fails inside that file.

[reppolice]

Sure, I wanted to experiment a bit, since I might need to run this on machines with various characteristics, with and without Nvidia hardware. Speaking of which, is it too hard to dockerize this, ideally in nvidia-docker too?

[adamlerer]

I've done it. It's about equally painful as everything else about Unreal Engine :/

On Wed, Nov 16, 2016 at 1:29 PM, reppolice notifications@github.com wrote:

Sure, I wanted to experiment a bit, since I might need to run this on machines with various characteristics, with and without Nvidia hardware. Speaking of which, is it too hard to dockerize this, ideally in nvidia-docker too?

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/facebook/UETorch/issues/29#issuecomment-261030089, or mute the thread https://github.com/notifications/unsubscribe-auth/AFcCDSgSL_sGLn9OH4ZYscawS5QLYXxfks5q-0uUgaJpZM4KFU2F .

[adamlerer]

Okay, I did a fresh install of Torch and everything still works fine for me. You'll need to check lauxlib.h

[reppolice]

Things are becoming clearer :)

ls -l Engine/Plugins/ScriptPlugin/Lua ls: cannot access 'Engine/Plugins/ScriptPlugin/Lua': No such file or directory

Where is that Lua content coming from?

[adamlerer]

Engine/Plugins/UETorch/Setup.sh sets it up. Did that script succeed when you ran it?

[reppolice]

It certainly ran for half a day :) I will rerun

[adamlerer]

Haha, no that's the UE setup that takes a long time. Before that, you need to run Engine/Plugins/UETorch/Setup.sh, which should be very fast. If you type git branch -v in the UnrealEngine directory you should see a 4.13-UETorch branch in there.

On Thu, Nov 17, 2016 at 5:31 PM, reppolice notifications@github.com wrote:

It certainly ran for half a day :) I will rerun

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/facebook/UETorch/issues/29#issuecomment-261390885, or mute the thread https://github.com/notifications/unsubscribe-auth/AFcCDbAW4EMrVIG56rE5_pxAbzc7N2C9ks5q_NXLgaJpZM4KFU2F .

[reppolice]

Well, the only thing which looks a bit error-y, in that it is verbose, is the git, but it finishes with a puzzling message "picking up" the lua from Torch, still no Lua under Engine:

Your name and email address were configured automatically based on your username and hostname. Please check that they are accurate. You can suppress this message by setting them explicitly. Run the following command and follow the instructions in your editor to edit your configuration file:

git config --global --edit

After doing this, you may fix the identity used for this commit with:

git commit --amend --reset-author

7 files changed, 36 insertions(+), 58 deletions(-) === Setting up Lua... === LUA= /home/ue/distro/install/bin/lua

[reppolice]

I was going to look into how and why that Lua directory never appears on my machines, but instead symlinked the "real" lua directory, in my case /home/ue/distro/install/ , to Engine/Plugins/ScriptPlugin/Lua and voila, the build completes. Considering that your installation instructtions specify the lua you want under torch, Setup.sh could just as well use it, until a date far off into the future when torch changes so much that it breaks this method.

[adamlerer]

Hey @reppolice, I'm happy you got it working. I miswrote the directory, it was supposed to be Engine/Plugins/ScriptPlugin/Source/Lua, Setup.sh should symlink your Lua install into there. Is that missing for you? Or was it really the case that you needed to create Engine/Plugins/ScriptPlugin/Lua?

[adamlerer]

Let me clarify:

I made a mistake in this thread, telling you to check Engine/Plugins/ScriptPlugin/Lua. The location where Lua gets installed by UETorch/Setup.sh, and where ScriptPlugin looks for it, is Engine/Plugins/ScriptPlugin/Source/Lua. I don't see how creating a folder in the wrong place would have made it work.

[reppolice]

Oh the twists! Well, My Source... directories were there and pointed to the same Torch installation, I will redo the whole thing just to see how lucky I can get, but it is a weird indication of perhaps a similar mistake in the Setup.sh

[reppolice]

So, starting the UE build from scratch but without redoing my Torch, it now works. It is very hard to say what changed from the failed builds of previous days, but right now the tooling seems to be stable enough to start a project in the editor. Cheers!