Mudlet dev file location

[SlySven]

My last pull request might cause problems if the lua files are not in a subdirectory of the src directory alongside the executable. This branch fixes that by doing a user guided search any time the main lua loader for the mudlet lua files can't be found. As a side effect of the work with the lua package.path and the lfs current directory stuff Mudlet can now handle the lua files being anywhere within the accessible file-system - so shadow builds are definitely working now!

Also, because of the need/ability to put a sensible default in for where to start looking for the lua files it is quite easy to provide installation commands in the project file to put the files in that place. So, on *nix, Mudlet is now "deployable" i.e. "make install" inside Qt Creator is now a workable proposition - provided a non-root user has sudo & install commands and a /etc/sudo.conf with a "Path askpass = ..." line or an environmental variable SUDO_ASKPASS pointing to something like /usr/bin/ssh-askpass so they can enter the relevant password. A conventional "make install" directly in the build directory should also work as well.

The path to the main lua file is stored in a human readable format in the "Mudlet 1.0.conf" file accessed via QSettings . The key is "systemLuaFilesPath" which can be deleted to prompt the search again the next time a profile is started (or reconnected).

Unfortunately there was an obsolete "all-in-one" LuaGlobal.lua file in the root of the source directory; to avoid confusion this commit deletes it and because of the significant size difference this code explains to the user in the details why the file they need to find has to be less than 10K (it is 4-5K presently) and not 60K bytes in size.

It might be useful if we could merge the mac_src.pro contents into the main src.pro - this commit does not touch the former file but the changes should still be compatible with all operating systems.

[vadi2]

Let me review this one before merging.

[Chris7]

Can you resubmit with file mode changes ignored?

sent from my phone On Mar 21, 2014 12:53 AM, "Vadim Peretokin" notifications@github.com wrote:

Let me review this one before merging.

Reply to this email directly or view it on GitHubhttps://github.com/Chris7/Mudlet2/pull/31#issuecomment-38249123 .

[SlySven]

Um, those source files are NOT executables so they should have the execute bits set which I realise is not a problem for Windoze but a niggle on *nix systems. Or is it the background noise. Not trying to be picky - just want to understand rationale... 8-)

On 21/03/14 12:15, Chris Mitchell wrote:

Can you resubmit with file mode changes ignored?

sent from my phone On Mar 21, 2014 12:53 AM, "Vadim Peretokin" notifications@github.com wrote:

Let me review this one before merging.

Reply to this email directly or view it on GitHubhttps://github.com/Chris7/Mudlet2/pull/31#issuecomment-38249123 .

— Reply to this email directly or view it on GitHub https://github.com/Chris7/Mudlet2/pull/31#issuecomment-38270528.

[vadi2]

Yeah, the file mode changes are fine. Although they should not be bundled up together with a commit that does something entirely different but split up into two.

[SlySven]

OK, I reverted those mode changes for another time. However I found that although I had the lua and geyser files all nicely bundled up the Makefile that qmake puts together ignores the fact that the user compiling the Mudlet code may not have the superuser capabilities to put the executable file or the other lua files in the right parts of the file system. To fix this I've put together a shell and awk script pair that can munge the Makefile so that it works (both to install and uninstall) from at least a GNU/Linux and possibly other unix systems from within the Qt Creator IDE at least. This requires awk, sudo and something to ask for passwords in a GUI environment and all three of those are fairly standard system components.

[SlySven]

Aaaaaarrrgghh! Why, oh why, do we have a separate mac_src.pro as well as a src.pro Qt/qmake project file? Please, please, please, can we merge them - wrapping bits in unix:/win32:/macx: conditionals as needed. I think this will be doable - I see that APP_MUDLET_LUA_FILES.files is set to "mudlet-lua" (a path NOT a file specifier ???) and APP_MUDLET_LUA_FILES.path to "Contents/Resources" - where are the geyser files put when building on a Mac - do they go in the SAME directory as the main Lua files? Maybe replacing APP_MUDLET_LUA_FILE with LUA and also adding LUA_GEYSER to the QMAKE_APP_BUNDLE would work if the other Mac specific bits are copied across and LUA_DEFAULT_DIR is set so that the LUA_DEFAULT_PATH define is passed into the compiler.

P.S. Does anyone use CMake - I note the CMakeLists.txt here refers a minimum Qt version of 4.4 and uses Phonon...!

[Chris7]

The CMake is largely a relic I believe. Since I've been on the project it hasn't been used (so 4 years maybe?). The mac_src.pro I believe is under Vadi's control so I have never bothered with it. Being as I have always built from source on any platform I never really got into the particulars of our src.pro file. Not saying you aren't doing good work though, just I've been blissfully ignorant of problems in it :)

On Tue, Mar 25, 2014 at 4:01 PM, Stephen Lyons notifications@github.comwrote:

Aaaaaarrrgghh! Why, oh why, do we have a separate mac_src.pro as well as a src.pro Qt/qmake project file? Please, please, please, can we merge them - wrapping bits in unix:/win32:/macx: conditionals as needed. I think this will be doable - I see that APP_MUDLET_LUA_FILES.files is set to "mudlet-lua" (a path NOT a file specifier ???) and APP_MUDLET_LUA_FILES.path to "Contents/Resources" - where are the geyser files put when building on a Mac - do they go in the SAME directory as the main Lua files? Maybe replacing APP_MUDLET_LUA_FILE with LUA and also adding LUA_GEYSER to the QMAKE_APP_BUNDLE would work if the other Mac specific bits are copied across and LUA_DEFAULT_DIR is set so that the LUA_DEFAULT_PATH define is passed into the compiler.

P.S. Does anyone use CMake - I note the CMakeLists.txt here refers a minimum Qt version of 4.4 and uses Phonon...!

Reply to this email directly or view it on GitHubhttps://github.com/Chris7/Mudlet2/pull/31#issuecomment-38613228 .

[SlySven]

For reference I'm using Qt 5.1 (haven 't got space at present to install a later version because of the temp file space needed during installation) at present building on Debian Wheezy (not Sid) GNU/Linux.

[SlySven]

When you say "building from source" what are you implying?

[vadi2]

"Why, oh why, do we have a separate mac_src.pro as well as a src.pro Qt/qmake project file?" - because Heiko made a separate one and hardcoded everything to his Mac.

" Please, please, please, can we merge them" - yes, that is in the plans.

Lua files - I haven't yet read at the changes you do, but they are supposed to be copied over as one. Mudlet then calls LuaGlobal, LuaGlobal loads the rest. The .paths are mac-specific stuff for packaging a nice bundle together.

CMake is a relic, however we do have a person who fixed it up for Windows and even made Mudlet compile with MSVC.

[SlySven]

Well for *nix make "install" sets .files is a list of files that get placed in the .path directory. I can believe that Contents/Resource could be a (relative) path so there is not a problem with the use of path as it is a build platform difference which macx: / unix: conditionals will cover. I know that it is possible to use wildcards to select files to add to the .files element so the only awkwardness would be if the Mac App-bundler wants only a path for that or will accept an explicit list of files as I have them in the src.pro file so then you could use something like:

QMAKE_APP_BUNDLE.files += LUA.files \
    LUA_GEYSER.files

I still don't know though whether the geyser files end up in a subdirectory off of the main ones on a Mac installation?

[vadi2]

Geyser files are not separate - and they are loaded by LuaGlobal at runtime, check out how that expects them to be.

On Wed, Mar 26, 2014 at 7:59 AM, Stephen Lyons notifications@github.comwrote:

Well for *nix make "install" sets .files is a list of files that get placed in the .path directory. I can believe that Contents/Resource could be a (relative) path so there is not a problem with the use of path as it is a build platform difference which macx: / unix: conditionals will cover. I know that it is possible to use wildcards to select files to add to the .files element so the only awkwardness would be if the Mac App-bundler wants only a path for that or will accept an explicit list of files as I have them in the src.pro file so then you could use something like:

QMAKE_APP_BUNDLE.files += LUA.files \ LUA_GEYSER.files

I still don't know though whether the geyser files end up in a subdirectory off of the main ones on a Mac installation?

— Reply to this email directly or view it on GitHubhttps://github.com/Chris7/Mudlet2/pull/31#issuecomment-38626532 .

[SlySven]

Well the version I'm seeing does pull in the geyser files from a geyser subdirectory off the main lua - and AFAITIK - all the files put into an "install set" end up in the same directory - which is why I had to do two of them one for the main ones and one for the geyser. Now I suspect that the same applies on the Mac hardware so that was why I was querying whether the Mac App bundle does that through some magic effect when you give it only a directory so that it actually reproduces the "relative to the given directory" paths to wild-carded files in sub-directories; so that :

APP_MUDLET_LUA_FILES.files = mudlet-lua

is maybe a short-cut for

APP_MUDLET_LUA_FILES.files = $${PWD}/mudlet-lua/*

Where $${PWD} is the qmake variable for where the source files it is compiling are.

[vadi2]

I haven't looked at what you're trying to do yet exactly, from my understanding you were trying to make make install work under Linux.

Yeah, on OSX, the whole mudlet-lua folder gets copied as-is into the Resources part of a bundle. Inside mudlet-lua is the lua/ folder, all those files there, and then the geyser/ folder. I've worked on ironing that bit out - it works well, and if I understand that your scope is fixing make install, that shouldn't be affected.

[SlySven]

Had to rewind/rebase/undo(?) that commit "Merge preparation:...". I was trying to be too clever I think, I thought I needed to tweak the multiple pull requests so that one would follow into the next. However as my current crop of developments all have the same parent it seems that to pull them all together would need an octopus merge (?) which is not as simple as I, at first, thought. Ho, hum, you live and learn. 8-P

And, re previous comments, testing the use of just a path to the .files member of an install set like what is currently in the mac_src.pro DOES act as a wild-card for all the files/directories in the given directory. So it will be possible to use the same reference for both the mac and unix cases - with the proviso that the unix case will only work - with those shell and awk scripts, or now, without - if we DO NOT use the aforesaid directory path shortcut but explicitly list each file we want with separate install sets for each target directory. As the mac_src.pro currently is, it will also be copying the luadoc subdirectory stuff - which I'm not sure is what we want.

As this current organisation of src.pro has it the lua files will end up off of (say) the /usr/local/share folder as:

/usr/local/share
               +->/<other applications (before mudlet)>
               +->/mudlet
               |        +->/lua
               |              +->luaGlobal.lua
               |              +->DB.lua
               |              +->DebugTools.lua
               |              +->/geyser
               |              |        +->GeyserColor.lua
               |              |        +->GeyserContainer.lua
               |              |        +->GeyserGauge.lua
               |              |        +->...
               |              +->GMCP.lua
               |              +->GUIUtils.lua
               |              +->...
               +->/<other applications (after mudlet)>

It strikes me that the "mudlet-lua" segment of the source file tree is unhelpfully redundant, at least in the target directory, as the sole lua subdirectory it contains is obviously related to mudlet and would show as:

/usr/local/share
               +->/<other applications (before mudlet)>
               +->/mudlet
               |        +->/mudlet-lua
               |                     +->/lua
               |                           +->luaGlobal.lua
               |                           +->DB.lua
               |                           +->DebugTools.lua
               |                           +->/geyser
               |                           |        +->geyser.lua
               |                           |        +->geyserColor.lua
etc.

Note that documentation, once we get around to specifying what it should be, belongs in a mudlet subdirectory of /usr/local/share/docs/ and NOT a doc subdirectory of /usr/local/share/mudlet! (http://www.pathname.com/fhs/pub/fhs-2.3.html#PURPOSE26)

[vadi2]

Hi,

I've now got a chance to come about to look at this. Running Mudlet from Qt Creator for the first time, it failed to find the Lua files - even though they were still relative to the binary in mudet-lua/. This was on Linux. Could you look into this?

[SlySven]

Have you created and done the "deploy" step?

Refer to the (hopefully) attached images.

In the snapshot of the QtCreator "mudlet_QTCreator_run_setting.png" you can see that I have added a "Deployment" stage by using the "Add Deploy Step" and selecting "make" as the type of step.

In that step one needs to override the (for me) default '/usr/bin/make' with "/usr/bin/sudo" and to furnish it with the arguments '-A sh -c "/usr/bin/make install"'. The -A argument to sudo tells it to use a helper "askpass" GUI program if it needs to ask for a password to elevate user privileges. The program is specified by the "Path askpass /path/to/askpass" line in the /etc/sudo.conf file. Note that this sudo configuration file is NOT the same file as the /etc/sudoers file or equivalent fragments of the latter in /etc/sudoers.d/*. For me the "/path/to/askpass" is "/usr/lib/openssh/gnome-ssh-askpass". Anyway, sudo is instructed to spawn a bourne shell and to invoke "make install" with root privileges. If editing your sudo setup USE THE PROVIDED visudo RUN AS root OR WITH sudo - this will help to prevent sanity errors locking (non-)root users out from the commands that sudo is intended to permit authorised user to use...!

You may add a similar "Clean step" to run "make uninstall" in the same "privileged" manner BEFORE a "make clean". It should be done before as the latter command would probably nuke the Makefile in the build directory that all the different make invocations use.

You will also want to tell Qt Creator to use the "installed" mudlet program as by default it will run the compiled instance of the executable in the src tree. I did this by adding a "Run configuration" which I called "installed executable" which specified the mudlet executable and the working {install destination} directory - note that like the rest of the settings mentioned here there is no qmake shortcut to set these things that can be specified in the src.pro project file.

Note also that we can enabled shadow building (which is a great help if one wishes to build against multiple Qt Library versions or build both debug and release object code versions.

Anyhow it is getting late for me but see if the things picked out in the attachments help.

On 12/04/14 23:17, Vadim Peretokin wrote:

Hi,

I've now got a chance to come about to look at this. Running Mudlet from Qt Creator for the first time, it failed to find the Lua files - even though they were still relative to the binary in |mudet-lua/|. This was on Linux. Could you look into this?

— Reply to this email directly or view it on GitHub https://github.com/Chris7/Mudlet2/pull/31#issuecomment-40293251.

[SlySven]

Note that NOT doing the deploy step will result in the lua files not being copied across to the /usr/local/share/mudlet/lua location in the system file hierarchy - which is where the search then starts when you get the "mudlet can't find some essential files" dialogue and search procedure stuff activated. Developers can get around this by searching for and using the copy of the Lua-Global.lua file in their src/mudlet-lua/ file system - though they might want to remember that mudlet with this feature compiled in will then store the location in the "systemLuaFilesPath=..." entry in the "~/.config/Mudlet/Mudlet 1.0.conf" ini file and continue to use that remembered location until it finds it can't or it is removed/edited to an invalid (not a READABLE copy [write access not sought or needed] of the Lua-Global.lua file in the given directory) entry.

[vadi2]

Hi Stephen,

I haven't done deploy, I don't want to override my stable Mudlet with the unstable one. I think we should still retain the capability to run Mudlet from the src/ directory and have it find the relative Lua files to Itself (and if it can't, then search in the system-wide locations). This would continue to allow us to run a stable Mudlet for everyday use and work on a development version by using the 'run' command from Qt Creator.

[SlySven]

OK - I've transplanted in my runtime debug control "framework" onto the basis for this pull request. It is on the mudletDev_fileLocation_debugControls branch. That provides a series of place-holders which I have populated with an area for controls on the last tab of the profile preferences (e.g. the Mudlet "settings" thing) - there you will find a check-box that overrides the stored path to the Mudlet Lua files to the canonical version of the path "../src/mudlet-lua/lua" i.e. the files in the source tree for an instance of Mudlet being executed from there either as an in-source compilation or a shadow build. This setting is saved between Mudlet invocations and will act on profiles opened after it is checked. Resetting it will act in the same way.

The overall effect of the "frameworl" is to provide a way for developers to plug in their own, possibly temporary controls, to alter the run-time behaviour of any desired aspect of Mudlet. My example that is also included is something that allows the Room symbols on the 2D Mapper to be rendered less than 100% opaque so that I could investigate exit line drawing close to or under them.

For simplicity I guess we should close/abort this pull request and pull in the new branch that extends/supersedes this one.

[vadi2]

Okay, let's do that. I'm having an issue building that branch, I'll file a ticket.

[vadi2]

It doesn't look like I can - could you enable issues in your repository, so we can file issues on things that aren't merged yet?

[SlySven]

Done + given you (Chris / Vadim) contributor status on my repo.

[SlySven]

Closed, replace with pull request #34 which offers the above mentioned way to force the use of in source/build tree's Lua files.