Running BPBible 0.5 on Linux

[GoogleCodeExporter]

I've tried topb download and run this beta. So far it wouldn't start at all on 
Linux -- 

1. It's looking for the xulrunner binary in PATH, but on Debian it doesn't 
exist. It's called xulrunner-stub. Can a search for xulrunner-stub be added as 
well?

2. Then it complains it can't import/find wx.wc module. How do I fix that? I'd 
like to test the beta and provide feedback.

Thanks, and keep up the great work.

Original issue reported on code.google.com by war...@gmail.com on 5 Jan 2011 at 5:38

[GoogleCodeExporter]

1. Up till now I have only tested BPBible 0.5 on Ubuntu, and I don't think that 
is going to change.  However, I'm happy to receive comments about other distros 
and I have hopefully added support for xulrunner-stub in r1168 and r1169 
(though I haven't had a chance to test it).

2. As for wx.wc, that is the wxWebConnect library, which exists in a somewhat 
patched form at https://github.com/jonmmorgan/wxwebconnect and 
https://github.com/jonmmorgan/pywebconnect.  It can cause quite a bit of 
trouble getting it working, and I haven't really managed to make it reliable to 
build.  I have captured a brief dump of the build process at 
https://github.com/jonmmorgan/pywebconnect/wiki/Build-instructions, but I'm 
still not entirely sure that will work.  I might be able to get a better source 
distribution for it in the next week or so, which would not require SWIG and 
could just be run directly.  It's probably not worth trying until I do that 
(though you are welcome to try).

Original comment by jonmmor...@gmail.com on 6 Jan 2011 at 12:25

• Changed title: Running BPBible 0.5 on Linux
• Changed state: Accepted
• Added labels: Type-Defect, Priority-Medium, Milestone-Release0.5

[GoogleCodeExporter]

Thanks for your detailed response, Jon.

1. Just tested, this works now. And I can always test things on Debian.

2. I did check the links you provided. Some things in the build sequence I 
didn't completely understand (like what's a "matching" copy of wxWebConnect, 
does this imply there's version compatibility issues between wxWebConnect and 
wxWidgets? You have to get THE right version?)

The reason I didn't try building this thing (besides lack of building info) is 
because I only have access to a netbook right now, not suited for compiling 
wxWidgets on it. I think I'm gonna wait for "a better source distribution" for 
now.

3. On a separate note, even though there's a YouTube video, running the Linux 
version of BPBible is a bit of a pain for an average Joe, I think. I have some 
experience in building Debian packages, and I'd like to change that.

For starters, it wouldn't be hard to make a Debian/Ubuntu package that would 
ship with _Sword.so, _wc.so, etc. Eventually, I think we could package the 
python sword bindings and python wxWebConnect bindings as well.

A new DEB package for every minor/major release like 0.5.x would not take a lot 
of my time.

What do you guys think about this? If I did it, would you be able to test it on 
Ubuntu, and host it here?

Original comment by war...@gmail.com on 7 Jan 2011 at 5:04

[GoogleCodeExporter]

With respect to packaging, I agree it is not easy to set up and could be 
improved.  A month or so ago one of the Debian CrossWire Packaging Team said he 
was intending to package BPBible for Ubuntu (the discussion is at 
http://lists.alioth.debian.org/pipermail/pkg-crosswire-devel/2010-December/00137
2.html).  You might want to look at this first.  Based on that email, the Sword 
bindings have already been packaged.  I think packaging as part of the general 
Debian and Ubuntu repositories would be more useful than hosting packages here.

The matching copy of wxWebConnect just means a copy of wxWebConnect that 
matches the version of the Python bindings [it doesn't refer to wxWidgets - any 
version of 2.8 should work, and I know it has also been used with 2.9 on Mac].  
Taking the latest of each should probably be OK, but the Python bindings will 
definitely not work with the stock version of wxWebConnect from Kirix.  (I 
would want to include both wxWebConnect and the Python bindings in any source 
distribution of the bindings, to make sure there is no such mismatch).

Original comment by jonmmor...@gmail.com on 8 Jan 2011 at 4:42

[GoogleCodeExporter]

> I think packaging as part of the general Debian and Ubuntu repositories would 
be more useful than hosting packages here.

No doubt about it, but after reading the correspondence at the 
pkg-crosswire-devel mailing list I got the impression that the BPBible 
v0.5.x-specific changes introduced in the wxWebConnect and its Python bindings 
are not mature enough, which might make it very hard for those guys to push 
these changes upstream, and hence, get these libs packaged in Debian, am I 
right?

So yes, packaging as part of Debian/Ubuntu would be ideal, but that might take 
ages to happen, in light of the above. Therefore I still think a hackish DEB 
package that ships its own wxWebConnect libs pre-compiled would be tons better 
than just a zipped source code file for people to tinker with. Not ideal, no, 
but way better.

Personally, I could do with a zipped source archive, I don't need the DEB 
package myself. So if I make one, will you host it until a better solution like 
official Debian packages comes up?

Original comment by war...@gmail.com on 8 Jan 2011 at 6:51

[GoogleCodeExporter]

I'm not sure whether hosting a package here would be better than anywhere else 
(e.g. an Ubuntu PPU, Google Sites download, ...).  If it is then we would 
certainly consider doing it.

I haven't yet had time to make the changes necessary to have a zipped source 
code archive.  I don't know when I will do it.  However, I have uploaded the 
binaries (which I should have done earlier) to 
https://sites.google.com/site/jmmorganswordmodules/.  You could try that.  If 
you want to, download _wc.so and wc.py and put them in your Python 
site-packages directory.  I got it from 
/usr/lib/python2.6/dist-packages/wx-2.8-gtk2-unicode/wx.  Maybe you could try 
putting it there, but I don't know whether Debian puts things in the same 
places.

Original comment by jonmmor...@gmail.com on 14 Jan 2011 at 1:54

[GoogleCodeExporter]

Thanks for the binaries, Jon.

Here's my experience so far -- 

1. Debian looks for wc.py and _wc.so in the same location as Ubuntu, so no 
trouble here.

2. Then things go bad, though. BPBible loads and then (when I think it tries to 
render the content) it throws an ugly error box with gibberish instead of 
proper characters and complains it can't find libxul.so. I hit "Save" in that 
error window, the result is log1.txt

3. I have both xulrunner-1.9.1 (for xiphos) and xulrunner-1.9.2 (for firefox) 
installed. There's no conflict between them, they both work fine on my box. 
It's all within Debian package management.

The libraries are there:

% dlocate libxul.so
xulrunner-1.9.2: /usr/lib/xulrunner-1.9.2/libxul.so
xulrunner-1.9.1: /usr/lib/xulrunner-1.9.1/libxul.so

I assumed on Ubuntu they belong in /usr/lib, so I tried symlinking from 
/usr/lib/libxul.so to /usr/lib/xulrunner-1.9.2/libxul.so or to the 1.9.1 
version.

The error is gone (does this confirm my theory about Ubuntu putting libxul.so 
in /usr/lib?), but BPBible won't even load then and zsh tells me it segfaults, 
although no such message is in the logs. I saved a log to log2.txt

Just a wild guess: Did you compile smth against a specific version of 
xulrunner? If so, what's the exact xulrunner version you're running? and your 
Ubuntu version?

Also, where's libxul.so on your system? What does "find /usr -name 'libxul.so'" 
say?

I'd like to bug-test but can't get it to work so far.

Original comment by war...@gmail.com on 14 Jan 2011 at 11:45

Attachments:

• log1.txt
• log2.txt

[GoogleCodeExporter]

It shouldn't be compiled against an exact version of XULRunner.  However, it 
does need to run against some version of XULRunner 1.9.2.  In Ubuntu they are 
not in /usr/lib, but it finds the XULRunner directory in 
/usr/lib/xulrunner-1.9.2.13.  The version I am running has changed from XR 
1.9.2.11 to 1.9.2.12 to 1.9.2.13 without it stopping working.  It should link 
to libxul.so in the directory specified as /usr/lib/xulrunner-1.9.2.  It looks 
like it has detected XULRunner 1.9.2.13 OK.  I'm a little concerned about 
"Couldn't set locale or even english!!!! Bad things may happen!!!"  Don't know, 
though.

Original comment by jonmmor...@gmail.com on 15 Jan 2011 at 12:36

[GoogleCodeExporter]

Are you running 32-bit or 64-bit?  Those binaries are for 32-bit, so I assume 
you will be using 32-bit.  It's possible there are some differences in 
alignment of the standard structures, but that isn't supposed to happen...

Up until the Pango warning about invalid UTF-8 (which I would guess is the 
problem, though I don't know what causes it) it looks like a fairly normal log. 
 From the web, it seems like such errors can be to do with having an invalid 
locale, but obviously that wasn't a problem with 0.4.7.  Do you get any similar 
Pango warnings from 0.4.7?

Original comment by jonmmor...@gmail.com on 15 Jan 2011 at 11:34

[GoogleCodeExporter]

The "Couldn't set locale or even english!!!! Bad things may happen!!!" was 
always there, it's a long-standing bug on linux (issue 123, issue 159). Here's 
a log from r1077:

13:57:29 Couldn't set at all, smudging...
13:57:29 Couldn't set locale or even english!!!! Bad things may happen!!!
13:57:29 Couldn't add wx catalog 'en'

r1077 doesn't throw pango warnings, no, this is a new issue with r1171. I'm not 
sure, but these pango warnings seem to be connected with the gibberish in the 
message about not finding the libxul.so library. This is another issue, it may 
even be pretty harmless, or not visible to the user. The primary issue that the 
library is not found, no idea why :-(

Original comment by war...@gmail.com on 15 Jan 2011 at 12:04

[GoogleCodeExporter]

Oh, I forgot -- I'm running a 32-bit system.

Original comment by war...@gmail.com on 15 Jan 2011 at 12:13

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

OK, here's an update --

I installed xulrunner from luvid-updates 
(http://packages.ubuntu.com/lucid-updates/i386/xulrunner-1.9.2/download). 
BPBible loads now (and the 0.5.b1 looks very nice, too :)

It does display another error about some other missing libs (log.txt) But this 
may have smth to do with the Ubuntu/Debian xulrunner mismatch. Dunno.

I guess I do have to compile my own libs against a Debian xulrunner to settle 
this, what do you think?

Original comment by war...@gmail.com on 15 Jan 2011 at 12:24

Attachments:

• log.txt

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

OK, I symlinked the two missing libs to their proper Debian locations and now 
BPBible starts without throwing visible errors.

Here's what I did -- 

cd /usr/lib/ && ln -s libsqlite3.so.0 libsqlite3.so
cd /usr/lib/ && ln -s libsoftokn3.so  nss/libsoftokn3.so

Now, of course iceweasel/firefox complains about missing libs and would't start 
:)

I can at least start testing now, though. I will open issues per every bug.

I'm not sure about this specific issue, though. As far as I understand, it's an 
issue of wc.so lib that needs to be compiled on Debian Squeeze against current 
Debian xulrunner. I might find time to try it, but I might need help as the 
procedure is tricky.

Then I could try and package a BPBible Lucid package with your binaries as well 
as a Debian Squeeze package with mine.

Just thinking out loud here, maybe there's a better course of action?

Original comment by war...@gmail.com on 15 Jan 2011 at 1:45

[GoogleCodeExporter]

Congratulations for persisting and getting that far.  It's unfortunately not 
easy.

Obviously there is a problem finding libraries.  However, I'm not sure that it 
is due to the Debian / Ubuntu mismatch, since I have experienced at least some 
of them on Ubuntu.  Ultimately, I think they are problems with how wxWebConnect 
loads libraries:

1. It seems when looking for "libsqlite3.so" it does not see "libsqlite3.so.0" 
as a match.  I don't know enough about dynamic linking on Linux to know whether 
this is a problem with how it links with dlopen(), or whether it should also be 
searching for libsqlite3.so.<sonumber>.  I'll have to look into it.

2. libsoftokn3.so cannot be found: This is actually also a problem with my 
Ubuntu system.  However, I found that it threw up an error message which I 
could then dismiss, and after that BPBible seemed to work OK.  The reason 
libsoftokn3.so is loaded is because it is listed in the dependent libraries 
list for XULRunner (dependentlibs.list in the XULRunner directory).  However, 
unlike all the other libraries in that list, it was not in the XULRunner 
directory.  I don't know whether that is a problem with Ubuntu / Debian 
packaging or with wxWebConnect's expectations.  That is another thing I will 
need to look into.  Certainly on Windows both sqlite3.dll and softkon3.dll are 
in the XULRunner directory as well as being in dependentlibs.list.  
Intriguingly, I found that on my Ubuntu machine libsoftokn3.so was in my 
Firefox-3.6.13 directory but not in my XULRunner 1.9.2.13 directory.  No idea 
why yet.

Original comment by jonmmor...@gmail.com on 16 Jan 2011 at 1:55

[GoogleCodeExporter]

Well, I don't know about Linux dll linking either, but yeah, some part of 
BPBible (whether xulrunner or wxWebConnect) is just looking for a specific 
*name* of the libs, like libsqlite3.so and not libsqlite3.so.0, for example.

When packaging for Debian/Ubuntu it'd be obviously a very, very bad idea to 
alter a file in another package (like 
/usr/lib/xulrunner-1.9.2/dependentlibs.list), but making a symlink 
automatically when a package is installed is totally OK. So this is not a big 
issue, IMHO, but I'm no expert.

All I know is if I do --

cd /usr/lib/ && ln -s libsqlite3.so.0 libsqlite3.so
cd /usr/lib/ && ln -s libsoftokn3.so  nss/libsoftokn3.so

bpbible loads all the libs just fine, no complains, and we could easily make 
this symlinking part of Debian/Ubnuntu package.

Just to clarify: When I said Debian/Ubuntu mismatch I meant just name, not 
code, mismatches. Like on Ubuntu a file is called libsqlite3.so.0 and not 
libsqlite3.so, or vice versa, so you get fewer DLL load warnings on Ubuntu than 
I do on Debian. Like I said, no big deal.

There's almost certainly a code mismatch/incompatibility between Ubuntu and 
Debian xulrunners/wc.so's. That's because on Debian your wc.so makes bpbible 
segfault. Now *that's* an issue, as it means I probably need to recompile 
wxWidgets et al.

Original comment by war...@gmail.com on 16 Jan 2011 at 9:21

[GoogleCodeExporter]

Jon,

I'm trying to compile my own wc.so by following your build instructions. 

According to the README of wxWidgets 2.8.10, the latest available SWIG patch is 
swig-1.3.29.patch. But SWIG version 1.3.29 seems to only support Python 2.4, 
not even Python 2.5, which is way too low.

Question: Which SWIG version did you use? And which Python version?

Original comment by war...@gmail.com on 17 Jan 2011 at 2:00

[GoogleCodeExporter]

My understanding is that XULRunner and wxWebConnect are intended to be able to 
load from the XULRunner directory without needing any globally defined 
libraries (e.g. in /usr/lib).  Obviously it does work loading from the standard 
library path, but if I can find a way of avoiding that I would prefer it.  I'm 
inclined to think that making changes to /usr/lib should also be avoided if it 
is possible.

I think SWIG should support Python 2.4 and greater.  I used it with Python 2.6. 
 I used the pre-patched source package from 
http://wxpython.wxcommunity.com/tools/SWIG-1.3.29-wx.tar.gz.

Original comment by jonmmor...@gmail.com on 17 Jan 2011 at 2:27

[GoogleCodeExporter]

Thanks for the info, but where should I run your setup.py form, exactly?

wxwidgets2.8-2.8.10.1/wxPython/contrib?

Original comment by war...@gmail.com on 17 Jan 2011 at 10:42

[GoogleCodeExporter]

Incidentally, if I run it from wxwidgets2.8-2.8.10.1/wxPython/contrib, I get 
this error:

Traceback (most recent call last):
  File "./setup-wc.py", line 42, in <module>
    from config import *
  File "/home/*/Apps/bpbible/config.py", line 1, in <module>
    from backend.verse_template import VerseTemplate, SmartVerseTemplate
  File "/home/*/Apps/bpbible/backend/verse_template.py", line 3, in <module>
    from swlib.pysw import process_digits
  File "/home/*/Apps/bpbible/swlib/pysw.py", line 1499, in <module>
    change_locale("bpbible", "abbr")
  File "/home/*/Apps/bpbible/swlib/pysw.py", line 1453, in change_locale
    worked, locale, locale_encoding = get_locale(lang, additional=additional)
  File "/home/*/Apps/bpbible/swlib/pysw.py", line 1444, in get_locale
    assert locale.getEncoding() == "UTF-8", "Only UTF-8 locales supported (locale was %s)" % locale.getName()
AssertionError: Only UTF-8 locales supported (locale was en)

Is setup-wc.py supposed to import stuff my BPBible installation directory? A 
little confused here... Would appreciate your help.

Original comment by war...@gmail.com on 17 Jan 2011 at 10:49

[GoogleCodeExporter]

It should be run from the base directory (e.g. wxWidgets2.8-2.8.10.1).

As for the config error, presumably that shows your BPBible directory is in 
your Python path.  There is a config module in this directory.  It is probably 
actually looking for a config module used in setuptools.  I would remove the 
BPBible path from your PYTHONPATH if it is there, then try again.

Original comment by jonmmor...@gmail.com on 17 Jan 2011 at 10:59

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

OK, this is very weird.

1. I got my _wc.so to build (I had to adjust your build instructions in certain 
places), but it still doesn't work!

Traceback (most recent call last):
  File "./bpbible.py", line 53, in <module>
    import wx.wc
  File "/usr/lib/python2.6/dist-packages/wx-2.8-gtk2-unicode/wx/wc.py", line 8, in <module>
    import _wc
ImportError: /usr/lib/python2.6/dist-packages/wx-2.8-gtk2-unicode/wx/_wc.so: 
undefined symbol: _ZN22wxWebControlXmlHandlerC1Ev

Does this mean anything to you?

2. for some obscure reason your wc.so started working! I'll explore possible 
reasons for it, and test it on another Debian box.

Anyway, your _wc.so seems to work on my box... This probably needs more testing.

Original comment by war...@gmail.com on 18 Jan 2011 at 11:31

[GoogleCodeExporter]

1. Sorry, what it means is that not only did I give you wrong build 
instructions (which I have just fixed: thanks for the comment), but I also 
uploaded an old version of the setup-wc.py which does not include 
xh_webcontrol.cpp.  To fix this, you can either download a new version of 
setup-wc.py or just add the following line into your setup-wc.py below the line 
for webprefs.cpp:
                      '%s/webconnect/xh_webcontrol.cpp' % location,                   

2. I still have no idea why it fluctuates from non-working to working and vice 
versa.

Original comment by jonmmor...@gmail.com on 19 Jan 2011 at 1:40

[GoogleCodeExporter]

1. Good news, with the new setup-wc.py, it compiles and SEEMS to work fine on 
my box. No errors or warnings so far. Hooray, I got me my very own _wc.so ;)

I'll test it some more. I still want to make a DEB for 0.5, for the release.

2. It's not anymore, I think.

Thanks for all the help.

Original comment by war...@gmail.com on 19 Jan 2011 at 9:20

[GoogleCodeExporter]

Issue 231 has been merged into this issue.

Original comment by jonmmor...@gmail.com on 6 May 2012 at 7:47