Update Firefox to ESR31

[dmik]

We need to update our Firefox codebase to ESR31. This will include restoring OS/2 parts removed between ESR24 and ESR31 back in business.

[dmik]

According to http://hg.mozilla.org/releases/mozilla-esr31/, the latest ESR31 release is 31.6.0, FIREFOX_31_6_0esr_RELEASE, released Wed, 25 Mar 2015 23:30:15 -0400. This one I am about to pull into our repository.

[dmik]

I've committed FIREFOX_31_6_0esr_RELEASE to vendor-esr in 1fd092e86ee88b0ed46470ca969eb6b140e2ea59. Things I briefly noticed:

• OS/2 code is removed indeed.
• Many tests have been replaced / reorganized (the majority of changes seems to be here).
• Some parts got huge update (gfx, js-jit, stlport).

Merging these changes into master will surely not be very easy (more than 46 thousand files have been changed).

Note that I've also started off https://github.com/bitwiseworks/mozilla-os2/wiki/Developers to collect developer-related notes and described our repository structure there so that it's easier to maintain it in the future.

[dryeo]

On 04/07/15 04:48 PM, Dmitriy Kuminov wrote:

OS/2 code is removed indeed.

There's still some in nsprpub, nss and ldap/sdks (used by TB/SM). Comm-central has also been stripped of OS/2 code.

[dmik]

I guess that they consider nsprpub a separate project so the cleanup doesn't touch it.

[dmik]

BTW, it seems that most work of removing the OS/2 code was done here: https://bugzilla.mozilla.org/show_bug.cgi?id=969757. However, there are additions to it like this one https://bugzilla.mozilla.org/show_bug.cgi?id=974486 and this https://bugzilla.mozilla.org/show_bug.cgi?id=974370. This makes me think that there may be other removals and I really wonder how to best make sure they all are kept — checking each and every modified file is obviously a too much for a human being.

Perhaps I need to create a single giant diff between ESR24 and ESR31 and search for anything that look like os2 or OS/2 there.

[dryeo]

On 04/08/15 12:04 PM, Dmitriy Kuminov wrote:

Perhaps I need to create a single giant diff between ESR24 and ESR31 and search for anything that look like |os2| or |OS/2| there.

Be better to apply the 1.4MB diff from 969757 to 31 and then do a diff between 29 and 31 otherwise it'll be a huge undertaking.

[dmik]

Yes, but this still leaves us a chance that we skip something. The diff between esr24 and esr31 is huge indeed (290MB) but I think that a simple search for os2 and os/2 there should work. I will also be looking into the original 1.4 M diff in parallel.

[dmik]

Important note: there will be a single commit that brings the ESR31 changes and restores all the OS/2 things they have cut out around version 29. This will also include numerous adaptions of the existing OS/2 code to the significantly changed parts (e.g. they have reorganized a lot in toolkit/components/osfile). The reason for it to be a single commit rather a series of the separate ones is because of many conflicting changes. E.g. there are many files that were removed upstream but have been changed by us during our OS/2 work. Applying the original ERS31 commit would mean deleting these files and then resurrecting them again somehow including all our previous patches. This looks much more complex than doing a single "all-inclusive" commit. After all, we will be always able to compare the original ESR31 diff with what we merge into our repo and see what has been changed in between.

The above doesn't mean though that the result will immediately build and run. It is impossible to take all the details into account w/o even compiling the code. But it will be a minor work I hope.

This also explains why it is taking so long. But see above.

[dmik]

The main part of merge is done — I've resolved all conflicts and restored many removals. Now the last part — checking through the 1.4 M diff and others to pick up the remaining OS/2 removals and get rid of them in the final commit.

[dmik]

So, this historical merge is finally done, 49,288 changed files with 13,717 additions and 8,144 deletions, see 8b9ba001473bcabdff75269287409968abb56bac. I tried to do as much OS/2 work as possible (including adapting new code to OS/2 at once as described above) but of course some glitches are still possible. This is about to be sorted during the build stage.

Before building, I have to go through my todo list and check things that I marked as requiring attention (new config variables, some questionable changes etc.).

[dmik]

The above relates to Mozilla bug 969757. The OS/2 code removed in bugs 974486 and 974379 was restored separately, in 23d9cd12cb0dabb604f76566c336d27b5762baf0.

[dmik]

And here we have the first problem with configure. Of course, it's virtualenv:

Installing setuptools, pip...
  Complete output from command D:/Coding/mozilla/ma...nv/bin/python2.7.exe -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip:
  Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "D:/Coding/mozilla/master/python/virtualenv/virtualenv_support/pip-1.5.4-py2.py3-none-any.whl/pip/__init__.py", line 9, in <module>
  File "D:/Coding/mozilla/master/python/virtualenv/virtualenv_support/pip-1.5.4-py2.py3-none-any.whl/pip/log.py", line 9, in <module>
  File "D:/Coding/mozilla/master/python/virtualenv/virtualenv_support/pip-1.5.4-py2.py3-none-any.whl/pip/_vendor/colorama/__init__.py", line 2, in <module>
  File "D:/Coding/mozilla/master/python/virtualenv/virtualenv_support/pip-1.5.4-py2.py3-none-any.whl/pip/_vendor/colorama/initialise.py", line 5, in <module>
  File "D:/Coding/mozilla/master/python/virtualenv/virtualenv_support/pip-1.5.4-py2.py3-none-any.whl/pip/_vendor/colorama/ansitowin32.py", line 6, in <module>
  File "D:/Coding/mozilla/master/python/virtualenv/virtualenv_support/pip-1.5.4-py2.py3-none-any.whl/pip/_vendor/colorama/winterm.py", line 2, in <module>
  File "D:/Coding/mozilla/master/python/virtualenv/virtualenv_support/pip-1.5.4-py2.py3-none-any.whl/pip/_vendor/colorama/win32.py", line 8, in <module>
  File "C:/USR/lib/python2.7/ctypes/__init__.py", line 453, in <module>
    pythonapi = PyDLL(None)
  File "C:/USR/lib/python2.7/ctypes/__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
NameError: global name '_dlopen' is not defined
----------------------------------------
...Installing setuptools, pip...done.
Traceback (most recent call last):
  File "D:/Coding/mozilla/master/python/virtualenv/virtualenv.py", line 2339, in <module>
    main()
  File "D:/Coding/mozilla/master/python/virtualenv/virtualenv.py", line 825, in main
    symlink=options.symlink)
  File "D:/Coding/mozilla/master/python/virtualenv/virtualenv.py", line 993, in create_environment
    install_wheel(to_install, py_executable, search_dirs)
  File "D:/Coding/mozilla/master/python/virtualenv/virtualenv.py", line 961, in install_wheel
    'PIP_NO_INDEX': '1'
  File "D:/Coding/mozilla/master/python/virtualenv/virtualenv.py", line 903, in call_subprocess
    % (cmd_desc, proc.returncode))
OSError: Command D:/Coding/mozilla/ma...nv/bin/python2.7.exe -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip failed with error code 1
Traceback (most recent call last):
  File "D:/Coding/mozilla/master/python/mozbuild/mozbuild/virtualenv.py", line 473, in <module>
    manager.ensure()
  File "D:/Coding/mozilla/master/python/mozbuild/mozbuild/virtualenv.py", line 128, in ensure
    return self.build()
  File "D:/Coding/mozilla/master/python/mozbuild/mozbuild/virtualenv.py", line 371, in build
    self.create()
  File "D:/Coding/mozilla/master/python/mozbuild/mozbuild/virtualenv.py", line 147, in create
    raise Exception('Error creating virtualenv.')
Exception: Error creating virtualenv.

[dmik]

They have updated the pip package from 1.2.1 to 1.5.4 and setuptools from 0.6.c11 to 2.2 in the meantime. So I think the problem is there. Digging it now.

[dmik]

The problem is a combination of factors. First, it's the new pip that uses the colorama library for console output. Second, this colorama library uses the ctypes standard python module that provides a bridge between python and C functions in native DLLs. This module apparently misses any OS/2 support. OTOH, I know that Yuri used the pyqt library that provides python interface to Qt and that works on OS/2... I need to collect more info.

The colorama module attempts to load the system Win32 DLLs to do powerful console control (colors etc) and fails to use the load function. If the function itself succeeded it would return false (no WIn32 DLL) and it should switch colorama to non-WIn32 mode IIGIC. I will try to add a stub that returns false to see if it works.

[dmik]

Ok, this dirty hack to /usr/lib/python2.7/ctypes/__init__.py:

--- __init__.py.b   2015-02-26 21:23:20.000000000 +0400
+++ __init__.py 2015-04-23 21:55:04.000000000 +0400
@@ -140,6 +140,10 @@
 elif _os.name == "posix":
     from _ctypes import dlopen as _dlopen

+elif _os.name == "os2":
+    def _dlopen(name, mode):
+        return
+
 from _ctypes import sizeof, byref, addressof, alignment, resize
 from _ctypes import get_errno, set_errno
 from _ctypes import _SimpleCData

did the trick. I created a separate ticket for that task here: http://trac.netlabs.org/rpm/ticket/140.

Going further, configure now fails on the missing libpulse package. Investigating.

[dmik]

I got further through all configure stuff but it now fails upon the final Makefile generation step when processing master/media/libyuv/moz.build. The failure is this one:

The error appears to be part of the mozbuild.frontend.reader Python module itself! It is possible you have stumbled across a legitimate bug.

Traceback (most recent call last):
  File "D:/Coding/mozilla/master/python/mozbuild/mozbuild/frontend/reader.py", line 722, in read_mozbuild
    descend=descend, metadata=metadata):
  File "D:/Coding/mozilla/master/python/mozbuild/mozbuild/frontend/reader.py", line 819, in _read_mozbuild
    non_unified_sources = non_unified_sources):
  File "D:/Coding/mozilla/master/python/mozbuild/mozbuild/frontend/gyp_reader.py", line 111, in read_from_gyp
    params=params)
  File "D:/Coding/mozilla/master/media/webrtc/trunk/tools/gyp/pylib/gyp/__init__.py", line 132, in Load
    params['parallel'])
  File "D:/Coding/mozilla/master/media/webrtc/trunk/tools/gyp/pylib/gyp/input.py", line 2560, in Load
    variables, includes, depth, check, True)
  File "D:/Coding/mozilla/master/media/webrtc/trunk/tools/gyp/pylib/gyp/input.py", line 391, in LoadTargetBuildFile
    build_file_data, PHASE_EARLY, variables, build_file_path)
  File "D:/Coding/mozilla/master/media/webrtc/trunk/tools/gyp/pylib/gyp/input.py", line 1142, in ProcessVariablesAndConditionsInDict
    variables, build_file, 'variables')
  File "D:/Coding/mozilla/master/media/webrtc/trunk/tools/gyp/pylib/gyp/input.py", line 1142, in ProcessVariablesAndConditionsInDict
    variables, build_file, 'variables')
  File "D:/Coding/mozilla/master/media/webrtc/trunk/tools/gyp/pylib/gyp/input.py", line 1142, in ProcessVariablesAndConditionsInDict
    variables, build_file, 'variables')
  File "D:/Coding/mozilla/master/media/webrtc/trunk/tools/gyp/pylib/gyp/input.py", line 1194, in ProcessVariablesAndConditionsInDict
    ProcessConditionsInDict(the_dict, phase, variables, build_file)
  File "D:/Coding/mozilla/master/media/webrtc/trunk/tools/gyp/pylib/gyp/input.py", line 1071, in ProcessConditionsInDict
    variables, build_file)
  File "D:/Coding/mozilla/master/media/webrtc/trunk/tools/gyp/pylib/gyp/input.py", line 1149, in ProcessVariablesAndConditionsInDict
    expanded = ExpandVariables(value, phase, variables, build_file)
  File "D:/Coding/mozilla/master/media/webrtc/trunk/tools/gyp/pylib/gyp/input.py", line 872, in ExpandVariables
    cwd=build_file_dir)
  File "C:/USR/lib/python2.7/subprocess.py", line 709, in __init__
    errread, errwrite)
  File "C:/USR/lib/python2.7/subprocess.py", line 1326, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Looks like some OS/2 bits are missing again.

[dmik]

It turns out that python's subprocess.Popen() is broken on OS/2. At least here it fails to execute a complex command that involves piping (e.g. uname | sed BLAH like in the above case). Digging into python.

[dmik]

Created a python ticket for the above: http://trac.netlabs.org/rpm/ticket/141.

[dmik]

Okay, with the temporary workaround for Python (by replacing /bin/sh with /@unixroot/usr/bin/sh in subprocess.py) and a bunch of fixes in fd2ff6b0fa5c2838e32ad0f5c4ee823efbd63cb2 the configure stage now completes successfully.

I will try to build it now.

[dmik]

The build goes on. Fixing merge leftovers.

What really annoys me is that I get megabytes of this garbage now:

D:\Temp/ccerZFqd.s: Assembler messages:
D:\Temp/ccerZFqd.s:836: Warning: .stabs: description field '128bd' too big, try a different debug format

This comes from gas and seems to be somehow related to the compiler switch (from 4.7.3 to 4.9.2).

[dmik]

Passing -Wa,-W to gcc should shut up gas but the problem is that I don't know which file produces such an assembly (I need to build in single-thread mode to know this). Will leave it for later.

[dmik]

Faced the first more or less problem: libvpx was updated and requires more OS/2 porting. I did it mostly, but now fight with nasm that doesn't like one new .asm file:

nasm -o vp9_copy_sse2.o -f obj -I. -ID:/Coding/mozilla/master/media/libvpx/ -ID:/Coding/mozilla/master/media/libvpx/vpx_ports/  -g    D:/Coding/mozilla/master/m
edia/libvpx/vp9/common/x86/vp9_copy_sse2.asm
D:/Coding/mozilla/master/media/libvpx/vp9/common/x86/vp9_copy_sse2.asm:151: error: symbol `eaxm' undefined
D:/Coding/mozilla/master/media/libvpx/vp9/common/x86/vp9_copy_sse2.asm:151: error: symbol `ecxm' undefined
D:/Coding/mozilla/master/media/libvpx/vp9/common/x86/vp9_copy_sse2.asm:151: error: symbol `edxm' undefined
D:/Coding/mozilla/master/media/libvpx/vp9/common/x86/vp9_copy_sse2.asm:151: error: symbol `ebxm' undefined
D:/Coding/mozilla/master/media/libvpx/vp9/common/x86/vp9_copy_sse2.asm:152: error: symbol `eaxm' undefined
D:/Coding/mozilla/master/media/libvpx/vp9/common/x86/vp9_copy_sse2.asm:152: error: symbol `ecxm' undefined
D:/Coding/mozilla/master/media/libvpx/vp9/common/x86/vp9_copy_sse2.asm:152: error: symbol `edxm' undefined
D:/Coding/mozilla/master/media/libvpx/vp9/common/x86/vp9_copy_sse2.asm:152: error: symbol `ebxm' undefined

[dryeo]

On 04/29/15 05:34 PM, Dmitriy Kuminov wrote:

but now fight with |nasm| that doesn't like one new |.asm| file:

The NASM included with RPM/YUM is old, try upgrading. There's a newer one on Hobbes or the latest which I believe was forgotten to be released but available in their git repository. Hmm, release 2.11.08 seems to have been tagged yesterday (29/04/15) and tarball is available at www.nasm.us Officially libvpx does (did?) support both NASM and YASM

[dryeo]

There's also KO Myung-Hun port of libvpx, http://hobbes.nmsu.edu/download/pub/os2/dev/mm/libvpx-v1.3.0.zip. Probably pretty complete and he is good at adapting YASM code to NASM.

[dryeo]

Perhaps this,

diff --git a/media/libvpx/third_party/x86inc/x86inc.asm b/media/libvpx/third_party/x86inc/x86inc.asm
index 9e4918a..afba317 100644
--- a/media/libvpx/third_party/x86inc/x86inc.asm
+++ b/media/libvpx/third_party/x86inc/x86inc.asm
@@ -234,10 +234,10 @@ ALIGNMODE k7
         %define r%1mp %2
     %elif ARCH_X86_64 ; memory
         %define r%1m [rsp + stack_offset + %6]
-        %define r%1mp qword r %+ %1m
+        %define r%1mp qword r %+ %1 %+ m
     %else
         %define r%1m [esp + stack_offset + %6]
-        %define r%1mp dword r %+ %1m
+        %define r%1mp dword r %+ %1 %+ m
     %endif
     %define r%1  %2
 %endmacro

[dmik]

Yes, I know that our nasm is old, but thanks for pointing out, Dave! FF may be built using the system libvpx and that's what we will certainly do at some point.

[dmik]

BTW, both the KOMH port and FF 31 libvpx are 1.3.0 which means we can take KOMH patches for now... However, quickly looking at them shows that some things are missing. I will dig deeper.

Ah, it's clear what's missing. FF guys have libvpx pre-generated and these pre-generated files are obviously missing in the KOMH port. Sigh, you see how things get complicated when ppl try to create all-in-one things...

[dmik]

Actually there is not much to apply from the KOMH work, only the above .asm fix, the rest is either not related or arguable. Anyway, with 1f951f59a66081f1ca0ff3e443357098caeab38c and 9651c584c2805a2360a853a04156b567b91218e8 libvpx now builds.

[dmik]

A short report: going further step by step (refer to the commits). The current task is adopting the widget code and in particular, the nsWindow class. There are lots of changes over there.

[dmik]

BTW, widget code seems to be done, there is really not much left (I hope).

[dmik]

Note that in 7a04051ed7a238a9259d489dc3bf66f94464abea I disabled the Profiler code (accidentally enabled in 9732dbb1f4c5085fad34b9a4d068119ded894b3c) again as it is not ported to OS/2. I doubt that we will need it soon so I postpone this for later in #103.

[dmik]

It seems that everything is successfully built by now. There are some failures at link stage (due to missing symbols) but that's expected. Will fix them next.

[dmik]

Attempting to link XUL.DLL showed that there is a number new classes and methods that need to be implemented (e.g. VolatileBuffer, GetAppliactionParametersFrom(ACString)). Shouldn't be a big deal.

[dmik]

I have added all missing stuff and there is no unresolved externals any more, however the link stage still fails with Error 1 and no message. This smells like a wlink problem... I already saw it back then when it was not capable of handling too many objects. I need to investigate more.

[dmik]

Well, the wlink error is in the .map file, actually:

Error! E2021: size of segment text exceeds 64k by 21899 bytes (which is 0x558b).

Judging from the rest of map file, it refers to the following AUTO segment:

text                                  AUTO           0003:00000000   0001558b
rodata                                AUTO           0004:00000000   00000470

I have no idea why this segment is created now; it is not present in the previous build of Mozilla (which is the same GGG 4.9.2 compiler so I doubt the compiler is in vain). The same applies to the rodata segment. I need to do some research.

[StevenLevine]

I suspect something (probably the compiler) is generating 16-bit segments that should be 32-bit segments.

[dryeo]

On 05/22/15 01:26 PM, StevenLevine wrote:

I suspect something (probably the compiler) is generating 16-bit segments that should be 32-bit segments.

Actually as usual it is the assembler, we didn't finish porting to obj format. With this patch I successfully built xul.dll. Wish github supported attachments

From 94e7d872edf400f7857025d9a45649a2a2b62991 Mon Sep 17 00:00:00 2001
From: Dave Yeo <dave.r.yeo@gmail.com>
Date: Sat, 23 May 2015 16:46:56 -0700
Subject: [PATCH] Fix libvpx to use USE32 to avoid 16bit segments with obj format

---
 media/libvpx/third_party/x86inc/x86inc.asm          | 3 +++
 media/libvpx/vp9/common/x86/vp9_copy_sse2.asm       | 4 ++++
 media/libvpx/vp9/common/x86/vp9_intrapred_sse2.asm  | 5 +++++
 media/libvpx/vp9/common/x86/vp9_intrapred_ssse3.asm | 4 ++++
 4 files changed, 16 insertions(+)

diff --git a/media/libvpx/third_party/x86inc/x86inc.asm b/media/libvpx/third_party/x86inc/x86inc.asm
index afba317..e1905e2 100644
--- a/media/libvpx/third_party/x86inc/x86inc.asm
+++ b/media/libvpx/third_party/x86inc/x86inc.asm
@@ -83,6 +83,9 @@
         fakegot:
     %elifidn __OUTPUT_FORMAT__,aout
         section .text
+    %elifidn __OUTPUT_FORMAT__,obj
+       section .text  align=16 public use32 class=CODE
+       section  .rodata align=16 public use32 class=DATA
     %else
         SECTION .rodata align=%1
     %endif
diff --git a/media/libvpx/vp9/common/x86/vp9_copy_sse2.asm b/media/libvpx/vp9/common/x86/vp9_copy_sse2.asm
index dd522c6..d9d8987 100644
--- a/media/libvpx/vp9/common/x86/vp9_copy_sse2.asm
+++ b/media/libvpx/vp9/common/x86/vp9_copy_sse2.asm
@@ -10,7 +10,11 @@

 %include "third_party/x86inc/x86inc.asm"

+%ifidn __OUTPUT_FORMAT__,obj
+SECTION .text  align=16 public use32 class=CODE
+%else
 SECTION .text
+%endif

 %macro convolve_fn 1
 INIT_XMM sse2
diff --git a/media/libvpx/vp9/common/x86/vp9_intrapred_sse2.asm b/media/libvpx/vp9/common/x86/vp9_intrapred_sse2.asm
index 69b07f6..7c3f529 100644
--- a/media/libvpx/vp9/common/x86/vp9_intrapred_sse2.asm
+++ b/media/libvpx/vp9/common/x86/vp9_intrapred_sse2.asm
@@ -16,7 +16,12 @@ pw_8:  times 8 dw 8
 pw_16: times 8 dw 16
 pw_32: times 8 dw 32

+%ifidn __OUTPUT_FORMAT__,obj
+SECTION .text  align=16 public use32 class=CODE
+%else
 SECTION .text
+%endif
+

 INIT_MMX sse
 cglobal dc_predictor_4x4, 4, 5, 2, dst, stride, above, left, goffset
diff --git a/media/libvpx/vp9/common/x86/vp9_intrapred_ssse3.asm b/media/libvpx/vp9/common/x86/vp9_intrapred_ssse3.asm
index 88df9b2..e69ec87 100644
--- a/media/libvpx/vp9/common/x86/vp9_intrapred_ssse3.asm
+++ b/media/libvpx/vp9/common/x86/vp9_intrapred_ssse3.asm
@@ -31,7 +31,11 @@ sh_bfedcba9876543210: db 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0
 sh_b1233: db 1, 2, 3, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
 sh_b2333: db 2, 3, 3, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0

+%ifidn __OUTPUT_FORMAT__,obj
+SECTION .text  align=16 public use32 class=CODE
+%else
 SECTION .text
+%endif

 INIT_MMX ssse3
 cglobal h_predictor_4x4, 2, 4, 3, dst, stride, line, left
-- 
2.0.0

Hopefully formatting is correct, github not loading so using email.

[dryeo]

The line section .rodata... is not needed in x86inc.asm if DATA is changed to CODE. Seems .text is not ideal but changing it to .rodata or .data causes linking xul.dll to fail due to 16bit segment

[dryeo]

Next build break is building brwsrcmp.dll. Need to do s/bin/lib/ in configure.in for XPCOM_FROZEN_LDOPTS and MOZ_COMPONENT_NSPR_LIBS Re-add SHORT_LIBNAME = brwsrcmp to browser/components/build/Makefile.in and remove SHORT_LIBNAME from the list of depreciated variables in config.mk Hopefully speed things up as it takes forever to build here.

[dmik]

Dave, thanks for the hint, I guessed something like that. I committed a complete (I hope) solution in c99319a1b8f0f185297c61ffa4124fd8ded1c58c. With it, we don't have any extra text or rodata segments anymore, everything lives in the standard TEXT32 segment now (as for the normal code produced by C/C++). Going further. Will take your comments on brwsrcmp into account.

[dmik]

Finally, I have everything built. It doesn't start though yet, crashes with this in popuplog right before showing the window:

05-26-2015  11:17:14  SYS3175  PID fe87  TID 0001  Slot 00a9
D:\CODING\MOZILLA\MASTER-BUILD\DIST\BIN\FIREFOX.EXE
c0000005
14a13c7a
P1=00000001  P2=00000010  P3=XXXXXXXX  P4=XXXXXXXX
EAX=00000000  EBX=00000000  ECX=1bc00002  EDX=2003013c
ESI=174126bc  EDI=00000000
DS=0053  DSACC=d0f3  DSLIM=9fffffff
ES=0053  ESACC=d0f3  ESLIM=9fffffff
FS=150b  FSACC=00f3  FSLIM=00000030
GS=0000  GSACC=****  GSLIM=********
CS:EIP=005b:1d84e554  CSACC=d0df  CSLIM=9fffffff
SS:ESP=0053:0012fd6c  SSACC=d0f3  SSLIM=9fffffff
EBP=0012fdb8  FLG=00010202

XUL.DLL 0001:00a63c7a

But that's kinda expected. I think it's time to cook a debug build to get some logging.

[dmik]

Actually, w/o the debug build I already get something interesting. First, this is in stderr:

1432642878795   addons.xpi-utils    WARN    Extensions database D:\Users\mozdev\Mozilla\Firefox\Profiles\pycia5on.default\extensions.json exists but is not readable; rebuilding: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIFileInputStream.init]"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: resource://gre/modules/addons/XPIProvider.jsm -> resource://gre/modules/addons/XPIProviderUtils.js :: XPIDB_syncLoadDB :: line 540"  data: no] Stack trace: XPIDB_syncLoadDB()@resource://gre/modules/addons/XPIProvider.jsm -> resource://gre/modules/addons/XPIProviderUtils.js:540 < XPI_checkForChanges()@resource://gre/modules/addons/XPIProvider.jsm:3423 < XPI_startup()@resource://gre/modules/addons/XPIProvider.jsm:1884 < AMI_callProviders()@resource://gre/modules/AddonManager.jsm:869 < AMI_startup()@resource://gre/modules/AddonManager.jsm:745 < AMP_startup()@resource://gre/modules/AddonManager.jsm:2318 < AMC_observe()@addonManager.js:55 < <file:unknown>
1432642882043   addons.xpi  ERROR   Failed to process extension changes at startup: ReferenceError: OS is not defined (resource://gre/modules/osfile/osfile_unix_allthreads.jsm:44) JS Stack trace: @osfile_unix_allthreads.jsm:44:1 < @osfile_async_front.jsm:44:3 < @osfile.jsm:11:3 < @DeferredSave.jsm:11:1 < XPCU_moduleLambda@XPCOMUtils.jsm:226:7 < XPCU_defineLazyGetter/<.get@XPCOMUtils.jsm:177:9 < this.XPIDatabase.saveChanges@XPIProviderUtils.js:432:7 < XPIDB_makeAddonVisible@XPIProviderUtils.js:1329:5 < XPIDB_addAddonMetadata@XPIProviderUtils.js:1269:7 < addMetadata@XPIProvider.jsm:3076:11 < XPI_processFileChanges@XPIProvider.jsm:3255:1 < XPI_checkForChanges@XPIProvider.jsm:3428:1 < XPI_startup@XPIProvider.jsm:1884:1 < AMI_callProviders@AddonManager.jsm:869:11 < AMI_startup@AddonManager.jsm:745:1 < AMP_startup@AddonManager.jsm:2318:5 < AMC_observe@addonManager.js:55:7
DIVE is disabled - Panorama's shadow-buffer is enabled
*************************
A coding exception was thrown and uncaught in a Task.

Full message: ReferenceError: OS is not defined
Full stack: @resource://gre/modules/osfile/osfile_unix_allthreads.jsm:44:1
@resource://gre/modules/osfile/osfile_async_front.jsm:44:3
@resource://gre/modules/osfile.jsm:11:3
@resource://gre/modules/PlacesBackups.jsm:17:1
XPCU_moduleLambda@resource://gre/modules/XPCOMUtils.jsm:226:7
XPCU_defineLazyGetter/<.get@resource://gre/modules/XPCOMUtils.jsm:177:9
BG__initPlaces/<@file:///D:/CODING/MOZILLA/MASTER-BUILD/DIST/BIN/browser/components/nsBrowserGlue.js:1170:11
TaskImpl_run@resource://gre/modules/Task.jsm:298:13
TaskImpl@resource://gre/modules/Task.jsm:247:3
createAsyncFunction/asyncFunction@resource://gre/modules/Task.jsm:224:7
Task_spawn@resource://gre/modules/Task.jsm:139:5
BG__initPlaces@file:///D:/CODING/MOZILLA/MASTER-BUILD/DIST/BIN/browser/components/nsBrowserGlue.js:1200:1
BG_observe@file:///D:/CODING/MOZILLA/MASTER-BUILD/DIST/BIN/browser/components/nsBrowserGlue.js:237:11

*************************
Creating 12E7_01.TRP

And this in TRP:

 Filename: D:\CODING\MOZILLA\MASTER-BUILD\DIST\BIN\XUL.DLL 05/26/2015 01:52:18 59,083,908
 Address:  005B:10973C7A (0001:00A63C7A)
 Cause:    Attempted to read from 00000010
           (not a valid address)
...
 --------  ---------  --------  -------------  -----------------------
 Trap  ->  10973C7A   XUL       0001:00A63C7A  Matrix.h#269 __ZN7mozilla6layers20RotatedContentBuffer15IsClippingCheapEPNS_3gfx10DrawTargetERK11nsIntRegion + A 0001:00A63C70 (D:\Coding\mozilla\master\gfx\layers\RotatedBuffer.cpp)

[dmik]

Ok, the OS exception has gone now (see cd1efee183fea88a2473e2466735d758a407358b), looking for the rest. As for the trap, it seems that RotatedContentBuffer::IsClippingCheap(DrawTarget* aTarget, const nsIntRegion& aRegion) in RotateBuffer.cpp is called with aTarget being NULL for an unknown reason (yet).

[dryeo]

First run I get Trap -> 13BF4D4A XUL 0001:009F4D4A between mozilla::layers::RotatedContentBuffer::IsClippingCheap + A and mozilla::layers::RotatedContentBuffer::DrawTo - 116 (both in RotatedBuffer.o)

Which is probably the same as you.

[dmik]

Yes, I bet it's the same. Strange that you have different numbers. What build type are you using?

[dryeo]

On 05/26/15 08:50 AM, Dmitriy Kuminov wrote:

Yes, I bet it's the same. Strange that you have different numbers. What build type are you using?

Not sure what you mean by build type. I'm building FF and the different numbers may be due to targeting i686 rather then i486. Never did have your first error and I note that the choose profile window is mostly drawn when the crash happens (I have FF & SM set to permanently start with the Profile Manager) I currently have debug symbols turned off as well

[dmik]

This is what I meant. Ok. Regarding the crash. For some reason, a gfxContext instance passed to BasicLayerManager::PaintSelfOrChildren is a cairo-based one, but the code thinks it's a Thebes based one and crashes because the Thebes target is not set. I need to deeply learn the program logic to find why.

[dryeo]

Another problem is that -mstackrealign seems not to be being used when -sse2 is used

[dmik]

Yes, the -mstackrealign thingy is in my internal todo list. I didn't do it on merge so that it can be checked once again and committed in a separate step when everything builds.

[dmik]

And here we are! The first run of Firefox 31.6.0 on OS/2!

The commits will follow.