EXCEPTION_ACCESS_VIOLATION when trying generate font with FreeTypeFontGenerator

[GoogleCodeExporter]

Based on the March 6, 2012 blog post about generating fonts on the fly with 
FreeType, I implemented the following function:

{{{
private void createFont()
{
    FileHandle fontFile = Gdx.files.internal("font/arial.ttf");
    FreeTypeFontGenerator generator = new FreeTypeFontGenerator(fontFile);
    ftFont = generator.generateFont(40);
    generator.dispose();
}
}}}

When I try to execute this function, I get an EXCEPTION_ACCESS_VIOLATION.  The 
error report file seems to indicate that the offending call is:

com.badlogic.gdx.graphics.g2d.freetype.FreeType$Bitmap.getBuffer()Ljava/nio/Byte
Buffer;+4

And sure enough, if I step through the font generation, my instance dies on the 
second line of getPixmap(...) in the FreeType class:

{{{
public Pixmap getPixmap(Format format) {
    Pixmap pixmap = new Pixmap(getWidth(), getRows(), Format.Alpha);
--> BufferUtils.copy(getBuffer(), pixmap.getPixels(), 
pixmap.getPixels().capacity());
    ...
}
}}}

I've tried both desktop backends, both seem to exhibit this crash.  Any ideas?

Original issue reported on code.google.com by ben.dona...@graphitelab.com on 27 Mar 2012 at 11:13

[GoogleCodeExporter]

Forgot to mention, I don't get this crash on my home computer, which is running 
Windows 7 Home Edition 64-bit.

The workstation that DOES exhibit this crash is running Windows 7 Enterprise 
32-bit.

Original comment by ben.dona...@graphitelab.com on 27 Mar 2012 at 11:15

[GoogleCodeExporter]

Are you using the latest nightlies/trunk? i see if i can get my hands on a 
32-bit machine.

Original comment by badlogicgames on 27 Mar 2012 at 11:17

[GoogleCodeExporter]

I was using 3/21/2012 nightlies, but have subsequently tested with 3/28/2012 
nightlies as well.

I should also mention that I get the same crash when running the FreeTypeTest 
test within the gdx-tests suite.  I am building libgdx straight from the latest 
source when running the test suite.

Thanks for the quick response, and for your assistance!

Original comment by ben.dona...@graphitelab.com on 27 Mar 2012 at 11:24

[GoogleCodeExporter]

C:\Users\ben.donatelli>java -version
java version "1.6.0_31"
Java(TM) SE Runtime Environment (build 1.6.0_31-b05)
Java HotSpot(TM) Client VM (build 20.6-b01, mixed mode, sharing)

If this helps at all.

Original comment by ben.dona...@graphitelab.com on 27 Mar 2012 at 11:26

[GoogleCodeExporter]

Same problem here ... crash when loading ANY ttf-font under the desktop-app.

System:
Windows 7 Professional 64 bit
java version "1.6.0_31"
Java(TM) SE Runtime Environment (build 1.6.0_31-b05)
Java HotSpot(TM) Client VM (build 20.6-b01, mixed mode, sharing)

Original comment by tobias.w...@gmail.com on 12 Apr 2012 at 8:17

[GoogleCodeExporter]

Problem persists with JDK 7:
java version "1.7.0_03"
Java(TM) SE Runtime Environment (build 1.7.0_03-b05)
Java HotSpot(TM) Client VM (build 22.1-b02, mixed mode, sharing)

Original comment by ben.dona...@graphitelab.com on 12 Apr 2012 at 2:08

[GoogleCodeExporter]

I'm also seeing this on 32-bit Win 7 setup, using Java SE Runtime Environment 
1.6

Original comment by JasmineH...@gmail.com on 18 Apr 2012 at 5:10

[GoogleCodeExporter]

Issue 788 has been merged into this issue.

Original comment by badlogicgames on 18 Apr 2012 at 11:11

[GoogleCodeExporter]

Maybe this will help - this is the top of my crash-stack when loading a 
ttf-font:

Stack: [0x05070000,0x050c0000],  sp=0x050bf2bc,  free space=316k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  0x0000000c
j  
com.badlogic.gdx.graphics.g2d.freetype.FreeType$Bitmap.getBuffer()Ljava/nio/Byte
Buffer;+4
j  
com.badlogic.gdx.graphics.g2d.freetype.FreeType$Bitmap.getPixmap(Lcom/badlogic/g
dx/graphics/Pixmap$Format;)Lcom/badlogic/gdx/graphics/Pixmap;+20
j  
com.badlogic.gdx.graphics.g2d.freetype.FreeTypeFontGenerator.generateData(ILjava
/lang/String;Z)Lcom/badlogic/gdx/graphics/g2d/freetype/FreeTypeFontGenerator$Fre
eTypeBitmapFontData;+634

this is my code (works fine on android):

FileHandle fontFile = Gdx.files.internal(fontDir+"arial.ttf");
FreeTypeFontGenerator generator = new FreeTypeFontGenerator(fontFile);
FreeTypeBitmapFontData fontData = generator.generateData(15, FONT_CHARS, flip); 
// CRASH!

Original comment by tobias.w...@gmail.com on 18 Apr 2012 at 2:05

[GoogleCodeExporter]

what FONT_CHARS are you using?

Original comment by badlogicgames on 18 Apr 2012 at 2:07

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

I am using the following FONT_CHARS:
abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ()-'/

Original comment by ben.dona...@graphitelab.com on 18 Apr 2012 at 3:05

[GoogleCodeExporter]

String FONT_CHARS = "abcdefghijklmnopqrstuvwxyz";
(just to be sure that the error comes from somewhere else)

Original comment by tobias.w...@gmail.com on 19 Apr 2012 at 6:46

[GoogleCodeExporter]

Same problem too, tested with the nightly and the stable release.
Tested with FONT_CHARS provided on Comment 13 and the same problem too. I have 
only tested on desktop application.

Original comment by guilherm...@gmail.com on 19 Apr 2012 at 5:08

[GoogleCodeExporter]

This problem should finally be fixed in the latest nightlies. Please give those 
a try. If the problem prevails, i'll reopen this issue.

Original comment by badlogicgames on 29 Apr 2012 at 1:19

• Changed state: Fixed

[GoogleCodeExporter]

Updated to the April 30 nightlies.  Crash still occurs for me :(

I made sure to clean my Eclipse project, tried a few different TTF files, still 
doesn't work.

Original comment by ben.dona...@graphitelab.com on 30 Apr 2012 at 2:20

[GoogleCodeExporter]

Updated to latest nightly (01 May) and problem persists. Here is a stack trace 
if it helps in any way.
Thread [LWJGL Application] (Suspended)  
    FreeType$Bitmap.getBuffer() line: 334   
    FreeType$Bitmap.getPixmap(Pixmap$Format) line: 342  
    FreeTypeFontGenerator.generateData(int, String, boolean) line: 168  
    FreeTypeFontGenerator.generateFont(int, String, boolean) line: 71   
    MainMenu.<init>() line: 14  
    MyGame.create() line: 13    
    LwjglApplication.mainLoop() line: 144   
    LwjglApplication$1.run() line: 131  

And here is the console output
#A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x1492161c, pid=3512, tid=5916
#
# JRE version: 6.0_31-b05
# Java VM: Java HotSpot(TM) Client VM (20.6-b01 mixed mode windows-x86 )
# Problematic frame:
# C  0x1492161c

Original comment by guilherm...@gmail.com on 1 May 2012 at 3:46

[GoogleCodeExporter]

I'm also still seeing this problem - the issue needs to be re-opened.

Original comment by JasmineH...@gmail.com on 11 May 2012 at 4:12

[GoogleCodeExporter]

Original comment by badlogicgames on 12 May 2012 at 11:13

• Changed state: New

[GoogleCodeExporter]

Did you all update the gdx-natives.jar? Could you use a simple chars string, 
e.g. "abc" for testing?

Original comment by badlogicgames on 12 May 2012 at 11:15

[GoogleCodeExporter]

I'm still getting the problem with the nightlies I downloaded yesterday. I've 
tried it with several fonts (including Arial.ttf) and simple "ABC"-type strings.

Here are a few extracts from the log file:

# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000c, pid=3872, tid=5840
#
# JRE version: 6.0_31-b05
# Java VM: Java HotSpot(TM) Client VM (20.6-b01 mixed mode, sharing windows-x86 
)
# Problematic frame:
# C  0x0000000c

...

Stack: [0x044b0000,0x04500000],  sp=0x044ffb1c,  free space=318k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  0x0000000c
j  
com.badlogic.gdx.graphics.g2d.freetype.FreeType$Bitmap.getBuffer()Ljava/nio/Byte
Buffer;+4
j  
com.badlogic.gdx.graphics.g2d.freetype.FreeType$Bitmap.getPixmap(Lcom/badlogic/g
dx/graphics/Pixmap$Format;)Lcom/badlogic/gdx/graphics/Pixmap;+20
j  
com.badlogic.gdx.graphics.g2d.freetype.FreeTypeFontGenerator.generateData(ILjava
/lang/String;Z)Lcom/badlogic/gdx/graphics/g2d/freetype/FreeTypeFontGenerator$Fre
eTypeBitmapFontData;+630
j  
com.badlogic.gdx.graphics.g2d.freetype.FreeTypeFontGenerator.generateFont(ILjava
/lang/String;Z)Lcom/badlogic/gdx/graphics/g2d/BitmapFont;+4

...

---------------  S Y S T E M  ---------------

OS: Windows 7 Build 7601 Service Pack 1

CPU:total 2 (2 cores per cpu, 1 threads per core) family 15 model 75 stepping 
2, cmov, cx8, fxsr, mmx, sse, sse2, sse3, mmxext, 3dnow, 3dnowext

Memory: 4k page, physical 3144156k(1348580k free), swap 6286556k(4187800k free)

vm_info: Java HotSpot(TM) Client VM (20.6-b01) for windows-x86 JRE 
(1.6.0_31-b05), built on Feb  3 2012 18:44:09 by "java_re" with MS VC++ 7.1 
(VS2003)

Original comment by othoma...@gmail.com on 13 May 2012 at 12:02

[GoogleCodeExporter]

Tried the latest nightlies (5/24) and a simple char string "abc" and still got 
the crash.

Original comment by ben.dona...@graphitelab.com on 14 May 2012 at 10:25

[GoogleCodeExporter]

Additional info if this helps. Freetype crash happens on WindowsXP 32 bit too. 
But code works perfectly on Mac, Android. My code crashes on Win7, but another 
project which uses similar code but different font works on it. 

Original comment by juwalb...@gmail.com on 19 May 2012 at 8:53

[GoogleCodeExporter]

I have the same problem with my 32 bit windows 7 netbook. But not with my 
windows 7 64 bit PC.

Original comment by dslsh...@gmail.com on 26 May 2012 at 4:16

[GoogleCodeExporter]

Same here. Works on the phone, but doesn't work on Windows 7 x64. Tried a bunch 
of fonts and none of them work.

I did try to debug it a bit and saw that the characters used have no effect. It 
crashes on the first call of "bitmap.getPixmap(Format.RGBA8888);" - no matter 
which character I use.

Original comment by kichik on 14 Jun 2012 at 5:23

[GoogleCodeExporter]

I have the same problem on Windows desktop, and I still wait for the solution 
of this issue.

Original comment by dreamoes...@gmail.com on 23 Jun 2012 at 3:04

[GoogleCodeExporter]

Possibly related (current version is 2.4.10)

QUOTE:

CHANGES BETWEEN 2.4.9 and 2.4.10

  I. IMPORTANT BUG FIXES

    - Incremental glyph loading as needed by ghostscript was broken.

Original comment by mjoy...@vbservices.net on 30 Jun 2012 at 3:25

[GoogleCodeExporter]

a) I finally got the JNI/NDK stuff working....
b) Did a SVN get, did a rebuild of freetype-natives, compared jar contents....
c) Why does the one from the nightlies have a file dated APRIL ?
d) Why would my file sizes be LARGER, and not the same ?

e) When I use the one I built, it does not crash. When I use the smaller one 
from the nightlies, it crashes.

The gdx-freetype-natives.jar I built is attached for testing... good luck!

Additionally is a screenshot comparison of the file sizes and timestamps...

Original comment by mjoy...@vbservices.net on 30 Jun 2012 at 9:52

Attachments:

• gdx-freetype-natives.jar
• Screenshot-4.png

[GoogleCodeExporter]

Works for me with mjoy's 2.4.10. Thanks!

Original comment by kichik on 9 Sep 2012 at 8:52

[GoogleCodeExporter]

I now get what seems to be the same error on Android 2.3.3 running inside an 
emulator (armeabi).

09-15 22:13:35.901: W/dalvikvm(484): JNI WARNING: invalid values for address 
(0x0) or capacity (0)
09-15 22:13:35.912: I/dalvikvm(484): "GLThread" prio=5 tid=11 NATIVE
09-15 22:13:35.912: I/dalvikvm(484):   | group="main" sCount=0 dsCount=0 
obj=0x40524e70 self=0x2b7878
09-15 22:13:35.912: I/dalvikvm(484):   | sysTid=495 nice=0 sched=0/0 
cgrp=default handle=2848128
09-15 22:13:35.912: I/dalvikvm(484):   | schedstat=( 11325355113 7033671488 668 
)
09-15 22:13:35.912: I/dalvikvm(484):   at 
com.badlogic.gdx.graphics.g2d.freetype.FreeType$Bitmap.getBuffer(Native Method)
09-15 22:13:35.912: I/dalvikvm(484):   at 
com.badlogic.gdx.graphics.g2d.freetype.FreeType$Bitmap.getBuffer(FreeType.java:3
34)
09-15 22:13:35.912: I/dalvikvm(484):   at 
com.badlogic.gdx.graphics.g2d.freetype.FreeType$Bitmap.getPixmap(FreeType.java:3
42)
09-15 22:13:35.912: I/dalvikvm(484):   at 
com.badlogic.gdx.graphics.g2d.freetype.FreeTypeFontGenerator.generateData(FreeTy
peFontGenerator.java:168)
09-15 22:13:35.912: I/dalvikvm(484):   at 
com.badlogic.gdx.graphics.g2d.freetype.FreeTypeFontGenerator.generateFont(FreeTy
peFontGenerator.java:71)
...

I tried upgrading the Android libraries to FreeType 2.4.10 but it didn't help 
like it did for the desktop version. I also tried upgrading libgdx.so from 
0.9.6 to latest in git (58992dc21f02c8c6cef9618c976de84c6699e32a) but that 
didn't help either.

Original comment by kichik on 15 Sep 2012 at 11:33

[GoogleCodeExporter]

I came along the getBuffer error on a few devices and was unable to get rid of 
it no matter which freetype version I used. It also affected several android 
versions including 2.1 and 1.6. In addition freetype made a game I created for 
the last Ludum Dare crash on several desktop machines. I'll see if I can get a 
stacktrace from that error to post it here. For now I switched back to 
BitmapFonts as freetype was too unreliable in my experience.

Original comment by than...@gmail.com on 16 Sep 2012 at 5:46

[GoogleCodeExporter]

I have researched this issue a bit more and found a couple of issues causing 
this on Windows and the emulator.

On the emulator it's actually an exception thrown by CheckJNI 
(http://android-developers.blogspot.com/2011/07/debugging-android-jni-with-check
jni.html). It doesn't like it when env->NewDirectByteBuffer() is called with 
NULL buffer or 0 capacity. This happens when FreeType renders an empty 
character, like space. The bitmap is empty, FT_Bitmap::rows = 0, 
FT_Bitmap::pitch = 0, FT_Bitmap::width = 0, and FT_Bitmap::buffer = NULL. I 
couldn't find any evidence of this in FreeType documentation, but that's what 
I've seen multiple times in the debugger.

I haven't seen any instances of it crashing because of this without CheckJNI, 
but I'm in favor of listerning to the warnings :)

The Windows crash I found is a bit more complicated and not related to FreeType 
or its version after all. The compiler has just gone mad. 
Java_com_badlogic_gdx_graphics_g2d_freetype_FreeType_00024Bitmap_getBuffer() 
just accesses a random register, hoping it will have the right value. This is 
from the disassembly of gdx-freetype.dll from libgdx-0.9.6.zip.

--------
push    ebp
mov     ebp, esp
push    esi                       ; save ESI
push    ebx
mov     ecx, [ebp+arg_8]
mov     edx, [ecx+8]
mov     ebx, [ecx+8]
sar     edx, 1Fh
mov     eax, edx
xor     eax, ebx
sub     eax, edx
mov     edx, [ecx]
imul    eax, edx
mov     edx, eax
mov     [ebp+arg_8], eax
sar     edx, 1Fh
mov     eax, [ecx+0Ch]
mov     [ebp+arg_C], edx
mov     [ebp+arg_4], eax
lea     esp, [ebp-8]
pop     ebx
pop     esi                       ; restore ESI
pop     ebp
jmp     dword ptr [esi+394h]      ; XXX BUG use ESI as `env`
                                  ; try to call env->NewDirectByteBuffer
                                  ; BOOM
--------

It calculates the buffer size, gets the buffer address into EAX, sets up the 
stack properly, and finally calls a random address in memory. I have no 
explanation as to why it uses ESI as `env` instead of the one it gets on the 
stack. This should crash every time unless miraculously the JVM has `env` in 
ESI before calling this function. Since it works for some people on Windows, it 
seems, weirdly enough, this is sometimes the case.

mjoy's build, however, uses the proper value of `env`.

--------
sub     esp, 8
mov     edx, [esp+8+arg_8]
mov     [esp+8+var_8], ebx
mov     ebx, [esp+8+arg_0]    ; load env
mov     [esp+8+var_4], esi
mov     ecx, [edx+8]
mov     eax, [edx+8]
mov     esi, [ebx]            ; dereference env
mov     ebx, [esp+8+var_8]
sar     ecx, 1Fh
xor     eax, ecx
sub     eax, ecx
imul    eax, [edx]
mov     ecx, eax
sar     ecx, 1Fh
mov     [esp+8+arg_8], eax
mov     [esp+8+arg_C], ecx
mov     eax, [edx+0Ch]
mov     [esp+8+arg_4], eax
mov     eax, [esi+394h]       ; get NewDirectByteBuffer from env
mov     esi, [esp+8+var_4]
add     esp, 8
jmp     eax                   ; call env->NewDirectByteBuffer
--------

It seems to me like a compiler bug. I have no good suggestions as to how this 
can be avoided but burning the specific compiler version that was used to build 
this DLL and adding some safe guards in the build system to make sure it's 
never used again.

There may be other issues with FreeType 2.4.8 hiding behind this compiler bug. 
To verify, we need to build gdx-freetype.dll with a good compiler and FreeType 
2.4.8. Or FreeType can be upgraded 2.4.10, just in case. The changelog between 
2.4.8 and 2.4.10 seems to only be bug fixes, so I don't think it can hurt.

I have updated my pull request with a proposed workaround for the CheckJNI 
issue and attached a compiled JAR here. Using both this JAR and mjoy's 2.4.10 
natives JAR, I experience no more FreeType crashes on my desktop or phones.

Original comment by kichik on 16 Sep 2012 at 11:57

Attachments:

• gdx-freetype.jar

[GoogleCodeExporter]

I pulled in the 2.4.10 changes (thanks kichik) and compiled them on all 
platforms. Mac and Linux seem to work just fine, Windows 7 64-bit works (Java 7 
JDK), Windows 7 32-bit fails.

The Windows builds were created  by our build server, with MinGW and MinGW-64. 

32-bit MinGW
i586-mingw32msvc-g++ (GCC) 4.2.1-sjlj (mingw32-2)

64-bit MinGW
x86_64-w64-mingw32-g++ (GCC) 4.5.3 20110422 (prerelease)

The 64-bit cross compiler is reasonably up to date. The mingw32 compiler is 
what i got from the Ubuntu repositories. It's quite outdated it seems.

So i recompiled the 32-bit natives on Windows directly, with MinGW:
g++ (GCC) 4.5.2

And everything worked as expected.

Looks like we really ran into a compiler bug with MinGW 32-bit on Linux. Now i 
have to figure out how to get an up-to-date build chain for Windows 32-bit on 
my trusty old Ubuntu box.

Guys, i wouldn't have been able to figure this out without you. The reason this 
never turned up on my end was that locally i only had my Windows compiled 
binaries. I can reproduce it now, and the fix is obvious. Thanks a TON!

I'll let you know once the build server is updated. 

Original comment by badlogicgames on 17 Sep 2012 at 9:56

• Changed state: Accepted

[GoogleCodeExporter]

I'll try to get to the build server update tomorrow. It's already midnight :/

Original comment by badlogicgames on 17 Sep 2012 at 10:04

[GoogleCodeExporter]

Ok, screw sleep. I..

- upgraded the build server to Ubuntu 12.04
- replaced the build toolchain for Windows with mingw-w64 from the canonical 
repos (makes it easy for others to build stuff as well)
- fixed up some tangling issues in gdx-jnigen
- regenerated all the C/C++ code as well as the build scripts, using the new 
compiler prefixes for mingw (if you build stuff via mingw, you gotta update to 
mingw-w64 on your system containing the i683-xxx set of compiler tools)
- compiled all the binaries, pushed them to the repo (yeah, we need ivy)

this has fixed the issue on windows 32-bit for me. I also tested on windows 
64-bit, and mac os x.

Please, everyone test on linux if possible (nothing has changed really).

We should be all set, and i hope i can close this issue now. Thanks to 
everybody who contributed to the solution!

Original comment by badlogicgames on 18 Sep 2012 at 1:44

[GoogleCodeExporter]

Original comment by badlogicgames on 18 Sep 2012 at 1:44

• Changed state: Fixed

[GoogleCodeExporter]

gdx-freetype.dll seems good now. It actually uses `env` from the stack and not 
a random register. I also tested it on my machine and it works.

Thanks!

Original comment by kichik on 18 Sep 2012 at 1:49