Devanagari font not working

[manishp11]

I am printing text written in devanagari unicode to jpg and png

all i can see in images in boxes.

python 2 and 3 both, with pillow pip install

# Draw (Bitmap Font) Text to Image
# coding: utf-8
from PIL import Image, ImageDraw, ImageFont

def reverseColor(r, g, b):
    return (255 - r, 255 - g, 255 - b)
def grayscaleColor(r, g, b):
    a = (r + g + b) / 3
    return (a, a, a)

text = "तपःस्वाध्यायनिरतं तपस्वी वाग्विदां वरम्"
textColor = (0, 255, 0) # RGB lime
#textBackgroundColor = (255, 0, 0) # RGB Red
textX = 200 # text width in pixels
textY = 50 # text height in pixels
textTopLeftX = 100
textTopLeftY = 100

# create new image
# imgx = 1920 # image width in pixels
# imgy = 1080 # image height in pixels
# image = Image.new("RGB", (imgx, imgy))

# load image
image = Image.open("input.png")
(imgx, imgy) = image.size
image = image.resize((imgx, imgy), Image.BICUBIC)

font = ImageFont.truetype("Devnew.ttf", 15) # load default bitmap font
(width, height) = font.getsize(text)
textImage = font.getmask(text)
pixels = image.load()
for y in range(imgy):
    by = int(height * (y - textTopLeftY) / textY + 0.5)
    if by >= 0 and by < height:
        for x in range(imgx):
            bx = int(width * (x - textTopLeftX) / textX + 0.5)
            if bx >= 0 and bx < width:
                if textImage.getpixel((bx, by)) == 0: # text background
                    # pass # transparent background
                    # pixels[x, y] = textBackgroundColor
                    (r, g, b, a) = pixels[x, y]
                    #(r, g, b) = pixels[x, y]
                    #pixels[x, y] = grayscaleColor(r, g, b)
                else: # text foreground
                    pixels[x, y] = textColor                
                    (r, g, b, a) = pixels[x, y]
                    # (r, g, b) = pixels[x, y]
                    # pixels[x, y] = reverseColor(r, g, b)

image.save("output.png", "PNG")

[wiredfool]

That appears to be a very complicated way to draw text. Have you tried the more straightforward way to use the api? http://pillow.readthedocs.io/en/3.4.x/reference/ImageDraw.html#example-draw-partial-opacity-text

[rakeshvar]

This has been an issue for a long time. I had the same issue with Indic text rendering. After trying a lot, I finally use cffi in python3. See scribe.py and 'cffi_wrapper.py` in chamanti_ocr.

[manishp11]

@wiredfool my problem is not with drawing text i am able to draw it with the default font but facing issue with the font i am using actually not only this font i have tried. all are attached below.

@rakeshvar thats for the clearification can you share some working POC work for the fonts i am working with as i need these fonts only to work on.

[manishp11]

ok it doesn't seem I can upload them, so you can refer the name of the fonts.

• Devnew.ttf
• Lohit-Marathi.ttf
• Sarai_07.ttf
• chandas1-2.ttf
• gargi.ttf
• lohit_kok.ttf
• lohit_mai.ttf
• lohit_sd.ttf
• nakula.ttf
• samanata.ttf
• Lohit-Devanagari.ttf
• Samyak-Devanagari.ttf
• kalimati.ttf
• lohit_ks.ttf
• lohit_ne.ttf
• mangal.ttf
• sahadeva.ttf

[wiredfool]

I'm not sure I have access to any of those fonts, but I suspect that this is a case of not having the normalization step from unicode -> combined glyphs. There's a PR that adds support for it in #1682, but it needs work.

[manishp11]

@wiredfool i am security tester not a developer its wasn't already easy for me so #1682 seems tough for me to go for. thanks for your support i will keep looking for some other way around.

[wiredfool]

If you can render standard uncomplicated fonts, then I'd bet that #1682 is one solution, however convoluted.

FWIW, your rendering looks quite complicated, and it appears that it's a really slow equivalent to the standard ImageDraw.text command. In general, if you're looping over pixels in python, there's a better way.

[rakeshvar]

@manishp11 Check out this repo to see how to get unicode text as a numpy array in Python3. You will have to tweak with it to get what you need. It does not use Pillow or Harfbuf, etc.

[wiredfool]

That repo seems to work on my machine. It's pretty easy to skip the dependency on numpy and directly output an image from the scribe function:

    return Image.frombuffer('L', (width, height), data, "raw", 'L', 0, 1)
    #return np.frombuffer(data, dtype=np.uint8).reshape((height, width))

and then later just display the image, rather than trimming it and dumping to the console.

I think that image could be directly used as a font mask in ImageDraw.text.

The question I have is: What's the level of correctness/generality here? Is this a general solution for 'complex' scripts, bypassing the need for harfbuf with different dependencies, or is this just working because the string in the script has already been preprocessed?

[rakeshvar]

@wiredfool That repo uses the same mechanism that linux uses to display complex scripts. i.e. via the pango, cairo and pangocairo libraries. The repo does it in a dependable way using cffi. O, also in that repo I use eight-bit images. One can use 24-bit images instead. The string is not pre-processed in anyway. It is just raw unicode, as it ought to be.

[wiredfool]

Ok, so it's a complete solution through a different stack than we're using. This has potential.

Do you have a license for it? Hopefully something compatible with Pillow?

[aclark4life]

@wiredfool If I understand correctly, you'd like @rakeshvar to provide a license for https://github.com/rakeshvar/unicode_text_to_image_array so we can use it in Pillow?

[wiredfool]

Maybe, there's been development since then in a different implementation using harfbuzz and raqm.

Otoh, this might be a reasonable alternate implementation, as it's a small interface and we don't actually have to ship the binaries.

[nengine]

Hi, Please let me know complex font rendering for Indic fonts are working now? I am using PIL on windows. Thanks.

[HinTak]

Most likely you need a text = text.decode('utf8') if you are python 2.x . See my comments in

https://github.com/rougier/freetype-py/issues/57

And also the changes I recently made to freetype-py https://github.com/rougier/freetype-py/pull/52/commits/a4e35f785d4ab912dbfe2d28d2b50158512c4b35

[HinTak]

Actually from __future__ import unicode_literals should also work for python 2.x

[nengine]

Thanks HinTak, It is drawing fine with regular consonants, but what I meant was that when a vowel symbol is top on top of consonant like ကိ (little o shape on top of က) Pillow is rendering separately as shown below. Please let me know if this is something I could do with freetype-py too?

sylb = txt2png(u"ကိ",fopath, 80, 128)

[HinTak]

I have modified example 1 ( https://github.com/rougier/freetype-py/issues/54#issuecomment-298186703 ) to use the lohit font, and the initial phrase, and it seems to work:

+    filename= '/usr/share/fonts/lohit-devanagari/Lohit-Devanagari.ttf'
+    text    = 'तपःस्वाध्यायनिरतं तपस्वी वाग्विदां वरम्'

Here is the result (obviously I cannot read Devnagri, so you'll have to check it yourself):

I am fairly sure you need harfbuzz for "ကိ (little o shape on top of က)".

[rakeshvar]

@HinTak The rendering is not correct. It should look like तपःस्वाध्यायनिरतं स्वा ध्या नि etc. are wrong.

[nengine]

I did example 1 with text below and it came out similar to Pillow which is not correct. So it seems like I would need harfbuzz as you indicated. Would it be possible natively inside Pillow (or) freetype-py? @rakeshvar unicode_text_to_image_array repo works on Linux, but I can't make it work on Windows 10, so this is why I am interested in Pillow (or) freetype-py to work on Windows.

Many thanks,

text = u'ကိ'

[HinTak]

Here is freetype-py 's hello world example quickly changed to do the initial phrase (I can't read it - just cut-and-paste)

[HinTak]

Hmm, harfbuzz can be built for windows (and so does pango).... but it is quite an unnatural/involved task to do.

[khaledhosny]

There is already a pull request for complex text support https://github.com/python-pillow/Pillow/pull/2284.

[wiredfool]

This should be fixed in 4.2.0 with the merge of #2576

[nengine]

thanks @wiredfool. please let me know if this is going work with windows as well?

[wiredfool]

Short answer -- Not now. Dependencies are harder to build on windows, and libfribidi is GPL, so we wouldn't be able to distribute it in the binaries, at least by default.

[nengine]

No problem. Thanks.

[pankajsinghal]

@wiredfool @rakeshvar @aclark4life

This repo doesn't provide correct output.

The text कामकाजी महिलाओं के लिए देश में दिल्ली असुरक्षित, सिक्किम सबसे बेहतर: रिपोर्ट produces the following incorrect output

Was trying to produce a solution for https://stackoverflow.com/questions/39630916/how-can-i-print-hindi-sentencesunicode-on-image-in-python

[radarhere]

@pankajsinghal does this commit mean that your problem is resolved?

[KosukeHao]

have you solve the problem that pillow rendering hindi incorrect?

[Vegadhardik7]

Hey I had used pyvips to write hindi/devanagari text on an image using python.

First of all I installed pyvips, but when you import it will give you an error, so to overcome that issue you have to download file from pyvips and put it in your environment variable path and then you can put the text on image.

Actually I wrote article which contains step by step solution... I am sure this will help you.

https://www.infinitycodex.in/how-to-put-hindi-text-on-images-using

# IMPORTS

import os  
import pyvips  
import textwrap  
from PIL import *  
import pandas as pd  
from PIL import Image  
from PIL import ImageFont  
from PIL import ImageDraw   

# MAKE A FUNCTION  

def bgoutput(filename, text):  

   rendered_text, feedback = pyvips.Image.text(text,   
                         font='Mangal', fontfile='Mangal Regular.ttf',   
                         width=900, height=900,   
                         autofit_dpi=True) 

   rendered_text = rendered_text.gravity('centre', 1500, 1500)  

   image = rendered_text.new_from_image([0, 0, 0]).bandjoin(rendered_text)  
   image.write_to_file(f'{filename}.png') 

# GENERATE OUTPUT 1   

text = "परिस्थितियां विपरीत हो तो कुछ लोग टूट जाते हैं, और कुछ लोग रिकॉर्ड तोड़ देते हैं..!!"  
bgoutput('new', text)  

# COMBINE OUTPUT 1 WITH BACKGROUND IMAGE  

img = Image.open('new.png')   
b1 = Image.open('bg_img.png')  
img = img.resize((1000,1000))  
b1.paste(img,(50,50), mask=img)  

# GENERATE FINAL OUTPUT  

b1.save("final.png")

[radarhere]

The only complete example I can see here that doesn't involve other libraries is the original post. I expect the rest of the situations might easily be fixed with RAQM support, added in Pillow 4.2.0.

RAQM is now supported in Pillow on Windows, it just requires FriBiDi to be installed separately.

https://pillow.readthedocs.io/en/stable/installation.html

Pillow wheels since version 8.2.0 include a modified version of libraqm that loads libfribidi at runtime if it is installed. On Windows this requires compiling FriBiDi and installing fribidi.dll into a directory listed in the Dynamic-link library search order (Microsoft Learn) (fribidi-0.dll or libfribidi-0.dll are also detected). See Build Options to see how to build this version.

[radarhere]

ok it doesn't seem I can upload them, so you can refer the name of the fonts.

• Devnew.ttf
• Lohit-Marathi.ttf
• Sarai_07.ttf
• chandas1-2.ttf
• gargi.ttf
• lohit_kok.ttf
• lohit_mai.ttf
• lohit_sd.ttf
• nakula.ttf
• samanata.ttf
• Lohit-Devanagari.ttf
• Samyak-Devanagari.ttf
• kalimati.ttf
• lohit_ks.ttf
• lohit_ne.ttf
• mangal.ttf
• sahadeva.ttf

I was able to find fonts corresponding to Lohit-Marathi.ttf, gargi.ttf, nakula.ttf, Lohit-Devanagari.ttf, samanata.ttf, kalimati.ttf, mangal.ttf and sahadeva.ttf, and was able to get characters for all of them from

from PIL import Image, ImageDraw, ImageFont

text = "तपःस्वाध्यायनिरतं तपस्वी वाग्विदां वरम्"

fonts = [...]
image = Image.open("input.png").resize((750, len(fonts)*50))

for i, font_path in enumerate(fonts):
    font = ImageFont.truetype(font_path, 15)
    draw = ImageDraw.Draw(image)
    draw.text((0, i*50), text, font=font)

image.save("output.png")

The only font that generates boxes was the original one, Devnew.ttf.

hex(ord(text[0])) is 0x924. Inspecting Devnew.ttf with https://fontdrop.info/, that character cannot be seen, whereas it can when you inspect Lohit-Devanagari.ttf

So it seems that Devnew.ttf is simply missing the characters you're trying to write.

[radarhere]

Closing. If anyone has a complete example and specific font that doesn't work, using only Pillow, feel free to leave a comment, and this can be re-opened.