Open Shreeshrii opened 7 years ago
\documentclass[a5paper,11pt,twoside,onecolumn,openany,final]{memoir}
\usepackage[a5paper,hmargin=0.7in,vmargin=0.7in]{geometry}
\usepackage{multicol,polyglossia,etoolbox,fontspec}
%
\XeTeXgenerateactualtext=1
%
\defaultfontfeatures{Mapping=tex-text}
\setmainlanguage{gujarati}
\setotherlanguage{english}
%
\setmainfont[Scale=1,Script=Gujarati]{Rasa-Regular.ttf}
\newfontfamily{\vedafont}[Scale=1.2]{Siddhanta}
\newfontfamily{\devafont}[Script=Devanagari,Scale=1.2]{Siddhanta}
%
\usepackage[Latin, Devanagari, DevanagariExtended, Gujarati, VedicExtensions]{ucharclasses}
%\setTransitionsFor{Devanagari}{\begingroup\devafont}{\endgroup}
\setTransitionsFor{Devanagari}{\begingroup\vedafont}{\endgroup}
\setTransitionsFor{DevanagariExtended}{\begingroup\vedafont}{\endgroup}
\setTransitionsFor{VedicExtensions}{\begingroup\vedafont}{\endgroup}
%
\begin{document}
This trial is to use the combining marks and characters from Devanagari, DevanagariExtended and VedicExtensions unicode blocks with text in other Indic scripts, eg. Gujarati.
\\
This is the Gujarati text with the vedic accents which are from Devanagari, DevanagariExtended, VedicExtensions blocks.
\\
ૐ ભૂર્ભુવઃ સ્વઃ \\
ૐ ॥ હિર॑ણ્યવર્ણાં॒ હરિ॑ણીં સુ॒વર્ણ॑રજ॒તસ્ર॑જામ્ ।\\
ચ॒ન્દ્રાં હિ॒રણ્મ॑યીં લ॒ક્ષ્મીં જાત॑વેદો મ॒ આવ॑હ ॥ ૧॥\\
\\
તાં મ॒ આવ॑હ॒ જાત॑વેદો લ॒ક્ષ્મીમન॑પગા॒મિની᳚મ્ ।\\
યસ્યાં॒ હિર॑ણ્યં વિ॒ન્દેયં॒ ગામશ્વં॒ પુરુ॑ષાન॒હમ્ ॥ ૨॥\\
\\
This is plain devanagari text.
\\
अस्य श्रीपञ्चमुखिहनुमत्कवचस्तोत्रमन्त्रस्य, ब्रह्मा ऋषिः ।
गायत्री छन्दः । श्रीहनुमान् देवता । रां बीजम् । मं शक्तिः ।
\end{document}
\newfontfamily{\vedafont}[Scale=1.2]{Siddhanta}
\setTransitionsFor{Devanagari}{\begingroup\vedafont}{\endgroup}
When using the above, devanagari text is not rendered ok. see attached image
\newfontfamily{\devafont}[Script=Devanagari,Scale=1.2]{Siddhanta}
\setTransitionsFor{Devanagari}{\begingroup\devafont}{\endgroup}
When using the above, the devanagari script text is ok, but the combining marks don't work. Please see attached image.
Is there a way to use ucharclasses for both combining marks and devanagari? Thanks!
I'm not sure what would cause this, other than if some of the combining characters are treated as being "not part of the same unicode block". Would you happen to have a reduced case with the shortest possible word (or even few non-sense letters that involve the combining characters?)
This might be another instance of https://github.com/Pomax/ucharclasses/issues/13
Sorry for the delay in reply. I did some more testing and what I find is that the combining marks (that I tested with) from DevanagariExtended and VedicExtensions unicode blocks do combine correctly with other scripts.
The problem is with the devanagari block which has codepoints for devanagari script but also has certain combining marks which can be used with other Indic scripts.
When I do NOT specify Script=Devanagari then the combining happens correctly, but not otherwise.
A reduced case is attached below.
\documentclass[a4paper,11pt,twoside,onecolumn,openany,final]{memoir}
\usepackage[a4paper,hmargin=0.7in,vmargin=0.7in]{geometry}
\usepackage{multicol,polyglossia,etoolbox,fontspec}
\usepackage{xcolor}
%
\XeTeXgenerateactualtext=1
%
\defaultfontfeatures{Mapping=tex-text}
\setmainlanguage{english}
\setotherlanguages{sanskrit,tamil,gujarati}
%
\setmainfont[Color=teal]{FreeSerif}
\newfontfamily{\englishfont}[Color=blue]{FreeSerif}
\newfontfamily{\latinfont}[Scale=1, Color=brown]{FreeSerif}
\newfontfamily{\devanagarifont}[Scale=1,Color=purple]{Siddhanta}
\newfontfamily{\tamilfont}[Script=Tamil]{Lohit Tamil Classical}
\newfontfamily{\gujaratifont}[Path=../fonts/,Scale=1,Script=Gujarati]{Rasa-Regular.ttf}
\newfontfamily{\vedafont}[Scale=1,Color=red]{Siddhanta}
%
\usepackage[Latin, Devanagari, DevanagariExtended, Gujarati, Tamil, VedicExtensions]{ucharclasses}
\setTransitionsForLatin{\latinfont}{}
\setTransitionTo{DevanagariExtended}{\vedafont}
\setTransitionTo{VedicExtensions}{\vedafont}
\setTransitionTo{Devanagari}{\devanagarifont}
\setTransitionTo{Gujarati}{\gujaratifont}
\setTransitionTo{Tamil}{\tamilfont}
\setlength{\parindent}{0pt}
%
\begin{document}
\obeylines
This trial is to use the combining marks and characters from Devanagari, DevanagariExtended and VedicExtensions unicode blocks with text in other Indic scripts, eg. Gujarati, Tamil, etc.
\medskip
"The Devanagari Extended block, as well as the Vedic Extensions block and the Devanagari block, include characters that are used to indicate tone in Vedic Sanskrit. "
"This block of characters is used chiefly for Vedic Sanskrit, although many of the characters are generic and can be used by other Indic scripts. "
\medskip
Source: http://www.unicode.org/versions/Unicode9.0.0/ch12.pdf
\medskip
Sample text for Devanagari
\hrule
ठ॑ શ્રી॑ ગ॑ த॑ 0951 DEVANAGARI STRESS SIGN UDATTA
ठ॒ શ્રી॒ ગ॒ த॒ 0952 DEVANAGARI STRESS SIGN ANUDATTA
\medskip
Sample text for DevanagariExtended
\hrule
ठ꣱ શ્રી꣱ ગ꣱ த꣱ A8F1 COMBINING DEVANAGARI SIGN AVAGRAHA
ठꣳ શ્રીꣳ ગꣳ தꣳ A8F3 DEVANAGARI SIGN CANDRABINDU VIRAMA
\medskip
Sample text for VedicExtensions
\hrule
ठ᳚ શ્રી᳚ ગ᳚ த᳚ 1CDA VEDIC TONE DOUBLE SVARITA
\medskip
\end{document}
\documentclass[a4paper,11pt,twoside,onecolumn,openany,final]{memoir}
\usepackage[a4paper,hmargin=0.7in,vmargin=0.7in]{geometry}
\usepackage{multicol,polyglossia,etoolbox,fontspec}
\usepackage{xcolor}
%
\XeTeXgenerateactualtext=1
%
\defaultfontfeatures{Mapping=tex-text}
\setmainlanguage{english}
\setotherlanguages{sanskrit,tamil,gujarati}
%
\setmainfont[Color=teal]{FreeSerif}
\newfontfamily{\englishfont}[Color=blue]{FreeSerif}
\newfontfamily{\latinfont}[Scale=1, Color=brown]{FreeSerif}
\newfontfamily{\devanagarifont}[Scale=1,Color=magenta, Script=Devanagari]{Siddhanta}
%\newfontfamily{\devanagarifont}[Scale=1,Color=purple]{Siddhanta}
\newfontfamily{\tamilfont}[Script=Tamil]{Lohit Tamil Classical}
\newfontfamily{\gujaratifont}[Path=../fonts/,Scale=1,Script=Gujarati]{Rasa-Regular.ttf}
\newfontfamily{\vedafont}[Scale=1,Color=red]{Siddhanta}
%
\usepackage[Latin, Devanagari, DevanagariExtended, Gujarati, Tamil, VedicExtensions]{ucharclasses}
\setTransitionsForLatin{\latinfont}{}
\setTransitionTo{DevanagariExtended}{\vedafont}
\setTransitionTo{VedicExtensions}{\vedafont}
\setTransitionTo{Devanagari}{\devanagarifont}
\setTransitionTo{Gujarati}{\gujaratifont}
\setTransitionTo{Tamil}{\tamilfont}
\setlength{\parindent}{0pt}
%
\begin{document}
\obeylines
This trial is to use the combining marks and characters from Devanagari, DevanagariExtended and VedicExtensions unicode blocks with text in other Indic scripts, eg. Gujarati, Tamil, etc.
\medskip
"The Devanagari Extended block, as well as the Vedic Extensions block and the Devanagari block, include characters that are used to indicate tone in Vedic Sanskrit. "
"This block of characters is used chiefly for Vedic Sanskrit, although many of the characters are generic and can be used by other Indic scripts. "
\medskip
Source: http://www.unicode.org/versions/Unicode9.0.0/ch12.pdf
\medskip
Sample text for Devanagari
\hrule
ठ॑ શ્રી॑ ગ॑ த॑ 0951 DEVANAGARI STRESS SIGN UDATTA
ठ॒ શ્રી॒ ગ॒ த॒ 0952 DEVANAGARI STRESS SIGN ANUDATTA
\medskip
Sample text for DevanagariExtended
\hrule
ठ꣱ શ્રી꣱ ગ꣱ த꣱ A8F1 COMBINING DEVANAGARI SIGN AVAGRAHA
ठꣳ શ્રીꣳ ગꣳ தꣳ A8F3 DEVANAGARI SIGN CANDRABINDU VIRAMA
\medskip
Sample text for VedicExtensions
\hrule
ठ᳚ શ્રી᳚ ગ᳚ த᳚ 1CDA VEDIC TONE DOUBLE SVARITA
\medskip
\end{document}
Questions:
Is it possible to create a informal unicode range for the above in Devanagari range along with DevanagariExtended and VedicExtensions - so as to apply the rules for combining marks to these.
Alternately, is it possible to define just these 4 codepoints as a separate custom range?
Thanks!
It sounds like the package might need some knowledge on what is "script" and what is script-agnostic diacritics, so that diacritics don't insert block change commands, which would absolutely break up the combining of diacritic with base glyph(s).
http://unicode.org/charts/PDF/U0900.pdf
Vedic tone marks
0951 $॑ DEVANAGARI STRESS SIGN UDATTA
= Vedic tone svarita
• mostly used for svarita, with rare use for udatta
• used also in Vedic texts written in other scripts
0952 $॒ DEVANAGARI STRESS SIGN ANUDATTA
= Vedic tone anudatta
• used also in Vedic texts written in other scripts
ShreeDevi
भजन - कीर्तन - आरती @ http://bhajans.ramparivar.com
On Thu, Aug 11, 2016 at 11:18 PM, Mike Kamermans notifications@github.com wrote:
It sounds like the package might need some knowledge on what is "script" and what is script-agnostic diacritics, so that diacritics don't insert block change commands, which would absolutely break up the combining of diacritic with base glyph(s).
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Pomax/ucharclasses/issues/21#issuecomment-239236921, or mute the thread https://github.com/notifications/unsubscribe-auth/AE2_oxyy3Dy_25Sh_C02cy5mKwJOjBAaks5qe2BagaJpZM4Jd_sX .
It seems that this issue is font related. When I used a different devanagari font then both the combining marks with different script and regular devanagari script worked. I'll post the tex source and resultant pdf later.
I am guessing - it could be related to new features added in fontspec for texlive2016 - http://ctan.imsc.res.in/macros/latex/contrib/fontspec/README.md
Incorporate "new" font script tags for Indic fonts. E.g., when selecting
Script=Bengali
, fontspec will first query the font for thebng2
OpenType script, and if not found selectbeng
.
testuchar.tex.txt testuchar.log.txt testuchar.pdf
tex source, resulting pdf and log file are attached, showing that devanagari is being rendered correctly when using a different font .
\documentclass[a4paper,11pt,twoside,onecolumn,openany,final]{memoir}
\usepackage[a4paper,hmargin=0.7in,vmargin=0.7in]{geometry}
\usepackage{multicol,polyglossia,etoolbox,fontspec}
\usepackage{xcolor}
\XeTeXgenerateactualtext=1
%
\defaultfontfeatures{Mapping=tex-text}
\setmainlanguage{english}
\setotherlanguages{sanskrit,tamil,gujarati}
%
\setmainfont[Color=teal]{FreeSerif}
\newfontfamily{\englishfont}[Color=blue]{FreeSerif}
\newfontfamily{\latinfont}[Scale=1]{FreeSerif}
%\newfontfamily{\devanagarifont}[Scale=1,Color=magenta, Script=Devanagari]{Siddhanta}
%\newfontfamily{\devanagarifont}[Scale=1.5,Color=purple]{Siddhanta}
\newfontfamily{\devanagarifont}[Scale=1,Color=purple]{Lohit Devanagari}
\newfontfamily{\tamilfont}[Script=Tamil,Scale=1.2,Color=brown]{Lohit Tamil Classical}
\newfontfamily{\gujaratifont}[Scale=1,Script=Gujarati,Color=gray]{Lohit Gujarati}
\newfontfamily{\vedafont}[Scale=1.2,Color=red]{Siddhanta}
%
\usepackage[Latin, GeneralPunctuation, Devanagari, DevanagariExtended, Gujarati, Tamil, VedicExtensions, SuperscriptsAndSubscripts]{ucharclasses}
\setTransitionsForLatin{\latinfont}{}
\setTransitionTo{GeneralPunctuation}{\englishfont}
\setTransitionTo{SuperscriptsAndSubscripts}{\englishfont}
\setTransitionTo{DevanagariExtended}{\vedafont}
\setTransitionTo{VedicExtensions}{\vedafont}
\setTransitionTo{Devanagari}{\devanagarifont}
\setTransitionTo{Gujarati}{\gujaratifont}
\setTransitionTo{Tamil}{\tamilfont}
%
\setlength{\parindent}{0pt}
%
\begin{document}
\obeylines
This trial is to use the combining marks and characters from Devanagari, DevanagariExtended and VedicExtensions unicode blocks with text in other Indic scripts, eg. Gujarati, Tamil, etc.
\medskip
This sample also has superscripts for 2, 3 and 4 used with Tamil
\medskip
"The Devanagari Extended block, as well as the Vedic Extensions block and the Devanagari block, include characters that are used to indicate tone in Vedic Sanskrit. "
"This block of characters is used chiefly for Vedic Sanskrit, although many of the characters are generic and can be used by other Indic scripts. "
\medskip
Source: http://www.unicode.org/versions/Unicode9.0.0/ch12.pdf
\medskip
Sample combining mark from Devanagari
\hrule
ठ॑ શ્રી॑ ગ॑ த॑ 0951 DEVANAGARI STRESS SIGN UDATTA with Devanagari, Gujarati, Gujarati, Tamil Scripts
ठ॒ શ્રી॒ ગ॒ த॒ 0952 DEVANAGARI STRESS SIGN ANUDATTA with Devanagari, Gujarati, Gujarati, Tamil Scripts
\medskip
Sample combining mark from DevanagariExtended
\hrule
ठ꣱ શ્રી꣱ ગ꣱ த꣱ A8F1 COMBINING DEVANAGARI SIGN AVAGRAHA with Devanagari, Gujarati, Gujarati, Tamil Scripts
ठꣳ શ્રીꣳ ગꣳ தꣳ A8F3 DEVANAGARI SIGN CANDRABINDU VIRAMA with Devanagari, Gujarati, Gujarati, Tamil Scripts
\medskip
Sample combining mark from VedicExtensions
\hrule
ठ᳚ શ્રી᳚ ગ᳚ த᳚ 1CDA VEDIC TONE DOUBLE SVARITA with Devanagari, Gujarati, Gujarati, Tamil Scripts
\medskip
Tamil text ( with superscripts using FreeSerif as Latinfont and Englishfont)
Needed to add GeneralPunctuation unicode block also to cater to invisible space/combining marks.
Otherwise the font change was not happening correctly.
\hrule
ௐ கணாநாம் த்வா கணபதி
ௐ க³ணாநாம் த்வா க³ணபதி
ஜ்யேஷ்ட²ராஜம் ப்³ரஹ்மணாம் ப்³
\medskip
Tamil with accents and superscripts
\hrule
க³॒ம்த⁴॒த்³வாராம் து³ராத⁴॒ர்ஷாம்॒ நி॒த்யபுஷ்டாம் கரீ॒ஷிணீ᳚ம் ।
ஈ॒ஶ்வரீ॑ ꣳ ஸர்வபூ⁴தா॒நாம்॒ தாமி॒ஹோப॑ஹ்வயே॒ ஶ்ரியம் ॥ 9॥
\medskip
\hrule
devanagari is not correct with Siddhanta and FreeSerif font (without Script=Devanagari)
but is ok with Lohit Devanagari, Aparajita, Arial Unicode
\medskip
गं॒ध॒द्वा॒रां दु॑राध॒र्षां॒ नि॒त्यपु॑ष्टां करी॒षिणी᳚म् ।
ई॒श्वरी॑ꣳ सर्व॑भूता॒नां॒ तामि॒होप॑ह्वये॒ श्रियम् ॥ ९॥
\end{document}
hm, given that information, I'm not sure what the best course of action is here... it kind of feels like "using a modern font" solves the issue, in which case this is no longer a ucharclasses-specific issue? How do you feel about that?
I have called the ranges as DevanagariPreMarks, DevanagariMarks and DevanagariPostMarks.
Possibly these could be combined together to form an informal Devanagari group.
Thanks.
Here is a sample, which uses https://github.com/Shreeshrii/ucharclasses/blob/master/ucharclasses.sty
\documentclass[a5paper,11pt,twoside,onecolumn,openany,final]{memoir}
\usepackage[a5paper,hmargin=0.7in,vmargin=0.7in]{geometry}
\usepackage{multicol,polyglossia,etoolbox,fontspec}
\usepackage{xcolor}
%
\XeTeXgenerateactualtext=1
%
\defaultfontfeatures{Mapping=tex-text}
\setmainlanguage{english}
\setotherlanguages{sanskrit,tamil,gujarati}
%
\setmainfont[Color=teal]{FreeSerif}
\newfontfamily{\latinfont}[Scale=1]{FreeSerif}
\newfontfamily{\devanagarifont}[Scale=1,Color=blue]{Lohit Devanagari}
\newfontfamily{\tamilfont}[Script=Tamil,Scale=1.2,Color=brown]{Lohit Tamil Classical}
\newfontfamily{\gujaratifont}[Scale=1,Script=Gujarati,Color=gray]{Lohit Gujarati}
\newfontfamily{\vedafont}[Scale=1.2,Color=red]{Siddhanta}
%
\usepackage[Latin, GeneralPunctuation, DevanagariMarks, DevanagariPreMarks, DevanagariExtended, DevanagariPostMarks, Gujarati, Tamil, VedicExtensions, SuperscriptsAndSubscripts]{ucharclasses}
\setTransitionsForLatin{\latinfont}{}
\setTransitionTo{GeneralPunctuation}{\latinfont}
\setTransitionTo{SuperscriptsAndSubscripts}{\latinfont}
\setTransitionTo{DevanagariMarks}{\vedafont}
\setTransitionTo{DevanagariExtended}{\vedafont}
\setTransitionTo{VedicExtensions}{\vedafont}
\setTransitionTo{DevanagariPreMarks}{\devanagarifont}
\setTransitionTo{DevanagariPostMarks}{\devanagarifont}
\setTransitionTo{Gujarati}{\gujaratifont}
\setTransitionTo{Tamil}{\tamilfont}
%
\setlength{\parindent}{0pt}
%
\begin{document}
This document shows the use of vedic accents / combining marks and characters from Devanagari, DevanagariExtended and VedicExtensions unicode blocks with text in other Indic scripts, eg. Gujarati, Tamil, etc.
\medskip
\obeylines
Devanagari with vedic accents
\hrule
ओमिति॒ ब्रह्म॑ । ओमिती॒दꣳसर्वम्᳚ ।
\medskip
Gujarati with vedic accents
\hrule
ઓમિતિ॒ બ્રહ્મ॑ । ઓમિતી॒દꣳસર્વમ્᳚ ।
\medskip
Tamil with vedic accents and superscripts
\hrule
ஓமிதி॒ ப்³ரஹ்ம॑ । ஓமிதீ॒த³ꣳஸர்வம்᳚ ।
%
\end{document}
And, here is the resulting pdf ...
Fonts used are:
Siddhanta - http://svayambhava.blogspot.in/p/siddhanta-devanagariunicode-open-type.html
Lohit - https://fedorahosted.org/lohit/
could you turn that into a PR?
I have created a PR after adding two groups - Devanagari and VedicMarks.
Here is a sample which uses these informal groups.
\documentclass[a5paper,11pt,twoside,onecolumn,openany,final]{memoir}
\usepackage[a5paper,hmargin=0.7in,vmargin=0.7in]{geometry}
\usepackage{multicol,polyglossia,etoolbox,fontspec}
\usepackage{xcolor,metalogo}
%
\XeTeXgenerateactualtext=1
%
\defaultfontfeatures{Mapping=tex-text}
\setmainlanguage{english}
\setotherlanguages{sanskrit,tamil,gujarati}
%
\setmainfont[Color=teal]{FreeSerif}
\newfontfamily{\latinfont}[Scale=1]{FreeSerif}
\newfontfamily{\devanagarifont}[Scale=1,Color=blue]{Lohit Devanagari}
\newfontfamily{\tamilfont}[Script=Tamil,Scale=1.2,Color=brown]{Lohit Tamil Classical}
\newfontfamily{\gujaratifont}[Scale=1,Script=Gujarati,Color=gray]{Lohit Gujarati}
\newfontfamily{\vedafont}[Scale=1.2,Color=red]{Siddhanta}
%
\usepackage[Latin, GeneralPunctuation, Devanagari, Gujarati, Tamil, SuperscriptsAndSubscripts, VedicMarks]{ucharclasses}
\setTransitionsForLatin{\latinfont}{}
\setTransitionsForDevanagari{\devanagarifont}{}
\setTransitionsForVedicMarks{\vedafont}{\devanagarifont}
\setTransitionTo{GeneralPunctuation}{\latinfont}
\setTransitionTo{SuperscriptsAndSubscripts}{\latinfont}
\setTransitionTo{Gujarati}{\gujaratifont}
\setTransitionTo{Tamil}{\tamilfont}
%
\setlength{\parindent}{0pt}
%
\begin{document}
\obeylines
This document uses the combining marks and characters from Devanagari, DevanagariExtended and VedicExtensions unicode blocks with text in other Indic scripts, eg. Gujarati and Tamil.
\medskip
Devanagari with vedic accents
\hrule
ओमिति॒ ब्रह्म॑ । ओमिती॒दꣳसर्वम्᳚ ।
\medskip
Gujarati with vedic accents
\hrule
ઓમિતિ॒ બ્રહ્મ॑ । ઓમિતી॒દꣳસર્વમ્᳚ ।
\medskip
Tamil with vedic accents and superscripts
\hrule
ஓமிதி॒ ப்³ரஹ்ம॑ । ஓமிதீ॒த³ꣳஸர்வம்᳚ ।
\medskip
\hrule
\medskip
This was typeset using \XeLaTeX \ \the\XeTeXversion\XeTeXrevision
and the package ucharclasses for automatic font switching based on uniocde ranges.
\end{document}
Resulting pdf is attached below:
I am trying to use the combining marks and characters from Devanagari, DevanagariExtended and VedicExtensions unicode blocks with text in other Indic scripts, eg. Gujarati.
ucharclasses is working fine with xetex (texlive2016) to display the combining marks and characters with another script. however text in Devanagari script is not getting rendered correctly.
An MWE is posted below.
\textbackslash mainfont (Rasa-Regular) is in Gujarati script and also supports Latin range. \textbackslash vedafont (Siddhanta) supports Devanagari, DevanagariExtended, VedicExtensions and Latin ranges and is defined without specifying a script. \textbackslash devafont is the same font as \textbackslash vedafont but also specifies script=devanagari as part of definition.
\textbackslash ucharclasses with \textbackslash SetTransitionsFor for Devanagari, DevanagariExtended, VedicExtensions combining marks works well with the default Gujarati script. But text in Devnagari script does not render correctly, since the script is NOT specified for \textbackslash vedafont.
Using \textbackslash devafont for transition to devanagari renders the devanagari text correctly but the combining marks break and are displayed with broken circles.