search

bayaraa / d2gl

Diablo 2 LoD Glide/DDraw to OpenGL Wrapper
GNU General Public License v3.0
183 stars 30 forks source link

font rendering issue (Custom Font) #9

Closed CaiMiao closed 1 year ago

CaiMiao commented 1 year ago

https://github.com/bayaraa/d2gl/blob/d05ea289b6602738c704fa5c536b20bc08c044f2/d2gl/src/font.cpp#L72

CaiMiao commented 1 year ago

fyi data.csv atlas.png

tool: https://github.com/Chlumsky/msdf-atlas-gen range:

[0x00,0xff],[0x2000,0x206f],[0x3000,0x312f],[0x4e00,0x9fff],[0xff00,0xffef]

command (64em):

msdf-atlas-gen.exe -font arfangxinshuh7c95b5_eb_t.ttf -charset ss.txt -size 64 -fontname test -format png -dimensions 860 860 -imageout atlas.png -json json.json -csv data.csv

output (stdout):

Error: Could not fit 21212 out of 21454 glyphs into the atlas.
CaiMiao commented 1 year ago

i sincirely suggest implementing realtime atlas compiler to avoid extreme large textures which might over 5k^2~10k^2 pixels...

bayaraa commented 1 year ago

Yeah, I used that tool to generate current atlases. It can be generated on fly but it was extremely slow even using multiple cores specially on older cpus. Thats why i use pre generated atlases. Currently not planned to support other languages, maybe in the future. Btw I wonder how much glyphes required in japanese d2. I don't think that all 21454 required.

CaiMiao commented 1 year ago

Yeah, I used that tool to generate current atlases. It can be generated on fly but it was extremely slow even using multiple cores specially on older cpus. Thats why i use pre generated atlases. Currently not planned to support other languages, maybe in the future. Btw I wonder how much glyphes required in japanese d2. I don't think that all 21454 required.

you can't just ignore if mod creators decide to use some other ideographs... I think you can generate lower res texture on demand (when certain text display is called) in another thread.

edit: i think you can collect d2 text tbl (the final one loaded by the game) as charset reference as well

weili0677 commented 1 year ago

Yeah, I used that tool to generate current atlases. It can be generated on fly but it was extremely slow even using multiple cores specially on older cpus. Thats why i use pre generated atlases. Currently not planned to support other languages, maybe in the future. Btw I wonder how much glyphes required in japanese d2. I don't think that all 21454 required.

Hi,Mr.bayaraa which command you used to generated atlaes with ExocetBlizzardMedium.otf ?

bayaraa commented 1 year ago

@weili0677 

msdf-atlas-gen.exe -font fonts/exocetblizzardot-medium.otf -type msdf -format png -imageout atlas.png -csv data.csv -charset charset.txt -size 64 -minsize 64 -dimensions 860 860 -pxrange 20
msdf-atlas-gen.exe -font fonts/Formal436BT.ttf -type msdf -format png -imageout atlas.png -csv data.csv -charset charset.txt -size 64 -minsize 64 -dimensions 860 860 -pxrange 20
msdf-atlas-gen.exe -font fonts/exocet-reaper-medium.ttf -type msdf -format png -imageout atlas.png -csv data.csv -charset charset.txt -size 64 -minsize 64 -dimensions 860 860 -pxrange 20

this is commands i used. inside charset.txt [0, 255].

weili0677 commented 1 year ago

@weili0677

msdf-atlas-gen.exe -font fonts/exocetblizzardot-medium.otf -type msdf -format png -imageout atlas.png -csv data.csv -charset charset.txt -size 64 -minsize 64 -dimensions 860 860 -pxrange 20
msdf-atlas-gen.exe -font fonts/Formal436BT.ttf -type msdf -format png -imageout atlas.png -csv data.csv -charset charset.txt -size 64 -minsize 64 -dimensions 860 860 -pxrange 20
msdf-atlas-gen.exe -font fonts/exocet-reaper-medium.ttf -type msdf -format png -imageout atlas.png -csv data.csv -charset charset.txt -size 64 -minsize 64 -dimensions 860 860 -pxrange 20

this is commands i used. inside charset.txt [0, 255].

Thank you.you are very nice.

dabeibao commented 1 year ago

Yeah, I used that tool to generate current atlases. It can be generated on fly but it was extremely slow even using multiple cores specially on older cpus. Thats why i use pre generated atlases. Currently not planned to support other languages, maybe in the future. Btw I wonder how much glyphes required in japanese d2. I don't think that all 21454 required.

Is it possible to generate the atlases on fly and cache them? Not sure how many glyphes will be used in Japanese; but in Chinese the frequently used characters are about 3000~5000 only.

CaiMiao commented 1 year ago

Is it possible to generate the atlases on fly and cache them? Not sure how many glyphes will be used in Japanese; but in Chinese the frequently used characters are about 3000~5000 only.

I will say, forget about atlases and directly use freetype will be the best solution, but generate & cache on startup will be neat enough, though.

I have an idea to verify cache and do not generate them frequently - hashing. Either by collect the final language tbl loaded by game, sort unique characters as charset.txt (for later generation use), and hash it - or, just hash digest of binary data of all language tbl together (both equivalent so i prefer former).

bayaraa commented 1 year ago

@CaiMiao @weili0677 @dabeibao hey guys where can i find chinese or japanese version of diablo 2?

CaiMiao commented 1 year ago

@CaiMiao @weili0677 @dabeibao hey guys where can i find chinese or japanese version of diablo 2?

You can download Traditional Chinese directly in digital version on BNet, or you can find all those assets in Resurrected CASC archive. Keep in mind that, most Chinese players use customized (modified) tbls so only including glyphs used officially may not satisfy Chinese audiences.

For Japanese version, due to licensing issue with Capcom probably, Blizz did not include them as Digital nor in Resurrected. (They reworked Japanese localization for Resurrected) I'm uploading these assets which is extracted from Japanese physical release, right after installing LoD. CAPCOM.zip

CaiMiao commented 1 year ago

P.S. You have to create a Use file in /data/local includes the corresponding byte which you can find via below link, if you are using -direct to switch language. Or you can just use that tool if you don't mind changing MPQ permanently. https://d2mods.info/forum/viewtopic.php?f=7&t=60930

bayaraa commented 1 year ago

thanks.

weili0677 commented 1 year ago

this is for 1.13c https://drive.google.com/file/d/1jZCAJRu77BgFQs8TXC7TtjCnnOSMC7Rt/view?usp=sharing

this is for 1.14d https://drive.google.com/file/d/17boLX3Io3tUdXiHWBquvjOydtAtolM5m/view?usp=sharing

usage： create ProjectD2/game.exe shortcut,and append params: -3dfx -direct

@CaiMiao @weili0677 @dabeibao hey guys where can i find chinese or japanese version of diablo 2?

bayaraa commented 1 year ago

@weili0677 @CaiMiao @dabeibao hello all! finally i am working other language support. if it's not much trouble, could you translate these text to chinese (if possible japanese also)?

exit.dc6: "SAVE AND EXIT GAME"
returntogame.dc6: "RETURN TO GAME"
previous.dc6: "PREVIOUS MENU"
sprevious.dc6: "PREVIOUS MENU"
smallon.dc6: "ON"
smalloff.dc6: "OFF"
smallyes.dc6: "YES"
smallno.dc6: "NO"
soundoptions.dc6: "SOUND OPTIONS"
sound.dc6: "SOUND"
music.dc6: "MUSIC"
3dsound.dc6: "3D SOUND"
eax.dc6: "ENVIRONMENTAL EFFECTS"
3dbias.dc6: "3D BIAS"
npcspeech.dc6: "NPC SPEECH"
audiotext.dc6: "AUDIO AND TEXT"
audioonly.dc6: "AUDIO ONLY"
textonly.dc6: "TEXT ONLY"
videooptions.dc6: "VIDEO OPTIONS"
resolution.dc6: "RESOLUTION"
lightquality.dc6: "LIGHTING QUALITY"
high.dc6: "HIGH"
medium.dc6: "MEDIUM"
low.dc6: "LOW"
blendshadow.dc6: "BLENDED SHADOWS"
perspective.dc6: "PERSPECTIVE"
gamma.dc6: "GAMMA"
contrast.dc6: "CONTRAST"
automapoptions.dc6: "AUTOMAP OPTIONS"
automapmode.dc6: "AUTOMAP SIZE"
full.dc6: "FULL SCREEN"
mini.dc6: "MINI MAP"
automapfade.dc6: "FADE"
everything.dc6: "EVERYTHING"
center.dc6: "CENTER"
auto.dc6: "AUTO"
automapcenter.dc6: "CENTER WHEN CLEARED"
automapparty.dc6: "SHOW PARTY"
automappartynames.dc6: "SHOW NAMES"
cfgoptions.dc6: "CONFIGURE CONTROLS"
weili0677 commented 1 year ago

@weili0677 @CaiMiao @dabeibao hello all! finally i am working other language support. if it's not much trouble, could you translate these text to chinese (if possible japanese also)?

exit.dc6: "SAVE AND EXIT GAME"
returntogame.dc6: "RETURN TO GAME"
previous.dc6: "PREVIOUS MENU"
sprevious.dc6: "PREVIOUS MENU"
smallon.dc6: "ON"
smalloff.dc6: "OFF"
smallyes.dc6: "YES"
smallno.dc6: "NO"
soundoptions.dc6: "SOUND OPTIONS"
sound.dc6: "SOUND"
music.dc6: "MUSIC"
3dsound.dc6: "3D SOUND"
eax.dc6: "ENVIRONMENTAL EFFECTS"
3dbias.dc6: "3D BIAS"
npcspeech.dc6: "NPC SPEECH"
audiotext.dc6: "AUDIO AND TEXT"
audioonly.dc6: "AUDIO ONLY"
textonly.dc6: "TEXT ONLY"
videooptions.dc6: "VIDEO OPTIONS"
resolution.dc6: "RESOLUTION"
lightquality.dc6: "LIGHTING QUALITY"
high.dc6: "HIGH"
medium.dc6: "MEDIUM"
low.dc6: "LOW"
blendshadow.dc6: "BLENDED SHADOWS"
perspective.dc6: "PERSPECTIVE"
gamma.dc6: "GAMMA"
contrast.dc6: "CONTRAST"
automapoptions.dc6: "AUTOMAP OPTIONS"
automapmode.dc6: "AUTOMAP SIZE"
full.dc6: "FULL SCREEN"
mini.dc6: "MINI MAP"
automapfade.dc6: "FADE"
everything.dc6: "EVERYTHING"
center.dc6: "CENTER"
auto.dc6: "AUTO"
automapcenter.dc6: "CENTER WHEN CLEARED"
automapparty.dc6: "SHOW PARTY"
automappartynames.dc6: "SHOW NAMES"
cfgoptions.dc6: "CONFIGURE CONTROLS"
exit.dc6: "储存并退出游戏"
returntogame.dc6: "返回游戏"
previous.dc6: "前一菜单"
sprevious.dc6: "前一菜单"
smallon.dc6: "开"
smalloff.dc6: "关"
smallyes.dc6: "是"
smallno.dc6: "否"
soundoptions.dc6: "声音选项"
sound.dc6: "声音"
music.dc6: "音乐"
3dsound.dc6: "3D音效"
eax.dc6: "环场效果"
3dbias.dc6: "3D偏向"
npcspeech.dc6: "NPC语音"
audiotext.dc6: "语音和文字"
audioonly.dc6: "仅有语音"
textonly.dc6: "仅有文字"
videooptions.dc6: "影像选项"
resolution.dc6: "分辨率"
lightquality.dc6: "打光品质"
high.dc6: "高"
medium.dc6: "中"
low.dc6: "低"
blendshadow.dc6: "混合影子"
perspective.dc6: "透视"
gamma.dc6: "亮度"
contrast.dc6: "对比度"
automapoptions.dc6: "自动地图选项"
automapmode.dc6: "自动地图大小"
full.dc6: "全屏"
mini.dc6: "小型地图"
automapfade.dc6: "淡化"
everything.dc6: "所有"
center.dc6: "中间"
auto.dc6: "自动"
automapcenter.dc6: "清除时置中"
automapparty.dc6: "显示队伍"
automappartynames.dc6: "显示名称"
cfgoptions.dc6: "设定控制"

This is “Simplified Chinese”

bayaraa commented 1 year ago

@weili0677 thanks! sorry forgot these 3 🙏 

youdiedhardcore.dc6: "You have died"
youdiedinst.dc6: "Press ESC to continue."
youdiedsoftcore.dc6: "You lost gold"
weili0677 commented 1 year ago

@weili0677 thanks! sorry forgot these 3 🙏

youdiedhardcore.dc6: "You have died"
youdiedinst.dc6: "Press ESC to continue."
youdiedsoftcore.dc6: "You lost gold"
youdiedhardcore.dc6: "你已经死亡"
youdiedinst.dc6: "按ESC继续"
youdiedsoftcore.dc6: "你丢失了一些金币"
CaiMiao commented 1 year ago

iirc you can also steal the official translation from d2r strings

CaiMiao commented 1 year ago

ui.zip this is locale json of ui elements extracted from D2R. for Chinese there's zhCN for simplified variant and zhTW for traditional variant. youdiedsoftcore.dc6 is never used in retail game.

bayaraa commented 1 year ago

Thanks! What fonts are used in chinese/japanese on D2R?

CaiMiao commented 1 year ago

they can also extract from casc archive, but i dont suggest using any of them as they, including Exocet, are copyrighted software.

chinese / japanese font companies care about their font and they probably will warn you if they know you distributing font ilegally

From: Bayaraa @.> Sent: Tuesday, July 4, 2023 8:11:54 AM To: bayaraa/d2gl @.> Cc: Fumii Chuu @.>; Mention @.> Subject: Re: [bayaraa/d2gl] font rendering issue (Custom Font) (Issue #9)

Thanks! What fonts are used in chinese/japanese on D2R?

— Reply to this email directly, view it on GitHubhttps://github.com/bayaraa/d2gl/issues/9#issuecomment-1619281889, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABDM5Y5BSWBWNIVB34OLVZDXONNUVANCNFSM6AAAAAAVPGF4SE. You are receiving this because you were mentioned.Message ID: @.***>

weili0677 commented 1 year ago

Yes.@bayaraa so you can let the fonts files(.ttf or .otf) custom by myself.

they can also extract from casc archive, but i dont suggest using any of them as they, including Exocet, are copyrighted software. chinese / japanese font companies care about their font and they probably will warn you if they know you distributing font ilegally … ____ From: Bayaraa @.> Sent: Tuesday, July 4, 2023 8:11:54 AM To: bayaraa/d2gl @.> Cc: Fumii Chuu @.>; Mention @.> Subject: Re: [bayaraa/d2gl] font rendering issue (Custom Font) (Issue #9) Thanks! What fonts are used in chinese/japanese on D2R? — Reply to this email directly, view it on GitHub<#9 (comment)>, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABDM5Y5BSWBWNIVB34OLVZDXONNUVANCNFSM6AAAAAAVPGF4SE. You are receiving this because you were mentioned.Message ID: @.***>

bayaraa commented 1 year ago

Finally release multi language support! Japanese font included 1986 glyphes. Chinese font included whopping 3508 glyphes. Hope it's enough :D

CaiMiao commented 1 year ago

Finally release multi language support! Japanese font included 1986 glyphes. Chinese font included whopping 3508 glyphes. Hope it's enough :D

definitely it wont be enough... and currently D2GL just skip rendering glyph when certain glyph does not exists. i hope you can implement a fallback: render a ? (question mark) for missing glyphs to indicate there's something missing.

for example, below image shows missing 菜 glyph from here got skipped under official traditional chinese version 图片

bayaraa commented 1 year ago

this is embarrasing. I forgot to include option text glypes in charset list it seems.

CaiMiao commented 1 year ago

this is embarrasing. I forgot to include option text glypes in charset list.

i'm making another charlist which should include all glyph appears in D2R (incl. legacy) language json and they are indeed simplified chinese (currently in variable.h)

bayaraa commented 1 year ago

good idea. will add failsafe character display.

bayaraa commented 1 year ago

https://github.com/bayaraa/d2gl/issues/9?notification_referrer_id=NT_kwDOAB8v6LI1NzY0NTk0ODMzOjIwNDM4ODA#issuecomment-1616596326 copied from here :D

CaiMiao commented 1 year ago

chinese-d2r.txt Here you are.

This includes 3708 CJK ideographs which theoretically includes all Chinese characters that have appeared in D2R (HD and Legacy). Does not include ideographs beyond this, so Mod creators/translators should make their own font.

I considered uploading my generic font which includes

but it is too large for mass distribution (~150MB includes TC and SC variant).

Instead I'm uploading that char list for anyone who needs it chinese.txt

bayaraa commented 1 year ago

Thanks for charsets!

chinese-d2r.txt Here you are.

sorry can i have only chinese version. not included japanese or korean letters. i want them separately in d2gl.

I considered uploading my generic font which includes full GB2312-80 characters BIG5 level 1 characters SJIS level 1 characters total of 9240 CJK ideographs but it is too large for mass distribution (~150MB includes TC and SC variant). Instead I'm uploading that char list for anyone who needs it chinese.txt

absolutely d2gl will contain vanilla d2lod charsets only. modders should create custom atlases based on their needs!

CaiMiao commented 1 year ago

sorry can i have only chinese version. not included japanese or korean letters.

chinese-d2r.txt is chinese only

CaiMiao commented 1 year ago

also notice that you should use non-subset TC font from here and here, to avoid missing glyph issue.

bayaraa commented 1 year ago

chinese-d2r.txt is chinese only

sorry korean letters not included. but it has japanese hiragana, katakana. ぁあぃいぅうぇえぉおかがきぎくぐけげこごさざしじすずせぜそぞただちぢっつづてでとどなにぬねのはばぱひびぴふぶぷへべぺほぼぽまみむめもゃやゅゆょよらりるれろゎわゐゑをん゛゜ゝゞァアィイゥウェエォオカガキギクグケゲコゴサザシジスズセゼソゾタダチヂッツヅテデトドナニヌネノハバパヒビピフブプヘベペホボポマミムメモャヤュユョヨラリルレロヮワヰヱヲンヴヵヶ

also feels like these characters wont need chinese d2? ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ

CaiMiao commented 1 year ago

chinese-d2r.txt is chinese only

sorry korean letters not included. but it has japanese hiragana, katakana. ぁあぃいぅうぇえぉおかがきぎくぐけげこごさざしじすずせぜそぞただちぢっつづてでとどなにぬねのはばぱひびぴふぶぷへべぺほぼぽまみむめもゃやゅゆょよらりるれろゎわゐゑをん゛゜ゝゞァアィイゥウェエォオカガキギクグケゲコゴサザシジスズセゼソゾタダチヂッツヅテデトドナニヌネノハバパヒビピフブプヘベペホボポマミムメモャヤュユョヨラリルレロヮワヰヱヲンヴヵヶ

also feels like these characters wont need chinese d2? ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ

Latin font includes 0080-00FF so I include them as basic failsafe. hiragana and katakana lives in standard GB2312-80/BIG5 however, they are not taking too much spaces and for failsafe purpose as well.

weili0677 commented 1 year ago

chinese-d2r.txt Here you are.

This includes 3708 CJK ideographs which theoretically includes all Chinese characters that have appeared in D2R (HD and Legacy). Does not include ideographs beyond this, so Mod creators/translators should make their own font.

I considered uploading my generic font which includes

  • full GB2312-80 characters
  • BIG5 level 1 characters
  • SJIS level 1 characters
  • total of 9240 CJK ideographs

but it is too large for mass distribution (~150MB includes TC and SC variant).

Instead I'm uploading that char list for anyone who needs it chinese.txt

大哥，你是中国人吗？

CaiMiao commented 1 year ago

@bayaraa I strongly suggest you revert the v1.2.1 change that merge CJK font into one. There is two reason for this:

bayaraa commented 1 year ago

Those fonts really that ugly? I thought its very stylysh for my foreigner eye :(

I think combining Chinese and japanese into one is ok. But keep korean font separate. Cuz they dont need to load extra glypes (slower load).

Yeah Fzkai has copyright issue i assume. But shueymincho pretty sure my adobe subscription clears it out. Even tho this is non commercial project.

blackdarks commented 1 year ago

I used msdf-atlas-gen.exe to make fonts, but now I delete the directory immediately, and the in-game fonts still cannot return to the default, how to solve this?

CaiMiao commented 1 year ago

combine two different writing system is the main issue

Message ID: @.***>

bayaraa commented 1 year ago

@CaiMiao i see. i thought you wanted JCK into one big font. my bad. but on 1.2.1 all three languages use different fonts already (same as 1.2.0). only issue is i don't know Fzkai is simplified or traditional. and has copyright.

CaiMiao commented 1 year ago

https://helpx.adobe.com/fonts/using/font-licensing.html#act-app in most cases using a font in software require separate license. these font atlas will probably not be considered as 'artwork', as it does not include any design with artistic integrity

在 2023/07/14，09:42，Bayaraa @.***> 写道：



But shueymincho pretty sure my adobe subscription clears it out. Even tho this is non commercial project.

— Reply to this email directly, view it on GitHubhttps://github.com/bayaraa/d2gl/issues/9#issuecomment-1635136715, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABDM5Y54PUJ47SU4ERS3JRLXQCPZLANCNFSM6AAAAAAVPGF4SE. You are receiving this because you were mentioned.Message ID: @.***>

CaiMiao commented 1 year ago

as of v1.2.2 this issue should be solved. will open another when other issue raises.