[extras/linenoise] UTF-8 support not working properly

[github-actions[bot]]

[extras/linenoise] UTF-8 support not working properly

should add -DUSE_UTF8 for UTF-8 support

to all following lines

the problem is that - although it "works" - different characters are not shown at all:

e.g. accented characters, or ø - our symbol for null

https://github.com/arturo-lang/arturo/blob/3a114155077768d3542ce012605e78b10a0cc6b2/src/extras/linenoise.nim#L24

{.passC: "-I" & parentDir(currentSourcePath()) .}

# TODO(extras/linenoise) UTF-8 support not working properly
#  should add -DUSE_UTF8 for UTF-8 support
#  to all following lines
#  the problem is that - although it "works" - different characters are not shown at all:
#  e.g. accented characters, or ø - our symbol for null
#  labels: bug, 3rd-party, repl
{.compile("linenoise/linenoise.c", "-I" & parentDir(currentSourcePath())).}
{.compile("linenoise/stringbuf.c", "-I" & parentDir(currentSourcePath())).}
{.compile("linenoise/utf8.c", "-I" & parentDir(currentSourcePath())).}

#=======================================
# Types
ndex 204ec243f..571c77aa9 100644
++ b/version/build

f03bc1611ea42487492a8e8754f10ead9ea92679

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[drkameleon]

I'm marking this as critical since I basically think it is.

Arturo was built was proper unicode support in mind and apparently everything works along these lines except for the REPL which is a shame!

@RickBarretto Can you please confirm this to make sure it's not something too macOS-related?

For example, this seems to be working:

But actually writing it, character by character, it seems like it's adding extra spaces or considers it like a "double" character (so, it seems to be adding an extra space visually - which doesn't really exist - and trying to delete the previous character leads in a mess).

Also, using anything more complicated than accented characters, leads to an even bigger mess:

(This is the greek equivalent of "Good morning" and that's where the cursor supposedly is - while it's actually next to the closing double-quote)

This is the result of writing Chinese - or trying to - "早上好" (another way to say Good morning lol). Now, the result is even more disheartening:

Funny enough, the above happens also when copy-pasting the text ❗

Weird, weird... 🤔

[RickBarretto]

I'm opening my VS right now to test it! :wink:

[RickBarretto]

On VsCode:

Running from Powershell:

Running from bash:

But, not exclusive of Arturo. For Nim's secret too:

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[scifx]

In REPL mode, UTF-8 always fails to work. Is there a fix in the works?

[drkameleon]

@scifx Very valid question, but I'm afraid I don't have a very real answer.

We've been experimenting lately, although I wouldn't say the experiments have been extremely fruitful.

In any case, I can assure you it's one of the things that personally bother me a lot... so, it will be prioritized, probably one the upcoming release is finalized.

Btw: which OS are you using?

[scifx]

You've been keeping an eye on it, that's good. I hope you can resolve it soon. Currently, I'm using the Windows system, but I used to use Linux, and I have also compiled it on my phone before. Therefore, this issue exists across all systems.