search

jgmdev / lite-xl-encoding

Encoding auto-detection and conversion support for Lite XL
MIT License
1 stars 1 forks source link

A couple issues with this plugin #1

Open kice opened 1 year ago

kice commented 1 year ago

I have few issues with this plugin that prevent me from opening files.

  1. Pre-build DLL cannot be loaded under windows. Most likely missing some dlls. Have to compile in Windows to make it works.
  2. Unable to load files path that is not ascii encoded. Since Lua internally use UTF-8 for string storage, while windows does not. Meaning we have to convert to UTF-16LE/active codepages first before opening the files. fopen will return NULL. I would suggest convert file path to UTF-16LE first and then call _wfopen, also local active code page does not contain all Unicode characters, and Windows allow almost all unicode character as file path. https://github.com/jgmdev/lite-xl-encoding/blob/b1ddf226277ea12a03ed9db2ddda458988020e91/src/encoding.c#L303
  3. When uchardet detect encoding incorrectly, and uchardet never gets right the first time, the encoding plugin just refuse to load the files. I think we could decode it with best effort if the detect result is incorrect or prompt the user for correct encoding.
jgmdev commented 1 year ago

Thanks for investigating the issues, I haven't tested this plugin on windows.

Pre-build DLL cannot be loaded under windows. Most likely missing some dlls. Have to compile in Windows to make it works.

Maybe needs some iconv dll, I wonder how the best way will be to handle this with the new lite xl package manager...

Unable to load files path that is not ascii encoded.

Could you work on adding the proper windows support and PR?

I think we could decode it with best effort if the detect result is incorrect or prompt the user for correct encoding.

Mmm, so maybe a commandview listing all valid encodings and let the user choose one?

kice commented 1 year ago

I tried to add missing dll by checking import table, also add the dll that iconv used, and so on. Somehow still unable to load encoding.dll; that's why I tried to compile it myself. However, I found that the dll I compiled is static linked everything, so you might want to do the same for a standalone release.

I am not sure how the predefined macros works with meson, so change if you see fit.

#ifndef _WIN32
#else
#include <windows.h>
#endif 

int f_detect(lua_State *L) {
  const char* file_name = luaL_checkstring(L, 1);

#ifndef _WIN32
  FILE* file = fopen(file_name, "rb");
#else
  wchar_t utf16[1024];
  memset(utf16, 0, sizeof(utf16));
  MultiByteToWideChar(CP_UTF8, 0, file_name, strlen(file_name), utf16, 1024);
  // utf8_to_utf16le(file_name, utf16, 1024, NULL);
  FILE* file = _wfopen(utf16, L"rb");
#endif

  if (!file) {
    lua_pushnil(L);
        lua_pushfstring(L, "unable to open file '%s', code=%d", file_name, errno);
        return 2;
  }

//rest of the function

I think not include any windows header would also work, but require to write a decoder for decoding the filename from UTF-8 to Unicode Point and then a encoder for UTF16-LE. Just keep in mind that Windows file path should support codepoint higher than 0xFFFF, which use UTF-16 surrogate pair and two uint16_t, such as rocket emoji πŸš€ U+1F680.

Or just use detect_string instead of detect.

For auto decoding, the fact is uchardet hardly get the encoding right for most asian languages. And some encoding is kinda of a subset of another encoding. For example, most of the bytes of shift-jis/CP932 can be decoded as GBK/CP936.

For me, if detect as UTF-8, UTF-16 LE, UTF-16 BE, then just decode with the result. For the rest, use the result to sort the list and then let user choose from it.

jgmdev commented 1 year ago

Ok, so I added your change https://github.com/jgmdev/lite-xl-encoding/commit/de1c9be297e034e8466dd7ee0d714b0481684379 hope that I copy pasted properly :)

I tried to compile it myself. However, I found that the dll I compiled is static linked everything

Can you share how did you build the plugin to accomplish this?

For auto decoding, the fact is uchardet hardly get the encoding right for most asian languages.

Does doc:reload-with-encoding tackles that?

For me, if detect as UTF-8, UTF-16 LE, UTF-16 BE, then just decode with the result. For the rest, use the result to sort the list and then let user choose from it.

So currently encoding_detect checks if one of the registered boms in bom_list is found on the given string, if that is the case then it converts from that bom charset into utf8 and if success returns the found bom charset.

If the bom detection and conversion fails then it checks if the string is valid utf8 and just return that charset. Otherwise it fallbacks to uchardet and if uchardet fails it just errors.

What you want is on error to allow manually specifying the charset or change the order of the steps above?

jgmdev commented 1 year ago

Doing a local test using ntldd to check linked libraries and building under msys2 mingw64 this is what I got:

ADVAPI32.dll => C:\Windows\SYSTEM32\ADVAPI32.dll (0x000001feed4e0000)
GDI32.dll => C:\Windows\SYSTEM32\GDI32.dll (0x000001feee610000)
libiconv-2.dll => C:\msys64\mingw64\bin\libiconv-2.dll (0x000001feed990000)
IMM32.dll => C:\Windows\SYSTEM32\IMM32.dll (0x000001feed4e0000)
KERNEL32.dll => C:\Windows\SYSTEM32\KERNEL32.dll (0x000001feed990000)
msvcrt.dll => C:\Windows\SYSTEM32\msvcrt.dll (0x000001feed990000)
ole32.dll => C:\Windows\SYSTEM32\ole32.dll (0x000001feed990000)
OLEAUT32.dll => C:\Windows\SYSTEM32\OLEAUT32.dll (0x000001feee220000)
SETUPAPI.dll => C:\Windows\SYSTEM32\SETUPAPI.dll (0x000001feeedb0000)
SHELL32.dll => C:\Windows\SYSTEM32\SHELL32.dll (0x000001feedd60000)
libstdc++-6.dll => C:\msys64\mingw64\bin\libstdc++-6.dll (0x000001feedd60000)
USER32.dll => C:\Windows\SYSTEM32\USER32.dll (0x000001feee450000)
VERSION.dll => C:\Windows\SYSTEM32\VERSION.dll (0x000001feed560000)
WINMM.dll => C:\Windows\SYSTEM32\WINMM.dll (0x000001feed4e0000)

From this the revelant dlls would be:

libstdc++-6.dll => C:\msys64\mingw64\bin\libstdc++-6.dll (0x000001feedd60000) libiconv-2.dll => C:\msys64\mingw64\bin\libiconv-2.dll (0x000001feed990000)

kice commented 1 year ago
  1. I have no idea what is going on, but I think my compile environment is different than yours. I just cloned the repo and then 
    meson setup build
meson compile -C build

    meson is installed with ninja by meson-1.0.0-64.msi, and I am not using MSYS2, and gcc/g++ is installed by scoop.

meson setup build output ```log The Meson build system Version: 1.0.0 Source dir: C:\src\lite-xl-encoding Build dir: C:\src\lite-xl-encoding\build Build type: native build Project name: encoding Project version: 1.0 C compiler for the host machine: gcc (gcc 11.2.0 "gcc (GCC) 11.2.0") C linker for the host machine: gcc ld.bfd 2.37 C++ compiler for the host machine: g++ (gcc 11.2.0 "g++ (GCC) 11.2.0") C++ linker for the host machine: g++ ld.bfd 2.37 Host machine cpu family: x86_64 Host machine cpu: x86_64 Did not find pkg-config by name 'pkg-config' Found Pkg-config: NO Found CMake: C:\Program Files\CMake\bin\cmake.EXE (3.25.0) Run-time dependency uchardet found: NO (tried pkgconfig and cmake) Looking for a fallback subproject for the dependency uchardet Executing subproject uchardet uchardet| Project name: uchardet uchardet| Project version: 0.0.7 uchardet| C compiler for the host machine: gcc (gcc 11.2.0 "gcc (GCC) 11.2.0") uchardet| C linker for the host machine: gcc ld.bfd 2.37 uchardet| C++ compiler for the host machine: g++ (gcc 11.2.0 "g++ (GCC) 11.2.0") uchardet| C++ linker for the host machine: g++ ld.bfd 2.37 uchardet| Compiler for C++ supports arguments -msse2: YES uchardet| Compiler for C++ supports arguments -mfpmath=sse: YES uchardet| Build targets in project: 3 uchardet| Subproject uchardet finished. Dependency uchardet from subproject subprojects/uchardet-0.0.7 found: YES 0.0.7 sdl2-config found: NO Run-time dependency sdl2 found: NO (tried pkgconfig and config-tool) Looking for a fallback subproject for the dependency sdl2 Executing subproject sdl2 sdl2| Project name: sdl2 sdl2| Project version: 2.26.0 sdl2| C compiler for the host machine: gcc (gcc 11.2.0 "gcc (GCC) 11.2.0") sdl2| C linker for the host machine: gcc ld.bfd 2.37 sdl2| Compiler for C supports arguments -Wdeclaration-after-statement: YES sdl2| Compiler for C supports arguments -Wshadow: YES sdl2| Compiler for C supports arguments -fno-strict-aliasing: YES sdl2| Compiler for C supports arguments -mpreferred-stack-boundary=2: NO sdl2| Dependency gl skipped: feature use_video_opengl disabled sdl2| Dependency opengl skipped: feature use_video_opengl disabled sdl2| Dependency glesv2 skipped: feature use_video_openglesv2 disabled sdl2| Dependency egl skipped: feature use_video_openglesv2 disabled sdl2| Dependency egl skipped: feature use_video_wayland disabled sdl2| Dependency alsa skipped: feature use_audio disabled sdl2| Dependency libpulse-simple skipped: feature use_audio disabled sdl2| Dependency jack skipped: feature use_audio disabled sdl2| Dependency libpipewire-0.3 skipped: feature use_audio disabled sdl2| Dependency vulkan skipped: feature use_video_vulkan disabled sdl2| Run-time dependency threads found: YES sdl2| Run-time dependency libusb-1.0 found: NO (tried pkgconfig and cmake) sdl2| Dependency wayland-client skipped: feature use_video_wayland disabled sdl2| Dependency wayland-cursor skipped: feature use_video_wayland disabled sdl2| Dependency wayland-egl skipped: feature use_video_wayland disabled sdl2| Dependency wayland-protocols skipped: feature use_video_wayland disabled sdl2| Dependency wayland-scanner skipped: feature use_video_wayland disabled sdl2| Dependency libdecor-0 skipped: feature use_video_wayland disabled sdl2| Program wayland-scanner skipped: feature use_video_wayland disabled sdl2| Program ./find-dylib-name.py found: YES (C:\Program Files\Meson\meson.exe runpython C:\src\lite-xl-encoding\subprojects\SDL2-2.26.0\./find-dylib-name.py) sdl2| Library m found: YES sdl2| Library dl found: NO sdl2| Library iconv found: NO sdl2| Message: Subsystem "atomic" is ENABLED sdl2| Message: Subsystem "audio" is DISABLED sdl2| Message: Subsystem "cpuinfo" is DISABLED sdl2| Message: Subsystem "events" is ENABLED sdl2| Message: Subsystem "file" is ENABLED sdl2| Message: Subsystem "filesystem" is ENABLED sdl2| Message: Subsystem "haptic" is DISABLED sdl2| Message: Subsystem "hidapi" is ENABLED sdl2| Message: Subsystem "joystick" is DISABLED sdl2| Message: Subsystem "loadso" is ENABLED sdl2| Message: Subsystem "locale" is ENABLED sdl2| Message: Subsystem "power" is DISABLED sdl2| Message: Subsystem "render" is ENABLED sdl2| Message: Subsystem "sensor" is DISABLED sdl2| Message: Subsystem "threads" is ENABLED sdl2| Message: Subsystem "timers" is ENABLED sdl2| Message: Subsystem "video" is ENABLED sdl2| Checking for size of "void*" : 8 sdl2| Check usable header "alloca.h" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Check usable header "altivec.h" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Check usable header "audioclient.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "ctype.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "ddraw.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "dinput.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "dsound.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "dxgi.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "endpointvolume.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "fcitx/frontend.h" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Check usable header "float.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "iconv.h" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Check usable header "immintrin.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "inttypes.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "libsamplerate.h" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Check usable header "libudev.h" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Check usable header "libunwind.h" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Check usable header "limits.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "malloc.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "math.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "memory.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "mmdeviceapi.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "pthread.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "pthread_np.h" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Check usable header "semaphore.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "sensorsapi.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "signal.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "stdarg.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "stdint.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "stddef.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "stdio.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "stdlib.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "string.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "strings.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "sys/inotify.h" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Check usable header "sys/types.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "wchar.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "linux/input.h" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Check usable header "usbhid.h" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Check usable header "libusbhid.h" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Check usable header "usb.h" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Check usable header "libusb.h" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Check usable header "d3d.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "d3d11.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "d3d12.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Check usable header "windows.gaming.input.h" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Check usable header "xinput.h" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Header "fcntl.h" has symbol "O_CLOEXEC" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Header "math.h" has symbol "M_PI" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Header "pthread.h" has symbol "PTHREAD_MUTEX_RECURSIVE" : YES sdl2| Header "pthread.h" has symbol "PTHREAD_MUTEX_RECURSIVE_NP" : YES sdl2| Checking for function "_Exit" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "_i64toa" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "_ltoa" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Checking for function "_stricmp" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "_strlwr" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "_strnicmp" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "_strrev" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "_strupr" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "_uit64toa" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Checking for function "_uitoa" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Checking for function "_ultoa" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "_wcsdup" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "_wcsicmp" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "_wcsnicmp" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "abs" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "bsearch" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "acos" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "acosf" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "alloca" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "asin" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "asinf" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "atan" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "atan2" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "atan2f" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "atanf" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "atof" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "atoi" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "bcopy" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Checking for function "calloc" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "ceil" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "ceilf" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "clock_gettime" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "copysign" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "copysignf" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "cos" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "cosf" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "dlopen" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Checking for function "elf_aux_info" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Checking for function "exp" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "expf" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "fabs" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "fabsf" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "floor" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "floorf" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "fmod" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "fmodf" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "fopen64" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "free" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "fseeko" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "fseeko64" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "getauxval" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Checking for function "getenv" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "getpagesize" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Checking for function "iconv" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Checking for function "index" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Checking for function "inotify_init" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Checking for function "inotify_init1" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Checking for function "itoa" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "log" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "log10" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "log10f" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "logf" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "lround" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "lroundf" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "malloc" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "memcmp" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "memcpy" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "memmove" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "memset" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "mprotect" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Checking for function "nanosleep" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "poll" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Checking for function "pow" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "powf" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "pthread_set_name_np" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Checking for function "pthread_setname_np" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "putenv" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "qsort" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "realloc" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "rindex" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Checking for function "round" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "roundf" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "scalbn" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "scalbnf" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "sem_timedwait" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "setenv" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Checking for function "setjmp" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "sigaction" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Checking for function "sin" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "sinf" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "snprintf" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "sqrt" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "sqrtf" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "sscanf" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "strcasecmp" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "strchr" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "strcmp" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "strdup" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "strlcat" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Checking for function "strlcpy" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Checking for function "strlen" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "strncasecmp" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "strncmp" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "strrchr" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "strstr" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "strtod" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "strtok_r" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "strtol" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "strtoll" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "strtoul" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "strtoull" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "sysconf" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Checking for function "sysctlbyname" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Checking for function "tan" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "tanf" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "trunc" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "truncf" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "unsetenv" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Checking for function "vsnprintf" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "vsscanf" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "wcscmp" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "wcsdup" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "wcslen" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "wcsncmp" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for function "wcsstr" with dependencies -lm, -ldl, -liconv, , , threads: YES sdl2| Checking for type "XINPUT_GAMEPAD_EX" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Checking for type "XINPUT_STATE_EX" with dependencies -lm, -ldl, -liconv, , , threads: NO sdl2| Checking whether type "struct sigaction" has member "sa_sigaction" : NO sdl2| Checking if "__atomic_load_n test" : links: YES sdl2| Checking if "__sync_lock_test_and_set test" : links: YES sdl2| Checking if "input events test" compiles: NO sdl2| Library user32 found: YES sdl2| Library gdi32 found: YES sdl2| Library winmm found: YES sdl2| Library imm32 found: YES sdl2| Library ole32 found: YES sdl2| Library oleaut32 found: YES sdl2| Library version found: YES sdl2| Library uuid found: YES sdl2| Library advapi32 found: YES sdl2| Library setupapi found: YES sdl2| Library shell32 found: YES sdl2| Library dinput8 found: YES sdl2| Message: filesystem: windows sdl2| Message: haptic: dummy sdl2| Message: joystick: dummy sdl2| Message: loadso: windows sdl2| Message: power: sdl2| Message: sensor: dummy sdl2| Message: video: sdl2| Message: input: sdl2| Message: audio_driver: dummy sdl2| Message: video_driver: windows, dummy sdl2| Message: video_render: d3d, d3d11, d3d12 sdl2| Message: thread: windows sdl2| Message: timer: windows sdl2| Configuring SDL_config.h using configuration sdl2| Build targets in project: 5 sdl2| NOTICE: Future-deprecated features used: sdl2| * 0.58.0: {'meson.get_cross_property'} sdl2| * 0.64.0: {'copy arg in configure_file'} sdl2| Subproject sdl2 finished. Dependency sdl2 found: YES 2.26.0 (overridden) Build targets in project: 6 encoding 1.0 Subprojects sdl2 : YES 1 warnings uchardet: YES Found ninja-1.11.1 at "C:\Program Files\Meson\ninja.EXE" ```
  1. Since Lite-xl just refuse to open the file, so I cannot execute the doc:reload-with-encoding.
  2. on error to allow manually specifying the charset is what i want. Changing the order would not work since nobody in the chain has the correct encoding.
  3. I just found that for GBK the plugin will try to decode it as KOI8-R, which will fail; and correctly detect shift-jis but still fail to open it. You can try make some test files by this python snippet. 
    
with open("gbk.txt", "w", encoding="gbk") as f:
f.write("δΈ­ζ–‡ζ΅‹θ―•")

with open("shiftjis.txt", "w", encoding="cp932") as f: f.write("γ‚γ„γ†γˆγŠζ—₯本θͺž")

jgmdev commented 1 year ago

ok, so fixed the dynamically linked libraries with this https://github.com/jgmdev/lite-xl-encoding/commit/e2cd00ba2304e8a0625ed6851b49cd96f76b39cd now what is missing is asking the user for charset if not detected :)