search

open-eid / libdigidocpp

Libdigidocpp library offers creating, signing and verification of digitally signed documents, according to XAdES and XML-DSIG standards. Documentation http://open-eid.github.io/libdigidocpp
Other
97 stars 47 forks source link

addDataFile throws "Document file test ääää.txt" does not exists." when filename or directory contains umlauts #267

Closed jannikov closed 5 years ago

jannikov commented 5 years ago

I have libdigidocpp-3.13.8.1379 installed on windows 10

metsma commented 5 years ago

libdigidocpp API requires input to be UTF-8 encoding. Not ANSI local

jannikov commented 5 years ago

Yes, I have input in UTF8. image but anyway I got error image

metsma commented 5 years ago

57 line does not make any sense. Converting from utf-8 to utf-8? And some places is variable file and some filename? Whats the difference

jannikov commented 5 years ago

it only for example. To explain. And .net returned, that this file is exist

jannikov commented 5 years ago

I changed the names. One filename is BDOC container and another one is file that should be signed image

jannikov commented 5 years ago

Maybe there is some trick with codepage? I already double checked, I have UTF 8

jannikov commented 5 years ago

And why you are doing filename conversions if you expect it in the utf8. \libdigidocpp\source\src\util\File.cpp Check fileExists -> encodeName -> convertUTF8

metsma commented 5 years ago

Because we need to have cross platform similar API. It Encodes input (UTF-8) to Windows Wide character API.

metsma commented 5 years ago

Seems like PINVOKE resolves parameters to local ANSI encoding. This needs more investigating

jannikov commented 5 years ago

Thank you! Yes I tried to to change charset in the PINVOKE generated code. But there are only Unicode and ANSI options available. And no success because Unicode it have more bytes then UTF-8. At the moment I trying to find some workaround.

[global::System.Runtime.InteropServices.DllImport("digidoc_csharp", EntryPoint="CSharp_digidoc_Container_addDataFile", CharSet = System.Runtime.InteropServices.CharSet.Unicode)] public static extern void Container_addDataFile(global::System.Runtime.InteropServices.HandleRef jarg1, string jarg2, string jarg3);

One thing else. SWIG generated only one addDataFile method, the second addDataFile with steam argument is missing. I think it because they both having same name.

metsma commented 5 years ago

Thank you! Yes I tried to to change charset in the PINVOKE generated code. But there are only Unicode and ANSI options available. And no success because Unicode it have more bytes then UTF-8. At the moment I trying to find some workaround.

[global::System.Runtime.InteropServices.DllImport("digidoc_csharp", EntryPoint="CSharp_digidoc_Container_addDataFile", CharSet = System.Runtime.InteropServices.CharSet.Unicode)] public static extern void Container_addDataFile(global::System.Runtime.InteropServices.HandleRef jarg1, string jarg2, string jarg3);

Unicode maps to Wide character set. There are couple feature requests https://github.com/dotnet/corefx/issues/7804 https://github.com/dotnet/coreclr/issues/1012 Right now we need solve this in swig generated wrapper somehow

One thing else. SWIG generated only one addDataFile method, the second addDataFile with steam argument is missing. I think it because they both having same name.

Swig does not have c++ stream and C# stream mappings support right now

jannikov commented 5 years ago

SWIG even generates some wrapper for istream. But I dont't know how to map it

public virtual void bbb(SWIGTYPE_p_std__istream arg0, string fileName, string mediaType) { digidocPINVOKE.Container_bbb(swigCPtr, SWIGTYPE_p_std__istream.getCPtr(arg0), fileName, mediaType); if (digidocPINVOKE.SWIGPendingException.Pending) throw digidocPINVOKE.SWIGPendingException.Retrieve(); } global::System.Runtime.InteropServices.DllImport("digidoc_csharp", EntryPoint="CSharp_digidoc_Container_bbb")] public static extern void Container_bbb(global::System.Runtime.InteropServices.HandleRef jarg1, global::System.Runtime.InteropServices.HandleRef jarg2, string jarg3, string jarg4); `

metsma commented 5 years ago

SWIG even generates some wrapper for istream.

public virtual void bbb(SWIGTYPE_p_std__istream arg0, string fileName, string mediaType) { digidocPINVOKE.Container_bbb(swigCPtr, SWIGTYPE_p_std__istream.getCPtr(arg0), fileName, mediaType); if (digidocPINVOKE.SWIGPendingException.Pending) throw digidocPINVOKE.SWIGPendingException.Retrieve(); }

global::System.Runtime.InteropServices.DllImport("digidoc_csharp", EntryPoint="CSharp_digidoc_Container_bbb")] public static extern void Container_bbb(global::System.Runtime.InteropServices.HandleRef jarg1, global::System.Runtime.InteropServices.HandleRef jarg2, string jarg3, string jarg4); `

This is not usable in C# Here is list c++ std supported types https://github.com/swig/swig/tree/master/Lib/csharp

jannikov commented 5 years ago

Heia, I found workaround. Has been added 2 new methods and modified PINVOKE proxy. See code bellow, hope this code will be helpfull to you.

Container.cpp Container* Container::createWin(wchar_t *path) { string pathDecoded = util::File::decodeName(path); return create(pathDecoded.c_str()); }

Container.cs public static Container createWin(string path) { global::System.IntPtr cPtr = digidocPINVOKE.Container_createWin(path); Container ret = (cPtr == global::System.IntPtr.Zero) ? null : new Container(cPtr, false); if (digidocPINVOKE.SWIGPendingException.Pending) throw digidocPINVOKE.SWIGPendingException.Retrieve(); return ret; }

public virtual void addDataFileWin(string path, string mediaType) { digidocPINVOKE.Container_addDataFileWin(swigCPtr, path, mediaType); if (digidocPINVOKE.SWIGPendingException.Pending) throw digidocPINVOKE.SWIGPendingException.Retrieve(); }

ASiContainer.cpp void ASiContainer::addDataFileWin(wchar_t *path, wchar_t *mediaType) { string pathDecoded = File::decodeName(path); string mediaTypeDecoded = File::decodeName(mediaType); addDataFile(pathDecoded.c_str(), mediaTypeDecoded.c_str()); }

digidocPINVOKE.cs [global::System.Runtime.InteropServices.DllImport("digidoc_csharp", EntryPoint = "CSharp_digidoc_Container_createWin", CharSet = System.Runtime.InteropServices.CharSet.Unicode, CallingConvention = System.Runtime.InteropServices.CallingConvention.StdCall)] public static extern global::System.IntPtr Container_createWin(string jarg1);

[global::System.Runtime.InteropServices.DllImport("digidoc_csharp", EntryPoint= "CSharp_digidoc_Container_addDataFileWin", CharSet = System.Runtime.InteropServices.CharSet.Unicode, CallingConvention = System.Runtime.InteropServices.CallingConvention.StdCall)] public static extern void Container_addDataFileWin(global::System.Runtime.InteropServices.HandleRef jarg1, string jarg2, string jarg3);

metsma commented 5 years ago

Can you test https://ci.appveyor.com/project/open-eid/libdigidocpp/builds/21722020/job/xyc9cyf3879401vn/artifacts if build completes

jannikov commented 5 years ago

Yes UTF8Marshaler works!