search

Code52 / DownmarkerWPF

MarkPad - a visual Markdown editor (inspired by the Downmarker project)
http://code52.org/DownmarkerWPF/
Microsoft Public License
1.41k stars 460 forks source link

CJK Filename/Path Encoding Issue. #355

Closed Gateswong closed 11 years ago

Gateswong commented 11 years ago

Problem

When opened the file including CJK chars in filename/path. The 'RecentFiles' in MarkPadSettings.settings appears to be not able to read and cause an error.

In that '.settings' file, the filename is MarkPad 娑擃厽鏋冮崥?md which should be MarkPad 中文名.md.

Here is the exception:

(Inner Exception)
(Inner Exception)
(Inner Exception)
Exception Source:      mscorlib
Exception Type:        System.Text.DecoderFallbackException
Exception Message:     无法将位于索引 6 处的字节 [E5][90] 由指定的代码页转换为 Unicode。 // Unable to decode the byte in index 6 from default codepage to unicode.
Exception Target Site: Throw

---- Stack Trace ----
   System.Text.DecoderExceptionFallbackBuffer.Throw(Byte[] bytesUnknown, Int32 index)
       MarkPad.exe: N 3268910
   System.Text.DecoderExceptionFallbackBuffer.Fallback(Byte[] bytesUnknown, Int32 index)
       MarkPad.exe: N 00008
   System.Text.DecoderFallbackBuffer.InternalFallback(Byte[] bytes, Byte* pBytes, Char*& chars)
       MarkPad.exe: N 00041
   System.Text.UTF8Encoding.GetChars(Byte* bytes, Int32 byteCount, Char* chars, Int32 charCount, DecoderNLS baseDecoder)
       MarkPad.exe: N 10669793
   System.Text.UTF8Encoding.GetChars(Byte[] bytes, Int32 byteIndex, Int32 byteCount, Char[] chars, Int32 charIndex)
       MarkPad.exe: N 00206
   System.Xml.XmlConverter.ToChars(Byte[] buffer, Int32 offset, Int32 count, Char[] chars, Int32 charOffset)
       MarkPad.exe: N 00055

(Outer Exception)
Exception Source:      System.Runtime.Serialization
Exception Type:        System.Xml.XmlException
Exception Message:     “中文??md”包含无效的 UTF8 字节。
Exception Target Site: ToChars
未将对象引用设置到对象的实例。

(Outer Exception)
Exception Source:      System.Runtime.Serialization
Exception Type:        System.Runtime.Serialization.SerializationException
Exception Message:     反序列化对象 属于类型 System.Collections.Generic.List`1[[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]] 时出现错误。“中文??md”包含无效的 UTF8 字节。
Exception Target Site: ReadObjectHandleExceptions

---- Stack Trace ----
   System.Runtime.Serialization.XmlObjectSerializer.ReadObjectHandleExceptions(XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContractResolver)
       MarkPad.exe: N 00419
   System.Runtime.Serialization.Json.DataContractJsonSerializer.ReadObject(XmlDictionaryReader reader)
       MarkPad.exe: N 00061
   System.Runtime.Serialization.Json.DataContractJsonSerializer.ReadObject(Stream stream)
       MarkPad.exe: N 00107
   MarkPad.Settings.JsonSettingsStoreBase.DeserializeList(String serializedList)
       IsolatedStorageSettingsStore.cs: line 0056, col 13, IL 0031
   MarkPad.Settings.SettingsProvider.ReadList(String storedValue, Type propertyType)
       SettingsProvider.cs: line 0119, col 13, IL 0023
   MarkPad.Settings.SettingsProvider.ConvertValue(String storedValue, Type underlyingType)
       SettingsProvider.cs: line 0094, col 25, IL 0123
   MarkPad.Settings.SettingsProvider.GetSettings(Boolean fresh)
       SettingsProvider.cs: line 0060, col 17, IL 0153
   MarkPad.Infrastructure.MarkPadAutofacModule.<Load>b__0(IActivatingEventArgs`1 args)
       MarkPadAutofacModule.cs: line 0033, col 17, IL 0012
   Autofac.Builder.<>c__DisplayClass6.<OnActivating>b__5(Object s, ActivatingEventArgs`1 e)
       MarkPad.exe: N 00127
   Autofac.Core.Registration.ComponentRegistration.RaiseActivating(IComponentContext context, IEnumerable`1 parameters, Object& instance)
       MarkPad.exe: N 00073
   Autofac.Core.Resolving.InstanceLookup.Activate(IEnumerable`1 parameters)
       MarkPad.exe: N 00128
   Autofac.Core.Resolving.InstanceLookup.<Execute>b__0()
       MarkPad.exe: N 00008
   Autofac.Core.Lifetime.LifetimeScope.GetOrCreateAndShare(Guid id, Func`1 creator)
       MarkPad.exe: N 00112
   Autofac.Core.Resolving.InstanceLookup.Execute()
       MarkPad.exe: N 00132
   Autofac.Core.Resolving.ResolveOperation.GetOrCreateInstance(ISharingLifetimeScope currentOperationScope, IComponentRegistration registration, IEnumerable`1 parameters)
       MarkPad.exe: N 00167
   Autofac.Core.Resolving.InstanceLookup.ResolveComponent(IComponentRegistration registration, IEnumerable`1 parameters)
       MarkPad.exe: N 00024
   Autofac.Core.Activators.Reflection.<>c__DisplayClass2.<CanSupplyValue>b__0()
       MarkPad.exe: N 00031
   Autofac.Core.Activators.Reflection.ConstructorParameterBinding.Instantiate()
       MarkPad.exe: N 00094
   Autofac.Core.Activators.Reflection.ReflectionActivator.ActivateInstance(IComponentContext context, IEnumerable`1 parameters)
       MarkPad.exe: N 00184
   Autofac.Core.Resolving.InstanceLookup.Activate(IEnumerable`1 parameters)
       MarkPad.exe: N 00048
   Autofac.Core.Resolving.InstanceLookup.Execute()
       MarkPad.exe: N 00048
   Autofac.Core.Resolving.ResolveOperation.GetOrCreateInstance(ISharingLifetimeScope currentOperationScope, IComponentRegistration registration, IEnumerable`1 parameters)
       MarkPad.exe: N 00167
   Autofac.Core.Resolving.InstanceLookup.ResolveComponent(IComponentRegistration registration, IEnumerable`1 parameters)
       MarkPad.exe: N 00024
   Autofac.Core.Activators.Reflection.<>c__DisplayClass2.<CanSupplyValue>b__0()
       MarkPad.exe: N 00031
   Autofac.Core.Activators.Reflection.ConstructorParameterBinding.Instantiate()
       MarkPad.exe: N 00094
   Autofac.Core.Activators.Reflection.ReflectionActivator.ActivateInstance(IComponentContext context, IEnumerable`1 parameters)
       MarkPad.exe: N 00184
   Autofac.Core.Resolving.InstanceLookup.Activate(IEnumerable`1 parameters)
       MarkPad.exe: N 00048
   Autofac.Core.Resolving.InstanceLookup.<Execute>b__0()
       MarkPad.exe: N 00008
   Autofac.Core.Lifetime.LifetimeScope.GetOrCreateAndShare(Guid id, Func`1 creator)
       MarkPad.exe: N 00112
   Autofac.Core.Resolving.InstanceLookup.Execute()
       MarkPad.exe: N 00132
   Autofac.Core.Resolving.ResolveOperation.GetOrCreateInstance(ISharingLifetimeScope currentOperationScope, IComponentRegistration registration, IEnumerable`1 parameters)
       MarkPad.exe: N 00167
   Autofac.Core.Resolving.ResolveOperation.Execute(IComponentRegistration registration, IEnumerable`1 parameters)
       MarkPad.exe: N 00044

(Outer Exception)
Date and Time:         2013/2/5 20:06:37
OS Version:            Microsoft Windows NT 6.1.7601 Service Pack 1

Application Domain:    MarkPad.exe
Assembly Codebase:     file:///D:/System.Windows/Program Files (x86)/Markpad/MarkPad.exe
Assembly Full Name:    MarkPad, Version=0.9.7.0, Culture=neutral, PublicKeyToken=null
Assembly Version:      0.9.7.0
Assembly Build Date:   2013/2/5 19:51:57

Exception Source:      Autofac
Exception Type:        Autofac.Core.DependencyResolutionException
Exception Message:     An exception was thrown while executing a resolve operation. See the InnerException for details.
Exception Target Site: Execute

---- Stack Trace ----
   Autofac.Core.Resolving.ResolveOperation.Execute(IComponentRegistration registration, IEnumerable`1 parameters)
       MarkPad.exe: N 00112
   Autofac.Core.Lifetime.LifetimeScope.ResolveComponent(IComponentRegistration registration, IEnumerable`1 parameters)
       MarkPad.exe: N 00164
   Autofac.Core.Container.ResolveComponent(IComponentRegistration registration, IEnumerable`1 parameters)
       MarkPad.exe: N 00014
   Autofac.ResolutionExtensions.TryResolveService(IComponentContext context, Service service, IEnumerable`1 parameters, Object& instance)
       MarkPad.exe: N 00070
   Autofac.ResolutionExtensions.TryResolve(IComponentContext context, Type serviceType, Object& instance)
       MarkPad.exe: N 00070
   Caliburn.Micro.Autofac.AutofacBootstrapper`1.GetInstance(Type service, String key)
       MarkPad.exe: N 00043
   Caliburn.Micro.BootstrapperBase.DisplayRootViewFor(Type viewModelType)
       MarkPad.exe: N 00037
   MarkPad.Infrastructure.AppBootstrapper.OnStartup(Object sender, StartupEventArgs e)
       AppBootstrapper.cs: line 0057, col 13, IL 0000
   System.Windows.Application.OnStartup(StartupEventArgs e)
       MarkPad.exe: N 00123
   System.Windows.Application.<.ctor>b__1(Object unused)
       MarkPad.exe: N 00043
   System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
       MarkPad.exe: N 00085
   MS.Internal.Threading.ExceptionFilterHelper.TryCatchWhen(Object source, Delegate method, Object args, Int32 numArgs, Delegate catchHandler)
       MarkPad.exe: N 00057

Temporary Solution

In source code, after I change Encoding.Default.GetString(ms.ToArray()); to var jsonString = Encoding.UTF8.GetString(ms.ToArray());, and this works in my win 7, but in other systems or languages are not tested.

    public abstract class JsonSettingsStoreBase : ISettingsStorage
    {
        public string SerializeList(List<string> listOfItems)
        {
            var ms = new MemoryStream();
            var writer = JsonReaderWriterFactory.CreateJsonWriter(ms);
            new DataContractJsonSerializer(typeof(List<string>)).WriteObject(ms, listOfItems);
            writer.Flush();
            var jsonString = Encoding.Default.GetString(ms.ToArray());

            return jsonString;
        }
shiftkey commented 11 years ago

Looks like we suck at parsing specific Unicode files. Let me do some investigating.

JakeGinnivan commented 11 years ago

This is an issue in my settings stuff. i will fix upstream, but because the nuget package puts the classes into the solution it can be fixed inline.

Tracking at https://github.com/JakeGinnivan/SettingsProvider.net/issues/1

shiftkey commented 11 years ago

@JakeGinninvan :sparkles:

JakeGinnivan commented 11 years ago

Upstream fixed

shiftkey commented 11 years ago

Turns out we pulled in the wrong version of SettingsProvider.NET - 1.6.1 will be necessary.

JakeGinnivan commented 11 years ago

Ok, all good to go