Crash when processing a file with multibyte characters

[manavy]

Hi, I tried to run jazzy with multibyte texts and I found it doesn't work with multibyte texts.

It can be reproduced by parsing this code.

/**
　ほげ
 */
class ClassA {
}

I’ve investigated a little bit and I seems like below. In values from sourcekitd(?) like offset or bodyoffset, a Japanese character is counted as 3. But NSRange in filteredSubstring counts a multibyte character as 1. As a result, the range index will be out of bounds.

*** Terminating app due to uncaught exception 'NSRangeException', reason: '-[__NSCFString substringWithRange:]: Range {31, 17} out of bounds; string length 47'
abort() called
terminating with uncaught exception of type NSException

Application Specific Backtrace 1:
0   CoreFoundation                      0x00007fff9517625c __exceptionPreprocess + 172
1   libobjc.A.dylib                     0x00007fff940e2e75 objc_exception_throw + 43
2   CoreFoundation                      0x00007fff9517610c +[NSException raise:format:] + 204
3   CoreFoundation                      0x00007fff95095b4d -[__NSCFString substringWithRange:] + 125
4   SourceKittenFramework               0x000000010850d4ef _TFE21SourceKittenFrameworkCSo8NSString17filteredSubstringfS0_FTSi3endSi_SS + 415
5   SourceKittenFramework               0x000000010850d5d2 _TToFE21SourceKittenFrameworkCSo8NSString17filteredSubstringfS0_FTSi3endSi_SS + 50
6   SourceKittenFramework               0x0000000108511be8 _TFV21SourceKittenFramework4File16parseDeclarationfS0_FGVSs10DictionarySSP8SwiftXPC16XPCRepresentable__GSqSS_ + 1544
7   SourceKittenFramework               0x0000000108513d5e _TFV21SourceKittenFramework4File17processDictionaryfS0_FTGVSs10DictionarySSP8SwiftXPC16XPCRepresentable__17cursorInfoRequestGSqCSo8NSObject__GS1_SSPS3___ + 1198
8   SourceKittenFramework               0x000000010851cb17 _TFFV21SourceKittenFramework4File15newSubstructureFS0_FTGVSs10DictionarySSP8SwiftXPC16XPCRepresentable__17cursorInfoRequestGSqCSo8NSObject__GSqGSaPS3____U0_FPS3__PS3__ + 471
9   SourceKittenFramework               0x0000000108519a92 _TPA__TFFV21SourceKittenFramework4File15newSubstructureFS0_FTGVSs10DictionarySSP8SwiftXPC16XPCRepresentable__17cursorInfoRequestGSqCSo8NSObject__GSqGSaPS3____U0_FPS3__PS3__ + 226
10  libswiftCore.dylib                  0x0000000108621426 _TFVSs17MapCollectionViewg9subscriptFQQ_5IndexQ0_ + 342
11  libswiftCore.dylib                  0x0000000108623727 _TTWVSs17MapCollectionViewSs14CollectionTypeFS0_g9subscriptFQQPS0_5IndexQQS1_9Generator7Element + 935
12  libswiftCore.dylib                  0x000000010861ec0e _TTWVSs26LazyRandomAccessCollectionSs14CollectionTypeFS0_g9subscriptFQQPS0_5IndexQQS1_9Generator7Element + 478
13  libswiftCore.dylib                  0x0000000108584d59 _TFSs34_copyCollectionToNativeArrayBufferUSs14CollectionType_USs13GeneratorType__Ss16ForwardIndexType_Ss18_SignedIntegerType_Ss33_BuiltinIntegerLiteralConvertible___FQ_GVSs22_ContiguousArrayBufferQQ_8_Element_ + 873
14  libswiftCore.dylib                  0x000000010861e9f6 _TTWVSs26LazyRandomAccessCollectionSs12SequenceTypeFS0_oi2tgUS0__USs13GeneratorType___fMQPS0_FTS2_TVSs24_CopyToNativeArrayBufferT___GVSs22_ContiguousArrayBufferQQS2_9Generator7Element_ + 406
15  libswiftCore.dylib                  0x00000001085a3593 _TFSa3mapU__fGSaQ__U__FFQd__Q_GSaQ__ + 1283
16  SourceKittenFramework               0x0000000108515c16 _TFV21SourceKittenFramework4File15newSubstructurefS0_FTGVSs10DictionarySSP8SwiftXPC16XPCRepresentable__17cursorInfoRequestGSqCSo8NSObject__GSqGSaPS3____ + 918
17  SourceKittenFramework               0x000000010851423c _TFV21SourceKittenFramework4File17processDictionaryfS0_FTGVSs10DictionarySSP8SwiftXPC16XPCRepresentable__17cursorInfoRequestGSqCSo8NSObject__GS1_SSPS3___ + 2444
18  SourceKittenFramework               0x00000001084f7232 _TFV21SourceKittenFramework9SwiftDocsCfMS0_FT4fileVS_4File10dictionaryGVSs10DictionarySSP8SwiftXPC16XPCRepresentable__17cursorInfoRequestGSqCSo8NSObject__S0_ + 578
19  SourceKittenFramework               0x00000001084f6f89 _TFV21SourceKittenFramework9SwiftDocsCfMS0_FT4fileVS_4File9argumentsGSaSS__S0_ + 521
20  sourcekitten                        0x000000010847ab6f _TFV12sourcekitten10DocCommand18runSwiftSingleFilefMS0_FGSaSS_GO8LlamaKit6ResultT__ + 2127
21  sourcekitten                        0x000000010847c6e2 _TFFV12sourcekitten10DocCommand3runFS0_FO10Commandant11CommandModeGO8LlamaKit6ResultT__U_FVS_10DocOptionsGS4_T__ + 338
22  sourcekitten                        0x000000010847cae8 _TTRXFo_oV12sourcekitten10DocOptions_oGO8LlamaKit6ResultT___XFo_iS0__oGS2_T___ + 56
23  LlamaKit                            0x000000010856574e _TFO8LlamaKit6Result7flatMapU__fGS0_Q__U__FFQd__GS0_Q__GS0_Q__ + 334
24  sourcekitten                        0x0000000108479559 _TFV12sourcekitten10DocCommand3runfS0_FO10Commandant11CommandModeGO8LlamaKit6ResultT__ + 233
25  sourcekitten                        0x000000010847bf17 _TTWV12sourcekitten10DocCommand10Commandant11CommandTypeFS2_3runUS2___fRQPS2_FOS1_11CommandModeGO8LlamaKit6ResultT__ + 119
26  Commandant                          0x00000001084c5ddf _TFC10Commandant15CommandRegistry10runCommandfS0_FTSS9argumentsGSaSS__GSqGO8LlamaKit6ResultT___ + 431
27  Commandant                          0x00000001084c69e1 _TFC10Commandant15CommandRegistry4mainfS0_FT14defaultCommandPS_11CommandType_12errorHandlerFCSo7NSErrorT__T_ + 1825
28  sourcekitten                        0x0000000108487e7e top_level_code + 1630
29  sourcekitten                        0x0000000108487eaa main + 42
30  libdyld.dylib                       0x00007fff8c85d5fd start + 1
31  ???                                 0x0000000000000033 0x0 + 51

[segiddins]

Yay grapheme clusters vs bytes!

[jpsim]

This is now fixed in the swift1p2 branch, but relies on new functions in Swift 1.2 (mostly samePositionIn(characters:)) so I won't be porting it over to master any time soon. Here's to hoping Apple officially releases Xcode 6.3 soon!

[jpsim]

This is now fixed in master.