[InputMethodKit][Mac] Bindings for the framework are missing.

[mandel-macaque]

InputMethodKit bindings are missing from Xamarin.Mac.

A PR was created but it has been been closed and uncompleted due to outstanding issues.

If this API is important you, please comment on this issues with details on your use case.

See also:

• 19462.

• https://github.com/xamarin/xamarin-macios/pull/19462#issuecomment-1956917706

[ShikiSuen]

I need to rewrite an input method in CSharp but found that Visual Studio 4 mac 8.10.17 doesn't have InputMethodKit.

[ShikiSuen]

@ShikiSuen said: I tried to add this C# file in a new VS2022 mac project. However, it bugs for error CS0246 with "BaseType".

@Chamons said: Sometimes those show up in binding projects but not in actual builds, do they show up in a build error?

In any case, binding built in frameworks with a binding project is going to give you some difficulty for sure.

This is tracked in #7487 as noted above, please comment there.

Yeah. It bugs a lot during build process:

[ShikiSuen]

By the way I don't know why RemObjects Fire can allow C# utilize InputMethodKit:

[chamons]

There appears to be some limited API bindings already created, but they are not enabled in the build as they are not complete. This issue is open to note the needed future work to enable it.

I believe RemObjects uses a different binding stack, which is why you might be seeing a difference here.

[ShikiSuen]

@chamons Thanks for your response.

So, may I confirm that, at this moment, is it impossible for general .NET / VSmac users to enable IMK support in C# through non-RemObjects approach?

RemObjects' header file is here for your possible curiosity:

// Import of InputMethodKit (12.3)
// Frameworks: InputMethodKit
// Targets: arm64, x86_64
// Dependent fx:Cocoa, Carbon, Foundation, rtl, AppKit, Carbon.HIToolbox
// Dependent libraries:
// Platform: Darwin
// 

namespace InputMethodKit
{
    class InputMethodKit.__Global
    {
        public static NSString IMKModeDictionary;
        public static NSString IMKControllerClass;
        public static NSString IMKDelegateClass;
        public static NSString kIMKCommandMenuItemName;
        public static NSString kIMKCommandClientName;
        public static const Int32 kIMKSingleColumnScrollingCandidatePanel;
        public static const Int32 kIMKScrollingGridCandidatePanel;
        public static const Int32 kIMKSingleRowSteppingCandidatePanel;
        public static const Int32 kIMKMain;
        public static const Int32 kIMKAnnotation;
        public static const Int32 kIMKSubList;
        public static const Int32 kIMKLocateCandidatesAboveHint;
        public static const Int32 kIMKLocateCandidatesBelowHint;
        public static const Int32 kIMKLocateCandidatesLeftHint;
        public static const Int32 kIMKLocateCandidatesRightHint;
        public static NSString IMKCandidatesOpacityAttributeName;
        public static NSString IMKCandidatesSendServerKeyEventFirst;
    }

    class InputMethodKit.IMKServer : NSObject
    {
        private id _private;
        [NonSwiftOnly]
        public id initWithName(NSString name) bundleIdentifier(NSString bundleIdentifier);
        [InitFromClassFactoryMethod]
        [Alias]
        [SwiftOnly]
        public static this withName(NSString name) bundleIdentifier(NSString bundleIdentifier);
        [NonSwiftOnly]
        public id initWithName(NSString name) controllerClass(Class controllerClassID) delegateClass(Class delegateClassID);
        [InitFromClassFactoryMethod]
        [Alias]
        [SwiftOnly]
        public static this withName(NSString name) controllerClass(Class controllerClassID) delegateClass(Class delegateClassID);
        public NSBundle bundle();
        public BOOL paletteWillTerminate();
        public BOOL lastKeyEventWasDeadKey();
    }

    [Category]
    class InputMethodKit.IMKServerInputCategory : NSObject
    {
        public BOOL inputText(NSString @string) key(NSInteger keyCode) modifiers(NSUInteger flags) client(id sender);
        public BOOL inputText(NSString @string) client(id sender);
        [NonSwiftOnly]
        public BOOL handleEvent(NSEvent @event) client(id sender);
        [Alias]
        [SwiftOnly]
        public BOOL handle(NSEvent @event) client(id sender);
        [NonSwiftOnly]
        public BOOL didCommandBySelector(SEL aSelector) client(id sender);
        [Alias]
        [SwiftOnly]
        public BOOL didCommand(SEL aSelector) client(id sender);
        public id composedString(id sender);
        public NSAttributedString originalString(id sender);
        public void commitComposition(id sender);
        public NSArray<id> candidates(id sender);
    }

    interface InputMethodKit.IIMKStateSetting
    {
        void activateServer(id sender);
        void deactivateServer(id sender);
        [NonSwiftOnly]
        id valueForTag(Int64 tag) client(id sender);
        [Alias]
        [SwiftOnly]
        id @value(Int64 tag) client(id sender);
        void setValue(id @value) forTag(Int64 tag) client(id sender);
        NSDictionary<id,id> modes(id sender);
        NSUInteger recognizedEvents(id sender);
        void showPreferences(id sender);
    }

    interface InputMethodKit.IIMKMouseHandling
    {
        [NonSwiftOnly]
        BOOL mouseDownOnCharacterIndex(NSUInteger index) coordinate(NSPoint point) withModifier(NSUInteger flags) continueTracking(BOOL* keepTracking) client(id sender);
        [Alias]
        [SwiftOnly]
        BOOL mouseDown(NSUInteger index) coordinate(NSPoint point) withModifier(NSUInteger flags) continueTracking(BOOL* keepTracking) client(id sender);
        [NonSwiftOnly]
        BOOL mouseUpOnCharacterIndex(NSUInteger index) coordinate(NSPoint point) withModifier(NSUInteger flags) client(id sender);
        [Alias]
        [SwiftOnly]
        BOOL mouseUp(NSUInteger index) coordinate(NSPoint point) withModifier(NSUInteger flags) client(id sender);
        [NonSwiftOnly]
        BOOL mouseMovedOnCharacterIndex(NSUInteger index) coordinate(NSPoint point) withModifier(NSUInteger flags) client(id sender);
        [Alias]
        [SwiftOnly]
        BOOL mouseMoved(NSUInteger index) coordinate(NSPoint point) withModifier(NSUInteger flags) client(id sender);
    }

    class InputMethodKit.IMKInputController : NSObject, InputMethodKit.IIMKStateSetting, InputMethodKit.IIMKMouseHandling
    {
        private id _private;
        [NonSwiftOnly]
        public id initWithServer(InputMethodKit.IMKServer server) @delegate(id @delegate) client(id inputClient);
        [InitFromClassFactoryMethod]
        [Alias]
        [SwiftOnly]
        public static this withServer(InputMethodKit.IMKServer server) @delegate(id @delegate) client(id inputClient);
        public void updateComposition();
        public void cancelComposition();
        [NonSwiftOnly]
        public NSMutableDictionary<id,id> compositionAttributesAtRange(NSRange range);
        [Alias]
        [SwiftOnly]
        public NSMutableDictionary<id,id> compositionAttributes(NSRange range);
        public NSRange selectionRange();
        public NSRange replacementRange();
        [NonSwiftOnly]
        public NSDictionary<id,id> markForStyle(NSInteger style) atRange(NSRange range);
        [Alias]
        [SwiftOnly]
        public NSDictionary<id,id> mark(NSInteger style) at(NSRange range);
        [NonSwiftOnly]
        public void doCommandBySelector(SEL aSelector) commandDictionary(NSDictionary<id,id> infoDictionary);
        [Alias]
        [SwiftOnly]
        public void doCommand(SEL aSelector) commandDictionary(NSDictionary<id,id> infoDictionary);
        public void hidePalettes();
        public NSMenu menu();
        public InputMethodKit.IMKServer server();
        public UNKNON_CONSTRAINED_TYPE<id,IIMKTextInput,INSObject> client();
        public void inputControllerWillClose();
        public void annotationSelected(NSAttributedString annotationString) forCandidate(NSAttributedString candidateString);
        public void candidateSelectionChanged(NSAttributedString candidateString);
        public void candidateSelected(NSAttributedString candidateString);
        public id @delegate { get; set; }
    }

    using IMKCandidatePanelType = NSUInteger;

    using IMKStyleType = NSUInteger;

    using IMKCandidatesLocationHint = NSUInteger;

    class InputMethodKit.IMKCandidates : NSResponder
    {
        protected id _private;
        [NonSwiftOnly]
        public id initWithServer(InputMethodKit.IMKServer server) panelType(IMKCandidatePanelType panelType);
        [InitFromClassFactoryMethod]
        [Alias]
        [SwiftOnly]
        public static this withServer(InputMethodKit.IMKServer server) panelType(IMKCandidatePanelType panelType);
        [NonSwiftOnly]
        public id initWithServer(InputMethodKit.IMKServer server) panelType(IMKCandidatePanelType panelType) styleType(IMKStyleType style);
        [InitFromClassFactoryMethod]
        [Alias]
        [SwiftOnly]
        public static this withServer(InputMethodKit.IMKServer server) panelType(IMKCandidatePanelType panelType) styleType(IMKStyleType style);
        public void show(IMKCandidatesLocationHint locationHint);
        public void hide();
        public BOOL isVisible();
        [NonSwiftOnly]
        public void updateCandidates();
        [Alias]
        [SwiftOnly]
        public void update();
        public void showAnnotation(NSAttributedString annotationString);
        public void showSublist(NSArray<id> candidates) subListDelegate(id @delegate);
        public NSRect candidateFrame();
        public NSInteger selectedCandidate();
        public void setCandidateFrameTopLeft(NSPoint point);
        public void showChild();
        public void hideChild();
        public void attachChild(InputMethodKit.IMKCandidates child) toCandidate(NSInteger candidateIdentifier) type(IMKStyleType theType);
        public void detachChild(NSInteger candidateIdentifier);
        public void setCandidateData(NSArray<id> candidatesArray);
        [NonSwiftOnly]
        public BOOL selectCandidateWithIdentifier(NSInteger candidateIdentifier);
        [Alias]
        [SwiftOnly]
        public BOOL selectCandidate(NSInteger candidateIdentifier);
        public void selectCandidate(NSInteger candidateIdentifier);
        [NonSwiftOnly]
        public void showCandidates();
        [Alias]
        [SwiftOnly]
        public void show();
        public NSInteger candidateStringIdentifier(id candidateString);
        public NSAttributedString selectedCandidateString();
        [NonSwiftOnly]
        public NSInteger candidateIdentifierAtLineNumber(NSInteger lineNumber);
        [Alias]
        [SwiftOnly]
        public NSInteger candidateIdentifier(NSInteger lineNumber);
        [NonSwiftOnly]
        public NSInteger lineNumberForCandidateWithIdentifier(NSInteger candidateIdentifier);
        [Alias]
        [SwiftOnly]
        public NSInteger lineNumberForCandidate(NSInteger candidateIdentifier);
        public void clearSelection();
        public IMKCandidatePanelType panelType { get; set; }
        public NSArray<id> selectionKeys { get; set; }
        public TISInputSourceRef selectionKeysKeylayout { get; set; }
        public NSDictionary<id,id> attributes { get; set; }
        public BOOL dismissesAutomatically { get; set; }
        public NSPoint candidateFrameTopLeft { get; set; }
        public NSArray<id> candidateData { get; set; }
    }

    using IMKStateSetting = InputMethodKit.IIMKStateSetting;

    using IMKMouseHandling = InputMethodKit.IIMKMouseHandling;

}

[chamons]

Yes, the reason for this current issue is that we have not completed that binding, thus it is unavailable.

There are workarounds you could do, such as writing a portion of your application in Objective-C and binding it to C#, but I agree that is not ideal.

[ShikiSuen]

@chamons Is it possible in current MS toolchain to let a Swift class inherit a C# class? I am thinking of doing IMK-related things in Swift instead.

[chamons]

There is very basic prototype support here for combining Swift and C#, but inheritance across languages would be difficult at best, and possibly impossible. I have not looked into it yet.

If you were going to go cross language, I would create the required functionality in Swift or Objective-C and keep the cross language interaction to a minimum, creating the required objects in C# using a binding and passing in a simple string or structure describing the data needed for the native code's operation. The more complex the message passing, or even inheritance, the more difficult the infrastructure will be to create.

[ShikiSuen]

@chamons After recent reverse-engineering and manual tests with inputMethodKit.framework from macOS 10.11 and 10.15, I can pretty sure that this entire framework is plagued with bugs. It appears that not all API officially exposed to the public are guaranteed with unit tests, 'cause some of those API functions do not work at all. Especially, IMKCandidates, the most terrible thing in this framework, is totally not usable since the born of inputMethodKit in macOS 10.05 Leopard (without deep hack). Imagine that:

1. it doesn't even respond to default candidates.
2. its candidate key layout mapping function doesn't work, nor its candidate selection key specifier function.
3. when you are trying to grab the identifier of the currently-selected candidate, you always get an NSNotFound. I don't know whether it is supposed to return the sequential index or another useful integer randomly-assigned by the IMK.

I guess that the Xamarin version of the IMK, if possible, can directly port everything except IMKCandidates. As long as you Microsoft can provide a beta version, I will find some time to see whether the port works well.

[chamons]

So this issue is tracking the fact that a PR was created in 2017 and abandoned as incomplete and requiring rework before it could continue.

The fact that the Apple API is still half-baked suggests investing additional time in reviving the effort could be better invested elsewhere. I'm going to leave this open, but given Xcode 14 and other efforts I would expect a community effort would be required to get this moving in the short/medium term.

[ShikiSuen]

I removed my previous mistaken comment. Reason: I forgot to set my current dir to somewhere writeable.

[ShikiSuen]

I hope that my choice of Object-Sharpie is correct. Here's the parsed headers:

using System;
using AppKit;
using CoreGraphics;
using Foundation;
using InputMethodKit;
using ObjCRuntime;

[Static]
[Verify (ConstantsInterfaceAssociation)]
partial interface Constants
{
    // extern const NSString * IMKModeDictionary;
    [Field ("IMKModeDictionary")]
    NSString IMKModeDictionary { get; }

    // extern const NSString * IMKControllerClass;
    [Field ("IMKControllerClass")]
    NSString IMKControllerClass { get; }

    // extern const NSString * IMKDelegateClass;
    [Field ("IMKDelegateClass")]
    NSString IMKDelegateClass { get; }
}

// @interface IMKServer : NSObject <IMKServerProxy>
[BaseType (typeof(NSObject))]
interface IMKServer : IIMKServerProxy
{
    // -(id)initWithName:(NSString *)name bundleIdentifier:(NSString *)bundleIdentifier;
    [Export ("initWithName:bundleIdentifier:")]
    NativeHandle Constructor (string name, string bundleIdentifier);

    // -(id)initWithName:(NSString *)name controllerClass:(Class)controllerClassID delegateClass:(Class)delegateClassID;
    [Export ("initWithName:controllerClass:delegateClass:")]
    NativeHandle Constructor (string name, Class controllerClassID, Class delegateClassID);

    // -(NSBundle *)bundle;
    [Export ("bundle")]
    [Verify (MethodToProperty)]
    NSBundle Bundle { get; }

    // -(BOOL)paletteWillTerminate __attribute__((availability(macos, introduced=10.7)));
    [Mac (10, 7)]
    [Export ("paletteWillTerminate")]
    [Verify (MethodToProperty)]
    bool PaletteWillTerminate { get; }

    // -(BOOL)lastKeyEventWasDeadKey __attribute__((availability(macos, introduced=10.7)));
    [Mac (10, 7)]
    [Export ("lastKeyEventWasDeadKey")]
    [Verify (MethodToProperty)]
    bool LastKeyEventWasDeadKey { get; }
}

[Static]
[Verify (ConstantsInterfaceAssociation)]
partial interface Constants
{
    // extern const NSString * kIMKCommandMenuItemName;
    [Field ("kIMKCommandMenuItemName")]
    NSString kIMKCommandMenuItemName { get; }

    // extern const NSString * kIMKCommandClientName;
    [Field ("kIMKCommandClientName")]
    NSString kIMKCommandClientName { get; }
}

// @interface IMKServerInput (NSObject)
[Category]
[BaseType (typeof(NSObject))]
interface NSObject_IMKServerInput
{
    // -(BOOL)inputText:(NSString *)string key:(NSInteger)keyCode modifiers:(NSUInteger)flags client:(id)sender;
    [Export ("inputText:key:modifiers:client:")]
    bool InputText (string @string, nint keyCode, nuint flags, NSObject sender);

    // -(BOOL)inputText:(NSString *)string client:(id)sender;
    [Export ("inputText:client:")]
    bool InputText (string @string, NSObject sender);

    // -(BOOL)handleEvent:(NSEvent *)event client:(id)sender;
    [Export ("handleEvent:client:")]
    bool HandleEvent (NSEvent @event, NSObject sender);

    // -(BOOL)didCommandBySelector:(SEL)aSelector client:(id)sender;
    [Export ("didCommandBySelector:client:")]
    bool DidCommandBySelector (Selector aSelector, NSObject sender);

    // -(id)composedString:(id)sender;
    [Export ("composedString:")]
    NSObject ComposedString (NSObject sender);

    // -(NSAttributedString *)originalString:(id)sender;
    [Export ("originalString:")]
    NSAttributedString OriginalString (NSObject sender);

    // -(void)commitComposition:(id)sender;
    [Export ("commitComposition:")]
    void CommitComposition (NSObject sender);

    // -(NSArray *)candidates:(id)sender;
    [Export ("candidates:")]
    [Verify (StronglyTypedNSArray)]
    NSObject[] Candidates (NSObject sender);
}

// @protocol IMKStateSetting
/*
  Check whether adding [Model] to this declaration is appropriate.
  [Model] is used to generate a C# class that implements this protocol,
  and might be useful for protocols that consumers are supposed to implement,
  since consumers can subclass the generated class instead of implementing
  the generated interface. If consumers are not supposed to implement this
  protocol, then [Model] is redundant and will generate code that will never
  be used.
*/[Protocol]
interface IMKStateSetting
{
    // @required -(void)activateServer:(id)sender;
    [Abstract]
    [Export ("activateServer:")]
    void ActivateServer (NSObject sender);

    // @required -(void)deactivateServer:(id)sender;
    [Abstract]
    [Export ("deactivateServer:")]
    void DeactivateServer (NSObject sender);

    // @required -(id)valueForTag:(long)tag client:(id)sender;
    [Abstract]
    [Export ("valueForTag:client:")]
    NSObject ValueForTag (nint tag, NSObject sender);

    // @required -(void)setValue:(id)value forTag:(long)tag client:(id)sender;
    [Abstract]
    [Export ("setValue:forTag:client:")]
    void SetValue (NSObject value, nint tag, NSObject sender);

    // @required -(NSDictionary *)modes:(id)sender;
    [Abstract]
    [Export ("modes:")]
    NSDictionary Modes (NSObject sender);

    // @required -(NSUInteger)recognizedEvents:(id)sender;
    [Abstract]
    [Export ("recognizedEvents:")]
    nuint RecognizedEvents (NSObject sender);

    // @required -(void)showPreferences:(id)sender;
    [Abstract]
    [Export ("showPreferences:")]
    void ShowPreferences (NSObject sender);
}

// @protocol IMKMouseHandling
/*
  Check whether adding [Model] to this declaration is appropriate.
  [Model] is used to generate a C# class that implements this protocol,
  and might be useful for protocols that consumers are supposed to implement,
  since consumers can subclass the generated class instead of implementing
  the generated interface. If consumers are not supposed to implement this
  protocol, then [Model] is redundant and will generate code that will never
  be used.
*/[Protocol]
interface IMKMouseHandling
{
    // @required -(BOOL)mouseDownOnCharacterIndex:(NSUInteger)index coordinate:(NSPoint)point withModifier:(NSUInteger)flags continueTracking:(BOOL *)keepTracking client:(id)sender;
    [Abstract]
    [Export ("mouseDownOnCharacterIndex:coordinate:withModifier:continueTracking:client:")]
    unsafe bool MouseDownOnCharacterIndex (nuint index, CGPoint point, nuint flags, bool* keepTracking, NSObject sender);

    // @required -(BOOL)mouseUpOnCharacterIndex:(NSUInteger)index coordinate:(NSPoint)point withModifier:(NSUInteger)flags client:(id)sender;
    [Abstract]
    [Export ("mouseUpOnCharacterIndex:coordinate:withModifier:client:")]
    bool MouseUpOnCharacterIndex (nuint index, CGPoint point, nuint flags, NSObject sender);

    // @required -(BOOL)mouseMovedOnCharacterIndex:(NSUInteger)index coordinate:(NSPoint)point withModifier:(NSUInteger)flags client:(id)sender;
    [Abstract]
    [Export ("mouseMovedOnCharacterIndex:coordinate:withModifier:client:")]
    bool MouseMovedOnCharacterIndex (nuint index, CGPoint point, nuint flags, NSObject sender);
}

// @interface IMKInputController : NSObject <IMKStateSetting, IMKMouseHandling>
[BaseType (typeof(NSObject))]
interface IMKInputController : IIMKStateSetting, IIMKMouseHandling
{
    // -(id)initWithServer:(IMKServer *)server delegate:(id)delegate client:(id)inputClient;
    [Export ("initWithServer:delegate:client:")]
    NativeHandle Constructor (IMKServer server, NSObject @delegate, NSObject inputClient);

    // -(void)updateComposition;
    [Export ("updateComposition")]
    void UpdateComposition ();

    // -(void)cancelComposition;
    [Export ("cancelComposition")]
    void CancelComposition ();

    // -(NSMutableDictionary *)compositionAttributesAtRange:(NSRange)range;
    [Export ("compositionAttributesAtRange:")]
    NSMutableDictionary CompositionAttributesAtRange (NSRange range);

    // -(NSRange)selectionRange;
    [Export ("selectionRange")]
    [Verify (MethodToProperty)]
    NSRange SelectionRange { get; }

    // -(NSRange)replacementRange;
    [Export ("replacementRange")]
    [Verify (MethodToProperty)]
    NSRange ReplacementRange { get; }

    // -(NSDictionary *)markForStyle:(NSInteger)style atRange:(NSRange)range;
    [Export ("markForStyle:atRange:")]
    NSDictionary MarkForStyle (nint style, NSRange range);

    // -(void)doCommandBySelector:(SEL)aSelector commandDictionary:(NSDictionary *)infoDictionary;
    [Export ("doCommandBySelector:commandDictionary:")]
    void DoCommandBySelector (Selector aSelector, NSDictionary infoDictionary);

    // -(void)hidePalettes;
    [Export ("hidePalettes")]
    void HidePalettes ();

    // -(NSMenu *)menu;
    [Export ("menu")]
    [Verify (MethodToProperty)]
    NSMenu Menu { get; }

    // -(id)delegate;
    // -(void)setDelegate:(id)newDelegate;
    [Export ("delegate")]
    [Verify (MethodToProperty)]
    NSObject Delegate { get; set; }

    // -(IMKServer *)server;
    [Export ("server")]
    IMKServer Server ();

    // -(id<IMKTextInput,NSObject>)client;
    [Export ("client")]
    NSObject<IMKTextInput, NSObject> Client ();

    // -(void)inputControllerWillClose __attribute__((availability(macos, introduced=10.7)));
    [Mac (10,7)]
    [Export ("inputControllerWillClose")]
    void InputControllerWillClose ();

    // -(void)annotationSelected:(NSAttributedString *)annotationString forCandidate:(NSAttributedString *)candidateString;
    [Export ("annotationSelected:forCandidate:")]
    void AnnotationSelected (NSAttributedString annotationString, NSAttributedString candidateString);

    // -(void)candidateSelectionChanged:(NSAttributedString *)candidateString;
    [Export ("candidateSelectionChanged:")]
    void CandidateSelectionChanged (NSAttributedString candidateString);

    // -(void)candidateSelected:(NSAttributedString *)candidateString;
    [Export ("candidateSelected:")]
    void CandidateSelected (NSAttributedString candidateString);
}

[ShikiSuen]

So, after years of awaiting, the result is the complete removal of IMK support? Sigh.