Closed maleadt closed 1 year ago
@maleadt Very nice!
Replacing the @eval ccall
with a generated :foreigncall
significantly improves the performance of message calling to the point that it is viable (although we can still improve it by caching the selector lookups, or performing them at parse time if that's possible).
Before:
julia> f() = @objc [[[NSHost currentHost] localizedName] UTF8String]
f (generic function with 1 method)
julia> @benchmark f()
BenchmarkTools.Trial: 1465 samples with 1 evaluation.
Range (min … max): 3.287 ms … 9.281 ms ┊ GC (min … max): 0.00% … 0.00%
Time (median): 3.370 ms ┊ GC (median): 0.00%
Time (mean ± σ): 3.409 ms ± 264.106 μs ┊ GC (mean ± σ): 0.24% ± 2.13%
▅▆▇▇█▇▇▇▇▆▅▄▄▃▃▁ ▁
▇▇████████████████▇████▆▅▆▄▁▄▁▄▅▄▄▁▅▁▄▄▁▄▆▆▆▄▆▅▆▆▆▆▆▅▄▆▆▁▄▄ █
3.29 ms Histogram: log(frequency) by time 3.88 ms <
Memory estimate: 162.00 KiB, allocs estimate: 3229.
After:
julia> @benchmark f()
BenchmarkTools.Trial: 10000 samples with 1 evaluation.
Range (min … max): 30.583 μs … 152.041 μs ┊ GC (min … max): 0.00% … 0.00%
Time (median): 32.500 μs ┊ GC (median): 0.00%
Time (mean ± σ): 33.221 μs ± 3.030 μs ┊ GC (mean ± σ): 0.00% ± 0.00%
▂▆▂▄▅█▄▂
▁▂▆████████▆▅▃▃▃▃▂▂▂▂▂▂▂▂▂▂▂▂▁▁▁▁▁▁▂▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ ▂
30.6 μs Histogram: frequency by time 44.3 μs <
Memory estimate: 2.06 KiB, allocs estimate: 56.
https://gist.github.com/fjolnir/2211379/d77f93efd80690125b933667f79600e67da66c1e is also interesting, where there's caching of class and instance methods, and automatic retain/release based on the function name.
performing them at parse time if that's possible:
Selector lookups are usually done at load time. For instance this is what the metalcpp wrapper is doing:
#define _MTL_PRIVATE_DEF_SEL(accessor, symbol) SEL s_k##accessor _MTL_PRIVATE_VISIBILITY = sel_registerName(symbol);
namespace MTK::Private::Selector
{
_MTK_PRIVATE_DEF_SEL( autoresizeDrawable,
"autoresizeDrawable" );
...
Something similar could be done in an __init__
function for a Julia package, and presumably this is what a wrapper tool would do in generated code.
However, being able to do the selector lookup on the fly with relatively low overhead is quite useful -- for situations where one wants to quickly wrap a portion of a framework by hand and also quick experimentation -- so I find this PR really great.
although we can still improve it by caching the selector lookups
Note: the process of invoking a method in Objective-C is:
So caching selector lookups and checking a hashtable in Julia at runtime is not likely to provide any improvement to just calling sel_registername
on each method invocation (there will be a hahstable lookup either in Julia or C)
the "convert object-c instance and SEL to a concrete method" is probably the largest portion of typical runtime overhead in objective-C, but it is also the source of its dynamism. The runtime does some caching of frequently used methods (https://developer.apple.com/documentation/objectivec/objective-c_runtime/objc_cache?language=objc) to mitigate this. i.e. it probably is neither correct nor necessary for us to do any other performance optimizations.
What the clang compiler also does is static analysis to make sure that objective-c method invocations will not generate runtime errors and that is what a Julia wrapper could do as well (i.e. not improve performance, but help ensure correct use).
Also, a good place to read about the low level details is: https://developer.apple.com/documentation/objectivec/objective-c_runtime?language=objc
However, being able to do the selector lookup on the fly with relatively low overhead is quite useful -- for situations where one wants to quickly wrap a portion of a framework by hand and also quick experimentation -- so I find this PR really great.
I wouldn't remove that, but require type annotations so that we can statically generate code instead of having to introspect into the ObjC runtime in order to build a ccall
expression. I'm exploring that right now, and it does look very promising.
This is how the package currently works:
julia> str = @objc [NSString string]
__NSCFConstantString Object
julia> @objc [str length]
0x0000000000000000
julia> f() = @objc [str length]
f (generic function with 1 method)
Current master branch:
julia> @benchmark f()
BenchmarkTools.Trial: 6509 samples with 1 evaluation.
Range (min … max): 702.417 μs … 3.822 ms ┊ GC (min … max): 0.00% … 50.89%
Time (median): 735.834 μs ┊ GC (median): 0.00%
Time (mean ± σ): 765.605 μs ± 129.942 μs ┊ GC (mean ± σ): 0.29% ± 1.94%
▅███▇▆▅▅▄▃▃▂▂▂ ▁▁ ▁ ▁ ▂
███████████████████▇█▇███▇▇▇▇▇▇▇▇▇▆▇█▇▇▇▆▇▇▆▆▆▇▃▄▅▅▄▅▁▄▅▄▄▁▅▅ █
702 μs Histogram: log(frequency) by time 1.25 ms <
Memory estimate: 59.15 KiB, allocs estimate: 1169.
This PR (removal of @eval
, use of generated ccall):
julia> @benchmark f()
BenchmarkTools.Trial: 10000 samples with 10 evaluations.
Range (min … max): 1.317 μs … 151.921 μs ┊ GC (min … max): 0.00% … 97.31%
Time (median): 1.354 μs ┊ GC (median): 0.00%
Time (mean ± σ): 1.389 μs ± 1.508 μs ┊ GC (mean ± σ): 1.06% ± 0.97%
▁▄▅▆▆█▆▆▅▄▃▃▃▂▁▁▁▁▂▂▂▂▂▂▂▃▁▁ ▁ ▂
▆████████████████████████████████▇▇█▇▇▆█▆▆▇▆▆▇▆▆▆▄▅▅▆▅▅▄▅▅▆ █
1.32 μs Histogram: log(frequency) by time 1.6 μs <
Memory estimate: 648 bytes, allocs estimate: 16.
Proposed static approach:
julia> str = @objc [NSString string]::id
Ptr{ObjectiveC.OpaqueObject} @0x00000001eb6b4d28
julia> @objc [str::id length]::NSUInteger
0x0000000000000000
julia> f() = @objc [str::Id length]::NSUInteger
f (generic function with 1 method)
julia> @benchmark f()
BenchmarkTools.Trial: 10000 samples with 946 evaluations.
Range (min … max): 98.661 ns … 137.024 ns ┊ GC (min … max): 0.00% … 0.00%
Time (median): 99.014 ns ┊ GC (median): 0.00%
Time (mean ± σ): 99.406 ns ± 2.098 ns ┊ GC (mean ± σ): 0.00% ± 0.00%
▃█▇▆▄▂ ▁▁▁▁ ▂
██████▇▆▅▄▃▅▅▅▅▅▄▅▆█████▅▅▅▅▅▄▄▃▃▅▅▅▄▄▄▄▅▄▄▅▅▆▆▃▅▄▅▄▁▁▃▁▃▄▄▄ █
98.7 ns Histogram: log(frequency) by time 109 ns <
Memory estimate: 0 bytes, allocs estimate: 0.
i.e. keeping most of the syntax, only adding some type assertions so that we have the information we need. Note the use of id
, I haven't found a good way of encoding additional type information yet (well, it's obvious to switch ::id
to ::NSString
, but I haven't found a good Object hierarchy yet that makes this possible without too much boilerplate).
Something that may be worth discussing: in https://github.com/JuliaInterop/ObjectiveC.jl/pull/11/commits/a8c859f3f704b460ddb7e287f5b5eda91f70f954 I decided to 'split' the Julia class definitions we generate with @objcwrapper
(a helper macro to generate object classes) into two: an abstract class to implement the hierarchy, and a concrete one that contains the object pointer. The motivation here is that Objective-C has multilevel inheritance, e.g., MTLEvent <: NSObject
is a perfectly valid object, but there's also MTLSharedEvent <: MTLEvent
which augments MTLEvent
. We can't express that in Julia, lacking concrete inheritance, and the traditional alternatives (composition instead of inheritance) break dispatch.
After https://github.com/JuliaInterop/ObjectiveC.jl/pull/11/commits/a8c859f3f704b460ddb7e287f5b5eda91f70f954, we have MTLEvent
be an abstract class so that MTLSharedEvent
can inherit from it, and objects will be put in the MTLEventInstance
or MTLSharedEventInstance
types, depending on the requested type. To make this all invisible, I generate a pseudo constructor for the abstract classes (so calling MTLEvent(ptr::id)
will give an MTLEventInstance
). Case in point, the change did not break the WIP wrappers I have over in https://github.com/JuliaGPU/Metal.jl/pull/117, yet it did allow me to remove the 'fake' MTLAbstractEvent
class I had introduced there. So this looks like a potentially interesting solution for supporting Objective-C's object model (although I may still be missing things, as I'm not really familiar with the language yet and am just designing things in function of the Metal APIs).
I also considered making everything an Object{T}
, but that would make methods ugly (e.g., foo(obj::Object{<:MTLEvent})
instead of now simply foo(obj::MTLEvent)
).
I was thinking about "Type Modeling" -- @maleadt your approach is better than what I was thinking about, and feels correct. I'll spend time looking at the specifics.
In terms of interop completeness, the othervoutstanding issues I can think of are:
MTLSize
, CGRect
) during Objective-C method invocations (maybe this already works?)These are main things I can think of needed to create a modern Mac app (e.g. a game) in Julia -- not that that needs to be the actual goal but it is amazing how much progress has been made!
This might be a good reference for blocks: https://github.com/SSheldon/rust-block, with the goal being an @objcblock macro similar to @cfunction.
Also, assuming but will check @class
macro still works, it will need to support syntax for Protocols, so that Delegates (e.g. AppDelegate
) can be implemented in Julia
Writeup on the calling convention of objc_msgsend
. https://www.mikeash.com/pyblog/objc_msgsends-new-prototype.html
Below is some code that investigates struct parameters and return values.
Some notes: according to https://www.mikeash.com/pyblog/objc_msgsends-new-prototype.html
objc_sendMsg_stret
-- though my code seems to work fine. It is possible the documentation is out of date, we should disassemble some pure ObjectiveC code to check. objc_msgSend
is that for the provided (instance, sel)
pair, it resolves method
(which is just a normal c function pointer, expecting the normal ABI) , and jumps to method
, then method
returns directly to the caller (so this behavior is not possible to implement w/o assembly)ccall
, the call signature of objc_msgSend
is dynamic -- it is the signature of method resolved for (instance, sel)
-- this is why syntax.jl
works. ##
using Libdl
using ObjectiveC
##
function compile(code)
mktempdir() do p
f = joinpath(p,"temp.m")
lib = joinpath(p,"libtemp.$(Libdl.dlext)")
open(f,"w") do io
write(io, code)
end
run(`clang -shared -fPIC -framework Foundation $f -o $lib`)
dlopen(lib)
end
end
macro dylib_str(code)
compile(code)
end
##
dl = dylib"""
#import <Foundation/Foundation.h>
#include <stdint.h>
typedef struct {int64_t a;int64_t b;int64_t c;} Bar;
@interface Foo : NSObject
- (int64_t) m0:(int64_t) x;
- (int64_t) m1:(Bar ) x;
- (Bar ) m2:(Bar ) x;
@end
@implementation Foo {}
- (int64_t) m0:(int64_t) x {return 2*x ;}
- (int64_t) m1:(Bar ) x {return x.a*x.b*x.c ;}
- (Bar ) m2:(Bar ) x {x.a*=2; x.b*=10; x.c*=20;return x;}
@end
"""
##
struct Bar
a::Int64
b::Int64
c::Int64
end
##
Foo = Class("Foo")
foo = @objc [Foo new]::id
##
b = Bar(3,4,5)
@show @objc [foo::id m0 :3::Int64]::Int64
@show @objc [foo::id m1 :b::Bar ]::Int64
@show @objc [foo::id m2 :b::Bar ]::Bar
##⬆ should not work (but looks like it does...), needs to use objc_msgSend_stret
dlclose(dl)
How to define Objective-C blocks in Julia (This seems hard -- the details seemed buried in Clang -- but also important most modern Objective-C apis are moving to blocks instead of delegates. e.g. MtlCommandBuffer hooks)
@tgymnich was looking at blocks yesterday :)
Okay, this is now at a point where it is sufficiently powerful to wrap all of Metal that Metal.jl needs... except for the two onCompleted
APIs that requires blocks. @tgymnich did you find anything out? Clang has documented the ABI, https://clang.llvm.org/docs/Block-ABI-Apple.html, and there's existing FFI implementations like https://crates.io/crates/block, but it looks pretty complicated.
@maleadt @tgymnich the lua implementation of blocks might be a little easier to understad: https://github.com/rweichler/objc.lua/blob/cbe2a80462fdcc270c37482f648135de079040e7/src/init.lua#L1996-L2070
This works in Julia:
using ObjectiveC, .Foundation
using CEnum
## NSBlock infrastructure
export NSBlock
@cenum Block_flags::Cint begin
BLOCK_DEALLOCATING = 0x0001
BLOCK_REFCOUNT_MASK = 0xfffe
BLOCK_IS_NOESCAPE = 1 << 23
BLOCK_NEEDS_FREE = 1 << 24
BLOCK_HAS_COPY_DISPOSE = 1 << 25
BLOCK_HAS_CTOR = 1 << 26
BLOCK_IS_GLOBAL = 1 << 28
BLOCK_HAS_STRET = 1 << 29
BLOCK_HAS_SIGNATURE = 1 << 30
end
const _NSConcreteGlobalBlock = cglobal(:_NSConcreteGlobalBlock)
const _NSConcreteStackBlock = cglobal(:_NSConcreteStackBlock)
struct JuliaBlockDescriptor
reserved::Culong
size::Culong
copy_helper::Ptr{Cvoid}
dispose_helper::Ptr{Cvoid}
end
struct JuliaBlock
isa::Ptr{Cvoid}
flags::Cint
reserved::Cint
invoke::Ptr{Cvoid}
descriptor::Ptr{JuliaBlockDescriptor}
# custom fields
lambda::Function
end
@objcwrapper NSBlock <: NSObject
# NSBlock is the opaque version of JuliaBlock, so make it possible to derive an id pointer
# from a boxed block (assuming the caller doesn't use the pointer beyond the box's lifetime)
Base.unsafe_convert(T::Type{id{Object}}, box::Base.RefValue{JuliaBlock}) =
reinterpret(T, Base.unsafe_convert(Ptr{Cvoid}, box))
## high-level macro
# static descriptor for a simple Julia-based block
const julia_block_descriptor = Ref(JuliaBlockDescriptor(0, sizeof(JuliaBlock), 0, C_NULL))
function julia_block_trampoline(_block, _self, args...)
block = unsafe_load(_block)
nsblock = NSBlock(reinterpret(id{NSBlock}, _block))
# call the user lambda
block.lambda(args...)
end
macro objcblock(callable, rettyp, argtyps)
quote
cb = @cfunction(julia_block_trampoline, $rettyp, (Ptr{JuliaBlock}, id{Object}, $argtyps...))
GC.@preserve cb begin
# set-up the block data structures
trampoline_ptr = Base.unsafe_convert(Ptr{Cvoid}, cb)
desc_ptr = Base.unsafe_convert(Ptr{Cvoid}, julia_block_descriptor)
block = JuliaBlock(_NSConcreteStackBlock, 0, 0, trampoline_ptr, desc_ptr, $callable)
# copy the block to the heap and have Objective-C use that object.
# our original block was a plain Julia struct so doesn't need to be released.
# we also don't need to worry about the lifetime of the block object, as the
# only references to Julia data it contains are the descriptor, which is
# statically allocated, and the lambda (Julia code currently isn't GC'd).
@objc [Ref(block)::id{NSBlock} copy]::id{NSBlock}
end
end
end
## demo
function hello(x)
println("Hello, world $(x)!")
return Cint(42)
end
block = @objcblock(hello, Cint, (Cint,))
# for validation, register our block with a class method
@objcwrapper BlockWrapper <: NSObject
wrapper_class = ObjectiveC.allocclass(:BlockWrapper, Class(:NSObject))
imp = ccall(:imp_implementationWithBlock, Ptr{Cvoid}, (id{NSBlock},), block)
types = (Cint, Object, Selector, Cint)
typestr = ObjectiveC.encodetype(types...)
@assert ccall(:class_addMethod, Bool,
(Ptr{Cvoid}, Ptr{Cvoid}, Ptr{Cvoid}, Ptr{Cchar}),
wrapper_class, sel"invoke:", imp, typestr)
ObjectiveC.register(wrapper_class)
# create a wrapper instance and call our block
wrapper = BlockWrapper(@objc [BlockWrapper alloc]::id{BlockWrapper})
ret = @objc [wrapper::id{BlockWrapper} invoke:41::Cint]::Cint
@show ret
Doesn't do any memory management; I'm not sure we need to if we won't capture things (by setting BLOCK_IS_GLOBAL; we can avoid doing so because of Julia's runtime supporting closures with @cfunction
).
Doesn't do any memory management; I'm not sure we need to if we won't capture things (by setting BLOCK_IS_GLOBAL; we can avoid doing so because of Julia's runtime supporting closures with @cfunction).
This sounds right and probably greatly simplifies everything -- though I was wondering why lua did not make this decision (but don;t really know much about lua)
Emulated closures can be slow, and not widely supported, so if you can use native closures (eventually) that is preferable. But there is not an immediate need for it.
Some type modeling questions (probably for a later PR...)
objcwrapper
how do we specify that the class conforms to a protocol (n.b. conforming to a protocol is just a runtime assertion (you don't have to actually implement any of the methods (i think)) -- so do we even need this?
-@class
init
usuallySomething that may be worth discussing: in a8c859f I decided to 'split' the Julia class definitions we generate with
@objcwrapper
(a helper macro to generate object classes) into two: an abstract class to implement the hierarchy, and a concrete one that contains the object pointer. The motivation here is that Objective-C has multilevel inheritance, e.g.,MTLEvent <: NSObject
is a perfectly valid object, but there's alsoMTLSharedEvent <: MTLEvent
which augmentsMTLEvent
. We can't express that in Julia, lacking concrete inheritance, and the traditional alternatives (composition instead of inheritance) break dispatch.After a8c859f, we have
MTLEvent
be an abstract class so thatMTLSharedEvent
can inherit from it, and objects will be put in theMTLEventInstance
orMTLSharedEventInstance
types, depending on the requested type. To make this all invisible, I generate a pseudo constructor for the abstract classes (so callingMTLEvent(ptr::id)
will give anMTLEventInstance
). Case in point, the change did not break the WIP wrappers I have over in JuliaGPU/Metal.jl#117, yet it did allow me to remove the 'fake'MTLAbstractEvent
class I had introduced there. So this looks like a potentially interesting solution for supporting Objective-C's object model (although I may still be missing things, as I'm not really familiar with the language yet and am just designing things in function of the Metal APIs).I also considered making everything an
Object{T}
, but that would make methods ugly (e.g.,foo(obj::Object{<:MTLEvent})
instead of now simplyfoo(obj::MTLEvent)
).
I faced this challenge in Gtk.jl and ended up doing something similar to this, with GObject
and GObjectLeaf
(making the interface the easier type, similar to Ref
vs. Base.RefValue
). I was never quite satisfied with this though, as it felt awkward, led to excess specialization, was a bit hard to extend in disjoint packages, and never quite mapped properly between the type systems (since calling interface methods requires an explicit type cast in the Julia Gtk.jl wrapper). If I was to revisit that decision, I might have gone with a runtime-traits-based approach instead, perhaps with optional mixins.
mutable struct GObject{Mixin}
classid::Int # or encoded in gptr
gptr::Ptr{Cvoid}
mixins::Mixin
end
Then at runtime, it would be sufficient to do the reflection calls needed to decide if a particular interface was applicable, and perhaps also to provide a custom dispatch table hook for places where it is required. This would have let it provide all of the features (multiple inheritance, concrete inheritance, interfaces with inheritance) that are not viable normally.
Secondly, with the current approach, I think I should have at least implemented many convert-trait methods, and coded them up to be visible to reflection:
abstract type AnyInterface; end
abstract type AnyObject; end
struct Interface{T} <: AnyInterface; obj::T; end
inherits_from(::Type, ::Type) = false
inherits_from(::Type{<:Interface}, ::Type{<:Abstract}) = true
convert(::Type{I}, obj::AnyObject) where {I<:AnyInterface} = I(obj)
then when I auto-generated interface calls such as label
, I could also auto-generate dispatch methods that try to select the best matching ccall:
struct HasLabel{T} <: AnyInterface; obj::T; end
label(obj::HasLabel) = ccall
function label(obj)
inherits_from(HasLabel, obj) && return label(HasLabel(obj))
throw(InterfaceMissingError(label, obj))
end
Thanks for the write-up, I should think about if an how to incorporate that here.
Regarding blocks, I pushed an initial version that simply looks like:
function hello(x)
println("Hello, world $(x)!")
return Cint(42)
end
block = @objcblock(hello, Cint, (Cint,))
This works when attaching the block to a class, now I'll see if it works for Metal. I'm not sure I'm correctly managing memory (i.e., I'm not doing much). As our block doesn't contain any GC-managed data (only a Julia function, and IIRC functions aren't garbage collected), I think I can get away by just copying it to the heap and doing nothing (well, not doing any refcounting). I'm not sure who's responsible for releasing the block object itself though.
; we can avoid doing so because of Julia's runtime supporting closures with
@cfunction
No on AArch64 sadly
; we can avoid doing so because of Julia's runtime supporting closures with
@cfunction
No on AArch64 sadly
I have an alternative, storing the jl_value_t
pointing to the callable (it be a singleton function or a closure struct), but that does require memory management. So I'll look into that.
only a Julia function, and IIRC functions aren't garbage collected
cfunction
is garbage collected (that's why it is unsafe to convert it to a pointer)
cfunction
is garbage collected (that's why it is unsafe to convert it to a pointer)
All of cfunction
, so also when not using closures?
But yeah I'll work on adding proper memory management (probably just rooting the Block in a global dict and evicting it from the Block's dispose callback).
OK, I got the basic functionality to support Metal.jl working, so I think I'll cut a 0.1 release so that I can more easily test this package in a realistic setting. FWIW, I removed the broken bits that I didn't (yet) need, including support for defining classes, so I'll create an issue to keep track of bringing those bits back. But since ObjectiveC.jl hadn't even been tagged, and was completely broken at this point, I don't feel bad removing it for now.
This PR updates and significantly reworks the ObjectiveC.jl package. Basic updates include adding a Project.toml, fixing deprecations, adding some tests, etc. However, I also decided to rework the core calling mechanism to not rely on introspection and dynamic code generation anymore, instead opting for a more static
@ccall
-esque approach that allows for far better performance. It should also combine nicely with e.g. a Clang.jl-based code generator, but that's future work.Reworking the core mechanism probably results in breakage of other functionality in this package, but it's hard to avoid that since there were no tests. In addition, because there hasn't even been a release, and because the package was broken on any recent Julia version anyway, it is probably fine to do so and gradually improve/unbreak functionality along the way.
cc @habemus-papadum