Reading from arguments doesn't work with bpftrace

[vchuravy]

julia> using UProbes

julia> function f(arg)
           if @query(:julia, :test, typeof(arg))
               @probe(:julia, :test, arg)
           end
       end
f (generic function with 1 method)

julia> while true
           i = rand(Int64)
           f(i)
       end

➜  ~ sudo bpftrace -p 31417 -e "usdt:julia:test { @[comm] = hist(arg0); }"
Attaching 1 probe...
Code not generated for probe: usdt:/tmp/jl_bnvl90:test

➜  ~ readelf -n /tmp/jl_bnvl90 

Displaying notes found in: .note.stapsdt
  Owner                 Data size       Description
  stapsdt              0x0000002f       NT_STAPSDT (SystemTap probe descriptors)
    Provider: julia
    Name: test
    Location: 0x0000000000001005, Base: 0x0000000000002000, Semaphore: 0x0000000000004000
    Arguments: -8@-8(%rsp)

[vchuravy]

@nhdaly would be grand if you could try and reproduce this on Mac. (I currently don't even know if these are what dtrace on mac expects)

[NHDaly]

Unfortunately the asm didn't parse on my mac:

julia> function f(arg)
           if @query(:julia, :test, typeof(arg))
               @probe(:julia, :test, arg)
           end
       end
f (generic function with 1 method)

julia> while true
           i = rand(Int64)
           f(i)
       end
<inline asm>:2:22: error: mach-o section specifier uses an unknown section type
        .pushsection .note.stapsdt,"?","note"
                     ^
<inline asm>:14:20: error: .popsection without corresponding .pushsection
        .popsection
                   ^
<inline asm>:16:22: error: mach-o section specifier uses an unknown section type
        .pushsection .stapsdt.base,"aG","progbits",.stapsdt.base,comdat
                     ^
<inline asm>:17:9: error: unknown directive
        .weak _.stapsdt.base
        ^
<inline asm>:18:9: error: unknown directive
        .hidden _.stapsdt.base
        ^
<inline asm>:20:9: error: unknown directive
        .size _.stapsdt.base, 1
        ^
<inline asm>:21:20: error: .popsection without corresponding .pushsection
        .popsection
                   ^
<inline asm>:2:22: error: mach-o section specifier uses an unknown section type
        .pushsection .note.stapsdt,"?","note"
                     ^
<inline asm>:14:20: error: .popsection without corresponding .pushsection
        .popsection
                   ^
<inline asm>:16:22: error: mach-o section specifier uses an unknown section type
        .pushsection .stapsdt.base,"aG","progbits",.stapsdt.base,comdat
                     ^
<inline asm>:17:9: error: unknown directive
        .weak _.stapsdt.base
        ^
<inline asm>:18:9: error: unknown directive
        .hidden _.stapsdt.base
        ^
<inline asm>:20:9: error: unknown directive
        .size _.stapsdt.base, 1
        ^
<inline asm>:21:20: error: .popsection without corresponding .pushsection
        .popsection
                   ^
<inline asm>:2:22: error: mach-o section specifier uses an unknown section type
        .pushsection .note.stapsdt,"?","note"
                     ^
<inline asm>:14:20: error: .popsection without corresponding .pushsection
        .popsection
                   ^
<inline asm>:16:22: error: mach-o section specifier uses an unknown section type
        .pushsection .stapsdt.base,"aG","progbits",.stapsdt.base,comdat
                     ^
<inline asm>:17:9: error: unknown directive
        .weak _.stapsdt.base
        ^
<inline asm>:18:9: error: unknown directive
        .hidden _.stapsdt.base
        ^
<inline asm>:20:9: error: unknown directive
        .size _.stapsdt.base, 1
        ^
<inline asm>:21:20: error: .popsection without corresponding .pushsection
        .popsection
                   ^
<inline asm>:2:22: error: mach-o section specifier uses an unknown section type
        .pushsection .note.stapsdt,"?","note"
                     ^
<inline asm>:14:20: error: .popsection without corresponding .pushsection
        .popsection
                   ^
<inline asm>:16:22: error: mach-o section specifier uses an unknown section type
        .pushsection .stapsdt.base,"aG","progbits",.stapsdt.base,comdat
                     ^
<inline asm>:17:9: error: unknown directive
        .weak _.stapsdt.base
        ^
<inline asm>:18:9: error: unknown directive
        .hidden _.stapsdt.base
        ^
<inline asm>:20:9: error: unknown directive
        .size _.stapsdt.base, 1
        ^
<inline asm>:21:20: error: .popsection without corresponding .pushsection
        .popsection
                   ^
ERROR: LLVM error: Error parsing inline asm

Stacktrace:
 [1] handle_error(::Cstring) at /Users/nathan.daly/.julia/packages/LLVM/tg8MX/src/core/context.jl:103
 [2] macro expansion at /Users/nathan.daly/.julia/packages/LLVM/tg8MX/src/base.jl:18 [inlined]
 [3] LLVMTargetMachineEmitToFile(::Ptr{LLVM.API.LLVMOpaqueTargetMachine}, ::Ptr{LLVM.API.LLVMOpaqueModule}, ::String, ::UInt32, ::Base.RefValue{Cstring}) at /Users/nathan.daly/.julia/packages/LLVM/tg8MX/lib/6.0/libLLVM_h.jl:297
 [4] emit at /Users/nathan.daly/.julia/packages/LLVM/tg8MX/src/targetmachine.jl:57 [inlined]
 [5] (::getfield(UProbes, Symbol("##9#13")){LLVM.Module,String})(::LLVM.TargetMachine) at /Users/nathan.daly/.julia/packages/UProbes/199dD/src/UProbes.jl:164
 [6] LLVM.TargetMachine(::getfield(UProbes, Symbol("##9#13")){LLVM.Module,String}, ::LLVM.Target, ::Vararg{Any,N} where N) at /Users/nathan.daly/.julia/packages/LLVM/tg8MX/src/targetmachine.jl:24
 [7] #emit_probe#5(::String, ::typeof(UProbes.emit_probe), ::Symbol, ::Symbol, ::Tuple{DataType}) at /Users/nathan.daly/.julia/packages/UProbes/199dD/src/UProbes.jl:163
 [8] emit_probe at /Users/nathan.daly/.julia/packages/UProbes/199dD/src/UProbes.jl:93 [inlined]
 [9] cache_dl(::Symbol, ::Symbol, ::Tuple{DataType}) at /Users/nathan.daly/.julia/packages/UProbes/199dD/src/UProbes.jl:75
 [10] #s13#4(::Any, ::Any, ::Any, ::Any, ::Type, ::Type, ::Any) at /Users/nathan.daly/.julia/packages/UProbes/199dD/src/UProbes.jl:64
 [11] (::Core.GeneratedFunctionStub)(::Any, ::Vararg{Any,N} where N) at ./boot.jl:524
 [12] f at ./REPL[10]:2 [inlined]
 [13] top-level scope at ./REPL[11]:3

i'll putz with this now.

[NHDaly]

Based on this random comment from the internet, it looks like the ASM is different for x86 gnu linux and x86 macos :( https://github.com/TeamShadow/shadow/issues/10#issuecomment-62177139

[vchuravy]

Ah that makes sense and I will need to figure out what the correct asm is for Mac/how dtrace probes look on Mac.

[jpsamaroo]

Arguments are working now via bpftrace, at least on the LLVM 4 branch with Julia ~master.

[jpsamaroo]

(Specifically on Linux)