search

JaneliaSciComp / NIDAQ.jl

National Instruments Data Acquisition Interface
Other
47 stars 16 forks source link

getproperties() takes excessive amount of time #16

Closed alexporter8013 closed 5 years ago

alexporter8013 commented 5 years ago

When running the getproperties function. The call takes an excessive amount of time. This isn't a problem when just using the REPL and exploring, but any type of data acquisition script will likely suffer catastrophic timing results due to the use of getproperties in every read and write call.

Recommend either a struct for a task (allows people to break things by using native calls on a task instead of high level calls) or just multiple dispatch to have a more focused ccall specifically for reads and writes.

julia> @time getproperties()
  2.729390 seconds (859.03 k allocations: 53.213 MiB, 0.43% gc time)
Dict{String,Tuple{Any,Bool}} with 7 entries:
  "Tasks"              => (SubString{String}["CS", "MOSI", "MyAccelerationTask", "MyVoltageTask", "SCK", "SPI", "testVoltageTask"], false)
  "NIDAQUpdateVersion" => (0x00000000, false)
  "NIDAQMinorVersion"  => (0x00000006, false)
  "Scales"             => (SubString{String}[""], false)
  "DevNames"           => (SubString{String}["Dev1"], false)
  "GlobalChans"        => (SubString{String}[""], false)
  "NIDAQMajorVersion"  => (0x00000012, false)

julia> @time t = analog_input("Dev1/ai0")
  0.378259 seconds (509.50 k allocations: 25.669 MiB, 1.47% gc time)
NIDAQ.AITask(Ptr{Nothing} @0x0000000030c0f900)

julia> @time start(t)
  0.007996 seconds (2.82 k allocations: 158.683 KiB)

julia> @time read(t, Float64, 1)
  2.931029 seconds (1.33 M allocations: 76.679 MiB, 0.49% gc time)
1-element Array{Float64,1}:
 -0.0006444593315980392

julia> using Profile

julia> @profile read(t, Float64, 1)
1-element Array{Float64,1}:
 1.5828057680061097e-8

julia> Profile.print()
1    .\client.jl:436; _start()
 1 .\client.jl:284; exec_options(::Base.JLOptions)
  1 .\client.jl:346; run_main_repl(::Bool, ::Bool, ::Bool, ::Bool, ::Bool)
   1 .\essentials.jl:741; invokelatest
    1 .\essentials.jl:742; #invokelatest#1
     1 .\client.jl:362; (::getfield(Base, Symbol("##734#736")){Bool,Bool,Bool,Bool})(::Module)
      1 ...bot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\REPL\src\REPL.jl:192; run_repl(::REPL.AbstractREPL, ::Any)
       1 ...bot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\REPL\src\REPL.jl:1035; run_frontend(::REPL.LineEditREPL, ::REPL.REPLBackendRef)
        1 ...orker\package_win64\build\usr\share\julia\stdlib\v1.1\REPL\src\LineEdit.jl:2273; run_interface(::REPL.Terminals.TextTerminal, ::REPL.LineEdit.ModalInterface, ::REPL.LineEdit.MIState)
         1 .\essentials.jl:741; invokelatest
          1 .\essentials.jl:742; #invokelatest#1
           1 ...t\worker\package_win64\build\usr\share\julia\stdlib\v1.1\REPL\src\REPL.jl:714; (::getfield(REPL, Symbol("#do_respond#38")){Bool,getfield(REPL, Symbol("##48#57")){REPL.LineEditREPL,REPL.REPLHi...
            1 ...t\worker\package_win64\build\usr\share\julia\stdlib\v1.1\REPL\src\REPL.jl:140; print_response(::REPL.AbstractREPL, ::Any, ::Any, ::Bool, ::Bool)
             1 ...t\worker\package_win64\build\usr\share\julia\stdlib\v1.1\REPL\src\REPL.jl:155; print_response(::IO, ::Any, ::Any, ::Bool, ::Bool, ::Any)
              1 .\essentials.jl:741; invokelatest
               1 .\essentials.jl:742; #invokelatest#1
                1 .\multimedia.jl:287; display(::Any)
                 1 ...worker\package_win64\build\usr\share\julia\stdlib\v1.1\REPL\src\REPL.jl:135; display(::REPL.REPLDisplay, ::Any)
                  1 ...worker\package_win64\build\usr\share\julia\stdlib\v1.1\REPL\src\REPL.jl:131; display(::REPL.REPLDisplay, ::MIME{Symbol("text/plain")}, ::Any)
                   1 .\arrayshow.jl:316; show(::IOContext{REPL.Terminals.TTYTerminal}, ::MIME{Symbol("text/plain")}, ::Array{Float64,1})
                    1 .\show.jl:1837; summary
                     1 .\show.jl:1840; summary(::IOContext{REPL.Terminals.TTYTerminal}, ::Array{Float64,1}, ::Tuple{Base.OneTo{Int64}})
                      1 .\show.jl:1893; showarg
                       1 .\strings\io.jl:31; print(::IOContext{REPL.Terminals.TTYTerminal}, ::Type)
                        1 .\show.jl:436; show(::IOContext{REPL.Terminals.TTYTerminal}, ::DataType)
                         1 .\show.jl:526; show_datatype(::IOContext{REPL.Terminals.TTYTerminal}, ::DataType)
                          1 .\show.jl:436; show(::IOContext{REPL.Terminals.TTYTerminal}, ::DataType)
                           1 .\show.jl:532; show_datatype(::IOContext{REPL.Terminals.TTYTerminal}, ::DataType)
                            1 .\show.jl:458; show_type_name(::IOContext{REPL.Terminals.TTYTerminal}, ::Core.TypeName)
                             1 .\strings\io.jl:156; string
                              1 .\strings\io.jl:121; print_to_string(::Symbol)
                               1 .\none:0; Type
                                1 .\iobuffer.jl:114; #IOBuffer#316(::Bool, ::Bool, ::Nothing, ::Bool, ::Int64, ::Int64, ::Type)
                                 1 .\strings\string.jl:71; StringVector
1421 .\task.jl:259; (::getfield(REPL, Symbol("##26#27")){REPL.REPLBackend})()
 1421 ...dbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\REPL\src\REPL.jl:117; macro expansion
  1421 ...dbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\REPL\src\REPL.jl:85; eval_user_input(::Any, ::REPL.REPLBackend)
   1421 .\boot.jl:328; eval(::Module, ::Any)
    1420 C:\Users\aporter\.julia\packages\NIDAQ\BztQ2\src\analog.jl:71; read(::NIDAQ.AITask, ::DataType, ::Int64)
     1420 C:\Users\aporter\.julia\packages\NIDAQ\BztQ2\src\properties.jl:185; getproperties
      1420 C:\Users\aporter\.julia\packages\NIDAQ\BztQ2\src\properties.jl:185; #getproperties#35
       3    C:\Users\aporter\.julia\packages\NIDAQ\BztQ2\src\properties.jl:101; _getproperties(::Tuple{Ptr{Nothing}}, ::String, ::Bool)
        3 .\none:0; #names
         3 .\reflection.jl:98; #names#7
          3 .\sort.jl:682; sort!
           3 .\sort.jl:694; #sort!#7
            3 .\sort.jl:634; sort!
             3 .\sort.jl:539; sort!
              2 .\sort.jl:545; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Array{Sym...
               1 .\sort.jl:545; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Array{Sy...
                1 .\sort.jl:545; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Array{Sy...
                 1 .\sort.jl:546; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Array{S...
                  1 .\sort.jl:546; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Array{S...
                   1 .\sort.jl:546; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Array{...
                    1 .\sort.jl:545; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Array...
                     1 .\sort.jl:545; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Array...
                      1 .\sort.jl:557; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Arra...
                       1 .\ordering.jl:49; lt
                        1 .\strings\basic.jl:319; isless
                         1 .\strings\basic.jl:317; cmp
                          1 .\c.jl:234; unsafe_convert
                           1 .\pointer.jl:58; unsafe_convert
               1 .\sort.jl:546; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Array{Sy...
                1 .\sort.jl:546; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Array{Sy...
                 1 .\sort.jl:546; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Array{S...
                  1 .\sort.jl:545; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Array{S...
                   1 .\sort.jl:546; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Array{...
                    1 .\sort.jl:545; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Array...
                     1 .\sort.jl:545; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Array...
                      1 .\sort.jl:557; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Arra...
                       1 .\ordering.jl:49; lt
                        1 .\strings\basic.jl:319; isless
                         1 .\strings\basic.jl:317; cmp
              1 .\sort.jl:546; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Array{Sym...
               1 .\sort.jl:546; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Array{Sy...
                1 .\sort.jl:545; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Array{Sy...
                 1 .\sort.jl:546; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Array{S...
                  1 .\sort.jl:545; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Array{S...
                   1 .\sort.jl:545; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Array{...
                    1 .\sort.jl:546; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Array...
                     1 .\sort.jl:545; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Array...
                      1 .\sort.jl:546; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Arra...
                       1 .\sort.jl:546; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Arra...
                        1 .\sort.jl:540; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.MergeSortAlg, ::Base.Order.ForwardOrdering, ::Arr...
                         1 .\sort.jl:455; sort!(::Array{Symbol,1}, ::Int64, ::Int64, ::Base.Sort.InsertionSortAlg, ::Base.Order.ForwardOrdering)
                          1 .\ordering.jl:49; lt
                           1 .\strings\basic.jl:319; isless
                            1 .\strings\basic.jl:317; cmp
       1409 C:\Users\aporter\.julia\packages\NIDAQ\BztQ2\src\properties.jl:102; _getproperties(::Tuple{Ptr{Nothing}}, ::String, ::Bool)
        1403 C:\Users\aporter\.julia\packages\NIDAQ\BztQ2\src\NIDAQ.jl:21; eval
         1403 .\boot.jl:328; eval
       5    C:\Users\aporter\.julia\packages\NIDAQ\BztQ2\src\properties.jl:103; _getproperties(::Tuple{Ptr{Nothing}}, ::String, ::Bool)
        2 .\strings\basic.jl:229; *
         1 .\strings\substring.jl:180; string(::String, ::String)
          1 .\strings\string.jl:60; _string_n
           1 .\essentials.jl:355; cconvert
            1 .\number.jl:7; convert
             1 .\boot.jl:738; Type
              1 .\boot.jl:708; toUInt64
               1 .\boot.jl:597; check_top_bit
         1 C:\Users\aporter\AppData\Local\Julia-1.1.0\lib\julia\sys.dll:?; #s5#3(::Any, ::Any, ::Any, ::Any)
        1 .\strings\io.jl:156; string
         1 .\strings\io.jl:123; print_to_string(::Symbol)
          1 .\show.jl:158; print
           1 .\io.jl:509; write
            1 .\iobuffer.jl:414; unsafe_write(::Base.GenericIOBuffer{Array{UInt8,1}}, ::Ptr{UInt8}, ::UInt64)
        1 .\strings\string.jl:247; getindex
       1    C:\Users\aporter\.julia\packages\NIDAQ\BztQ2\src\properties.jl:109; _getproperties(::Tuple{Ptr{Nothing}}, ::String, ::Bool)
        1 .\refpointer.jl:84; Ref{Bool32}(::Int64)
         1 .\refvalue.jl:8; Base.RefValue{Bool32}(::Int64)
          1 .\number.jl:7; convert(::Type{Bool32}, ::Int64)
       2    C:\Users\aporter\.julia\packages\NIDAQ\BztQ2\src\properties.jl:155; _getproperties(::Tuple{Ptr{Nothing}}, ::String, ::Bool)
        2 .\array.jl:705; iterate
         1 .\array.jl:729; getindex
         1 .\array.jl:199; length
alexporter8013 commented 5 years ago

I updated my fork with these lines to replace the current way of getting number of channels:

outdata_ref = Ref{Cuint}()
DAQmxGetTaskNumChans(t.th, outdata_ref)
num_channels = outdata_ref.x

It seems to work. Will commence a pull request once I'm finished breaking then fixing the package.

bjarthur commented 5 years ago

closed by https://github.com/JaneliaSciComp/NIDAQ.jl/commit/190fb1d3b692e2663add93f57eae12431eae3d74