search

FluxML / FluxBench.jl

Benchmarks for the FluxML ecosystem for deep learning, scientific machine learning, differentiable programming etc including AD and CUDA accelerated workloads
https://speed.fluxml.ai
14 stars 9 forks source link

Add Flux3D #9

Closed nirmal-suthar closed 3 years ago

nirmal-suthar commented 3 years ago

@DhairyaLGandhi How can I view buildkite logs? It is saying 'page not found'

DhairyaLGandhi commented 3 years ago

I think you need to add Flux3D to the project deps. Do you have a Buildkite login?

DhairyaLGandhi commented 3 years ago

I'm not sure why the ci pipeline isn't visible here when it works fine in Flux etc.

You can try running with FluxBench.submit()

nirmal-suthar commented 3 years ago

Does Fluxbench expect to have GPU enabled machine? Locally, Fluxbench.submit() is giving cuInit error!!

DhairyaLGandhi commented 3 years ago

This is what I see in the CI logs


(2/4) benchmarking "Flux3D"...
--
  | (1/2) benchmarking "Flux3D_TriMesh_Forward_Pass"...
  | ERROR: UndefVarError: fc not defined
  | Stacktrace:
  | [1] getproperty
  | @ ./Base.jl:26 [inlined]
  | [2] var"##sample#674"(__params::BenchmarkTools.Parameters)
  | @ FluxBench ~/.cache/julia-buildkite-plugin/depots/df646e51-ac52-453c-b19b-c355be13a749/packages/BenchmarkTools/eCEpo/src/execution.jl:383
  | [3] _run(b::BenchmarkTools.Benchmark{Symbol("##benchmark#672")}, p::BenchmarkTools.Parameters; verbose::Bool, pad::String, kwargs::Base.Iterators.Pairs{Symbol, Integer, NTuple{4, Symbol}, NamedTuple{(:samples, :evals, :gctrial, :gcsample), Tuple{Int64, Int64, Bool, Bool}}})
  | @ FluxBench ~/.cache/julia-buildkite-plugin/depots/df646e51-ac52-453c-b19b-c355be13a749/packages/BenchmarkTools/eCEpo/src/execution.jl:405
  | [4] #invokelatest#2
  | @ ./essentials.jl:710 [inlined]
  | [5] #run_result#37
  | @ ~/.cache/julia-buildkite-plugin/depots/df646e51-ac52-453c-b19b-c355be13a749/packages/BenchmarkTools/eCEpo/src/execution.jl:32 [inlined]
  | [6] run(b::BenchmarkTools.Benchmark{Symbol("##benchmark#672")}, p::BenchmarkTools.Parameters; progressid::Base.UUID, nleaves::Int64, ndone::Int64, kwargs::Base.Iterators.Pairs{Symbol, Any, NTuple{6, Symbol}, NamedTuple{(:verbose, :pad, :samples, :evals, :gctrial, :gcsample), Tuple{Bool, String, Int64, Int64, Bool, Bool}}})
  | @ BenchmarkTools ~/.cache/julia-buildkite-plugin/depots/df646e51-ac52-453c-b19b-c355be13a749/packages/BenchmarkTools/eCEpo/src/execution.jl:94
  | [7] macro expansion
  | @ ~/.cache/julia-buildkite-plugin/depots/df646e51-ac52-453c-b19b-c355be13a749/packages/BenchmarkTools/eCEpo/src/execution.jl:107 [inlined]
  | [8] macro expansion
  | @ ./timing.jl:287 [inlined]
  | [9] (::BenchmarkTools.var"#42#43"{Bool, String, Base.Iterators.Pairs{Symbol, Any, NTuple{7, Symbol}, NamedTuple{(:samples, :evals, :gctrial, :gcsample, :progressid, :nleaves, :ndone), Tuple{Int64, Int64, Bool, Bool, Base.UUID, Int64, Int64}}}, BenchmarkTools.BenchmarkGroup, Tuple{}})(progressid::Base.UUID, nleaves::Int64, ndone::Int64)
  | @ BenchmarkTools ~/.cache/julia-buildkite-plugin/depots/df646e51-ac52-453c-b19b-c355be13a749/packages/BenchmarkTools/eCEpo/src/execution.jl:106
  | [10] _withprogress(f::BenchmarkTools.var"#42#43"{Bool, String, Base.Iterators.Pairs{Symbol, Any, NTuple{7, Symbol}, NamedTuple{(:samples, :evals, :gctrial, :gcsample, :progressid, :nleaves, :ndone), Tuple{Int64, Int64, Bool, Bool, Base.UUID, Int64, Int64}}}, BenchmarkTools.BenchmarkGroup, Tuple{}}, name::String, group::BenchmarkTools.BenchmarkGroup; progressid::Base.UUID, nleaves::Int64, ndone::Int64, #unused#::Base.Iterators.Pairs{Symbol, Integer, NTuple{4, Symbol}, NamedTuple{(:samples, :evals, :gctrial, :gcsample), Tuple{Int64, Int64, Bool, Bool}}})
  | @ BenchmarkTools ~/.cache/julia-buildkite-plugin/depots/df646e51-ac52-453c-b19b-c355be13a749/packages/BenchmarkTools/eCEpo/src/execution.jl:71
  | [11] run(::BenchmarkTools.BenchmarkGroup; verbose::Bool, pad::String, kwargs::Base.Iterators.Pairs{Symbol, Any, NTuple{7, Symbol}, NamedTuple{(:samples, :evals, :gctrial, :gcsample, :progressid, :nleaves, :ndone), Tuple{Int64, Int64, Bool, Bool, Base.UUID, Int64, Int64}}})
  | @ BenchmarkTools ~/.cache/julia-buildkite-plugin/depots/df646e51-ac52-453c-b19b-c355be13a749/packages/BenchmarkTools/eCEpo/src/execution.jl:97
  | [12] macro expansion
  | @ ~/.cache/julia-buildkite-plugin/depots/df646e51-ac52-453c-b19b-c355be13a749/packages/BenchmarkTools/eCEpo/src/execution.jl:107 [inlined]
  | [13] macro expansion
  | @ ./timing.jl:287 [inlined]
  | [14] (::BenchmarkTools.var"#42#43"{Bool, String, Base.Iterators.Pairs{Symbol, Integer, NTuple{4, Symbol}, NamedTuple{(:samples, :evals, :gctrial, :gcsample), Tuple{Int64, Int64, Bool, Bool}}}, BenchmarkTools.BenchmarkGroup, Tuple{}})(progressid::Base.UUID, nleaves::Int64, ndone::Int64)
  | @ BenchmarkTools ~/.cache/julia-buildkite-plugin/depots/df646e51-ac52-453c-b19b-c355be13a749/packages/BenchmarkTools/eCEpo/src/execution.jl:106
  | [15] _withprogress(f::BenchmarkTools.var"#42#43"{Bool, String, Base.Iterators.Pairs{Symbol, Integer, NTuple{4, Symbol}, NamedTuple{(:samples, :evals, :gctrial, :gcsample), Tuple{Int64, Int64, Bool, Bool}}}, BenchmarkTools.BenchmarkGroup, Tuple{}}, name::String, group::BenchmarkTools.BenchmarkGroup; progressid::Nothing, nleaves::Float64, ndone::Float64, #unused#::Base.Iterators.Pairs{Symbol, Integer, NTuple{4, Symbol}, NamedTuple{(:samples, :evals, :gctrial, :gcsample), Tuple{Int64, Int64, Bool, Bool}}})
  | @ BenchmarkTools ~/.cache/julia-buildkite-plugin/depots/df646e51-ac52-453c-b19b-c355be13a749/packages/BenchmarkTools/eCEpo/src/execution.jl:77
  | [16] #run#41
  | @ ~/.cache/julia-buildkite-plugin/depots/df646e51-ac52-453c-b19b-c355be13a749/packages/BenchmarkTools/eCEpo/src/execution.jl:97 [inlined]
  | [17] #warmup#45
  | @ ~/.cache/julia-buildkite-plugin/depots/df646e51-ac52-453c-b19b-c355be13a749/packages/BenchmarkTools/eCEpo/src/execution.jl:141 [inlined]
  | [18] submit(submit::Bool, SUITE::BenchmarkTools.BenchmarkGroup)
  | @ FluxBench /var/lib/buildkite-agent/builds/rtx4000-gpuci4-julia-csail-mit-edu/julialang/fluxbench-dot-jl/src/FluxBench.jl:28
  | [19] submit(submit::Bool)
  | @ FluxBench /var/lib/buildkite-agent/builds/rtx4000-gpuci4-julia-csail-mit-edu/julialang/fluxbench-dot-jl/src/FluxBench.jl:27
  | [20] top-level scope
  | @ none:8
DhairyaLGandhi commented 3 years ago

@nirmal-suthar how do you feel about the PR? Could we get Flux3D set up with CUDA capability.

nirmal-suthar commented 3 years ago

We can merge this PR, once the build is passing.

Some work is needed on Flux3D side (GPU tests are failing with the newer CUDA version). So will open another PR once that is fixed.

DhairyaLGandhi commented 3 years ago

LGTM, one TODO is to add benchmarks for batched operations and see how those scale. In the meanwhile, let's get this in. Great work @nirmal-suthar !