JuliaReinforcementLearning / ReinforcementLearningTrajectories.jl

A generalized experience replay buffer for reinforcement learning
MIT License
8 stars 8 forks source link

push! performance tweaks #37

Closed jeremiahpslewis closed 1 year ago

jeremiahpslewis commented 1 year ago

This PR should speed up pushing to typical, multileveled Trace objects and reduces allocations, but comes at the cost of simple Trace object performance. @HenriDeh What do you think?

Running the ReinforcementLearning.jl/src/ReinforcementLearningCore/test/core/core.jl first example, 10% speed bump and 15% fewer allocations (by count)

After: 10.542 μs (133 allocations: 6.38 KiB)
Before: 11.666 μs (158 allocations: 6.95 KiB)

Simple Performance Test

using ReinforcementLearningTrajectories
using ReinforcementLearningTrajectories: AbstractTrace
using BenchmarkTools

t = Traces(;
a=[1, 2],
b=Bool[0, 1]
)

@btime push!(t, (; a=3, b=true))
# After: 283.039 ns (7 allocations: 224 bytes)
# Before: 184.911 ns (7 allocations: 192 bytes)

t = CircularArraySARTTraces(;
capacity=3,
state=Float32 => (2, 3),
action=Float32 => (2,),
reward=Float32 => (),
terminal=Bool => ()
)

@btime push!(t, ((state=ones(Float32, 2, 3), action=ones(Float32, 2))))
# After: 408.960 ns (10 allocations: 528 bytes)
# Before: 1.146 μs (22 allocations: 816 bytes)
codecov-commenter commented 1 year ago

Codecov Report

Merging #37 (e025c89) into main (1f2be9a) will decrease coverage by 0.09%. The diff coverage is 77.77%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@            Coverage Diff             @@
##             main      #37      +/-   ##
==========================================
- Coverage   69.34%   69.25%   -0.09%     
==========================================
  Files          13       13              
  Lines         561      566       +5     
==========================================
+ Hits          389      392       +3     
- Misses        172      174       +2     
Impacted Files Coverage Δ
src/traces.jl 83.52% <77.77%> (-0.72%) :arrow_down:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more