dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.47k stars 4.77k forks source link

CodeGen (JIT) issue classification #858

Open BruceForstall opened 4 years ago

BruceForstall commented 4 years ago

The CodeGen team attempts to classify GitHub issues with label area-CodeGen-coreclr with additional classifications beyond the GitHub labels. Each issue is given a "category", "theme", "skill level", "cost", and optionally "impact". These are added as a block of text at the bottom of the main issue description, e.g.:

category:implementation
theme:jit-coding-style
skill-level:beginner
cost:small
impact:small

We have a tool that allows us to extract these to Excel, modify and analyze them there, and update the GitHub issues that have been modified. We update periodically, but not necessarily regularly, to classify new issues.

The goal is to be able to get a better idea of what kinds of issues we have outstanding, and think about ways to address them.

We chose to use this mechanism instead of creating new GitHub labels, as we didn't want to bloat the GitHub label set in a way that would overwhelm and negatively affect other teams in the repo. You can still query for issues using specific classification tags.

As of now, the following are the used classifications:

Categories
===========
correctness
cq
design
documentation
eng-sys
implementation
performance
planning
proposal
question
reach
security
testing
throughput

Themes
===========
addressing-modes
alignment
altjit
asmdiffs
assertion-prop
barriers
basic-cq
benchmarks
big-bets
block-layout
block-opts
bounds-checks
boxing
build
calling-convention
codegen
constant-pool
copy-prop
cse
debug-dumps
debug-info
delegates
devirtualization
div-mod-rem
eh
emitter
expression-opts
floating-point
flowgraph
gc-info
gc-stress
generics
helpers
importer
inlining
interpreter
intrinsics
invalid-il
ir
jit-block-layout
jit-coding-style
jit-ee-interface
large-methods
liveness
loop-opt
lower
managed-c++
md-arrays
memory-usage
meta
minopts
morph
msil
needs-triage
ngen
null-checks
optimization
osr
pinning
pinvoke
planning
profile-feedback
prolog-epilog
range-check
ready-to-run
ref-counts
register-allocator
runtime
stack-allocation
structs
super-pmi
tail-call
testing
throughput
tiering
type-intrinsics
value-numbering
varargs
vector-codegen
verification
volatile
zero-init

Skill Level
===========
beginner
expert
intermediate

Cost
===========
extra-large
large
medium
small

Impact
===========
large
medium
small

Details of the breakdown of these classifications follows in subsequent comments to this issue.

@dotnet/jit-contrib

BruceForstall commented 4 years ago

Including the issues in https://github.com/dotnet/coreclr, this is the current state of CodeGen issues:

Row Labels Count of Category
cq 462
correctness 109
implementation 104
testing 35
throughput 35
design 17
proposal 16
documentation 8
eng-sys 8
reach 5
security 1
Grand Total 800
Row Labels Count of Theme
basic-cq 96
optimization 66
vector-codegen 48
register-allocator 48
intrinsics 45
jit-coding-style 40
inlining 39
ir 38
testing 34
structs 28
tail-call 26
loop-opt 23
benchmarks 21
emitter 15
bounds-checks 13
gc-stress 10
helpers 9
runtime 9
calling-convention 9
importer 9
assertion-prop 9
debug-dumps 9
super-pmi 8
value-numbering 8
throughput 8
barriers 8
gc-info 8
range-check 8
floating-point 7
cse 6
flowgraph 6
block-opts 6
msil 5
devirtualization 5
alignment 4
big-bets 4
pinning 4
ref-counts 4
stack-allocation 3
minopts 3
copy-prop 3
profile-feedback 3
prolog-epilog 3
large-methods 3
lower 3
md-arrays 2
morph 2
volatile 2
eh 2
type-intrinsics 2
liveness 2
interpreter 2
div-mod-rem 2
zero-init 2
debug-info 2
pinvoke 2
invalid-il 2
varargs 2
delegates 1
memory-usage 1
jit-ee-interface 1
block-layout 1
addressing-modes 1
elt-hooks 1
ready-to-run 1
generics 1
tiering 1
ngen 1
Grand Total 800
Row Labels Count of SkillLevel
expert 368
intermediate 340
beginner 92
Grand Total 800
Row Labels Count of Cost
medium 415
small 230
large 112
extra-large 43
Grand Total 800

There is certainly room for improvement in classification, e.g.,

CarolEidt commented 4 years ago

Thanks @BruceForstall ! It's a bit daunting, but it's great to have these tools.

BruceForstall commented 4 years ago

Here is the current state of the open area-CodeGen JIT issues:

Row Labels Count of Category
cq 479
correctness 116
implementation 104
throughput 35
testing 35
design 20
proposal 16
eng-sys 11
documentation 9
reach 4
security 1
Grand Total 830
Row Labels Count of Theme
basic-cq 94
optimization 75
vector-codegen 52
register-allocator 47
intrinsics 44
jit-coding-style 42
testing 41
ir 41
inlining 37
structs 29
tail-call 25
loop-opt 24
benchmarks 19
bounds-checks 17
emitter 15
gc-stress 11
super-pmi 10
helpers 9
calling-convention 9
debug-dumps 9
assertion-prop 9
barriers 8
throughput 8
runtime 8
value-numbering 8
range-check 8
importer 8
gc-info 7
floating-point 7
block-opts 6
cse 6
devirtualization 6
flowgraph 6
prolog-epilog 5
msil 5
big-bets 4
ref-counts 4
pinning 4
alignment 4
debug-info 3
profile-feedback 3
varargs 3
copy-prop 3
morph 3
minopts 3
stack-allocation 3
large-methods 3
lower 3
pinvoke 2
invalid-il 2
div-mod-rem 2
eh 2
interpreter 2
type-intrinsics 2
md-arrays 2
liveness 2
volatile 2
zero-init 2
block-layout 1
boxing 1
jit-ee-interface 1
delegates 1
addressing-modes 1
elt-hooks 1
ready-to-run 1
generics 1
null-checks 1
tiering 1
ngen 1
memory-usage 1
Grand Total 830
Row Labels Count of SkillLevel
expert 384
intermediate 350
beginner 96
Grand Total 830
Row Labels Count of Cost
medium 445
small 228
large 114
extra-large 43
Grand Total 830
BruceForstall commented 4 years ago

Here is the current state of the open area-CodeGen-coreclr JIT issues:

Row Labels Count of Category
cq 492
correctness 119
implementation 105
testing 37
throughput 35
design 20
proposal 16
documentation 9
eng-sys 9
reach 4
security 1
Grand Total 847
Row Labels Count of Theme
basic-cq 91
optimization 78
vector-codegen 52
intrinsics 50
register-allocator 47
testing 40
ir 40
jit-coding-style 40
inlining 37
structs 30
tail-call 26
loop-opt 24
emitter 17
bounds-checks 17
benchmarks 15
gc-stress 12
super-pmi 11
assertion-prop 9
throughput 9
importer 9
helpers 9
value-numbering 8
range-check 8
barriers 8
debug-dumps 8
runtime 8
calling-convention 8
floating-point 7
gc-info 7
prolog-epilog 6
devirtualization 6
block-opts 6
alignment 6
flowgraph 6
cse 6
msil 5
stack-allocation 4
debug-info 4
big-bets 4
pinning 4
ref-counts 4
profile-feedback 3
varargs 3
copy-prop 3
lower 3
minopts 3
large-methods 3
morph 3
interpreter 2
div-mod-rem 2
addressing-modes 2
ngen 2
zero-init 2
eh 2
invalid-il 2
type-intrinsics 2
md-arrays 2
liveness 2
codegen 2
volatile 2
pinvoke 2
null-checks 1
osr 1
question 1
tiering 1
verification 1
memory-usage 1
boxing 1
generics 1
elt-hooks 1
delegates 1
build 1
ready-to-run 1
block-layout 1
jit-ee-interface 1
Grand Total 847
Row Labels Count of SkillLevel
expert 388
intermediate 360
beginner 99
Grand Total 847
Row Labels Count of Cost
medium 456
small 231
large 116
extra-large 44
Grand Total 847
BruceForstall commented 4 years ago

Here is the current state of the open area-CodeGen-coreclr JIT issues:

Row Labels Count of Category
cq 557
implementation 111
correctness 98
throughput 35
testing 27
design 20
proposal 18
eng-sys 11
documentation 10
reach 4
planning 4
question 2
performance 1
security 1
(blank)  
Grand Total 899
Row Labels Count of Theme
basic-cq 101
optimization 97
register-allocator 56
vector-codegen 50
intrinsics 44
ir 40
inlining 39
jit-coding-style 38
testing 32
structs 31
loop-opt 28
bounds-checks 20
emitter 16
super-pmi 13
benchmarks 13
tail-call 12
debug-dumps 12
value-numbering 11
importer 11
throughput 11
devirtualization 11
cse 11
helpers 10
assertion-prop 10
range-check 9
calling-convention 8
floating-point 8
needs-triage 8
runtime 8
gc-info 7
barriers 7
prolog-epilog 6
flowgraph 6
pinning 6
altjit 5
block-opts 5
msil 5
codegen 4
gc-stress 4
big-bets 4
debug-info 4
stack-allocation 4
liveness 3
md-arrays 3
ref-counts 3
minopts 3
large-methods 3
alignment 3
profile-feedback 3
planning 3
osr 3
copy-prop 3
lower 3
invalid-il 3
morph 2
varargs 2
null-checks 2
addressing-modes 2
volatile 2
type-intrinsics 2
interpreter 2
div-mod-rem 2
eh 2
zero-init 2
asmdiffs 1
meta 1
jit-ee-interface 1
ready-to-run 1
memory-usage 1
block-layout 1
expression-opts 1
generics 1
jit-block-layout 1
boxing 1
constant-pool 1
delegates 1
managed-c++ 1
pinvoke 1
ngen 1
build 1
verification 1
tiering 1
(blank)  
Grand Total 899
Row Labels Count of SkillLevel
expert 414
intermediate 370
beginner 115
(blank)  
Grand Total 899
Row Labels Count of Cost
medium 462
small 254
large 140
extra-large 42
intermediate 1
(blank)  
Grand Total 899
Row Labels Count of Impact
large 1
medium 2
(blank)  
Grand Total 3
Row Labels Count of Source Count of Source2
JIT 445 49.44%
external 279 31.00%
.NET 176 19.56%
Grand Total 900 100.00%
JulieLeeMSFT commented 1 year ago

Here is the current state of the open area-CodeGen-coreclr JIT issues:

Category Count
correctness 132
cq 630
design 29
documentation 11
eng-sys 28
implementation 123
performance 37
planning 7
proposal 17
question 5
reach 3
security 1
testing 31
throughput 32
Others 4  
Total 1090
Theme Count
addressing-modes 6
alignment 5
altjit 1
asmdiffs 3
assertion-prop 13
barriers 6
basic-cq 93
benchmarks 41
big-bets 3
bitset 1
block-layout 4
block-opts 8
bounds-checks 27
boxing 2
build 9
calling-convention 10
codegen 28
constant-pool 0
copy-prop 3
cse 10
debug-dumps 10
debug-info 4
delegates 4
devirtualization 12
div-mod-rem 4
eh 6
emitter 14
expression-opts 7
floating-point 7
flowgraph 7
gc-info 9
gc-stress 3
generics 4
hardware-intrinsics 31
helpers 11
importer 16
inlining 62
interpreter 2
intrinsics 10
invalid-il 2
ir 38
jit-block-layout 1
jit-coding-style 26
jit-ee-interface 7
large-methods 2
liveness 4
loop-opt 39
lower 7
managed-c++ 1
md-arrays 4
memory-usage 6
meta 1
minopts 2
msil 5
needs-triage 3
ngen 2
null-checks 5
optimization 50
osr 3
pinning 6
pinvoke 2
planning 0
profile-feedback 25
prolog-epilog 6
range-check 6
ready-to-run 11
ref-counts 1
register-allocator 72
runtime 8
stack-allocation 14
structs 36
super-pmi 26
tail-call 9
testing 32
throughput 12
tiering 3
type-intrinsics 3
value-numbering 19
varargs 1
vector-codegen 50
verification 2
volatile 5
zero-init 4
Others  18
Total 1090
SkillLevel Count
expert 398
intermediate 358
beginner 133
(blank)  201
Total 1090

CC @dotnet/jit-contrib.