Open dan-zheng opened 4 years ago
@dan-zheng Math.swift
can definitely be broken down, but if that is the long term solution, isn't there a chance it might be lead to over-granular files?
@dan-zheng
Math.swift
can definitely be broken down, but if that is the long term solution, isn't there a chance it might be lead to over-granular files?
I think we can find a reasonable granularity, based on common sense. One file per function is obviously too fine-grained. One file per category of functions (e.g. ReductionOperations.swift
) seems reasonable. If the goal is to improve compilation time, changes should be backed by benchmark results.
If anyone pursues splitting large files into smaller ones to improve compilation time, please include some benchmark results, like in the PR description.
Compilation time (
swift build
) is very slow:I'm not sure when exactly it got so bad. Let's try to improve this!
Action items
pprof
or Xcode Instruments.This document describes Swift compiler performance tips.
Identifying hot spots using profiling tools like
pprof
or Xcode Instruments seems like a great first step. @marcrasi previously usedpprof
to generateTensorFlow
module compilation flamegraphs: perhaps that work can be polished and open-sourced in this repository.Type-checking is one big source of source of slowdown. Here are some sorted results from
swift build -Xswiftc -Xfrontend -Xswiftc -debug-time-function-bodies
(from Gist):Idea from @allevato: provide a contextual type for literal expressions to help the type checker.
It works:
I believe this should not be necessary, and may be a deficiency in the Swift type checker, specifically bidirectional type-checking and constraint solving. Whenever a contextual type exists, constraints should propagate from out to in, so
Scalar
is the only possible type for the literals2
and-2
inlogCoshLoss
. If we start constraint solving from the type variables for2
and-2
(which have many possible types), a huge disjunction constraint may be generated, leading to big slowdown.It would be nice to write a Swift forums question with a minimal reproducer of similar bad type-checking performance for literals with contextual type.
Suggestion from @rxwei: splitting larger files into more smaller files can help multithreaded compilation (
swift build
), since one thread can be spawned per file. Some files likeSources/TensorFlow/Operators/Math.swift
(currently 2835 lines) are huge and can be split.Note that we have one huge generated Swift file for TensorFlow bindings:
Sources/TensorFlow/Bindings/RawOpsGenerated.swift
(currently 36743 lines). That probably takes a while to compile.