Closed johndpope closed 5 years ago
found this https://github.com/tensorflow/swift-tutorials/blob/master/iris/swift_tensorflow_tutorial.ipynb
// ==== Take the gradient of the loss function === // Use Swift's Automatic Differentiation to calculate the // gradients used to optimize our model.
for epoch in 0..<numEpochs {
var epochLoss: Float = 0
var epochAccuracy: Float = 0
var batchCount: Int = 0
for (x, y) in trainDataset() {
let (loss, grad) = lossAndGradient(for: x, using: model, labels: y)
model.update(withGradients: grad) { $0 -= learningRate * $1 }
let logits = predictions(for: x, using: model)
epochAccuracy += accuracy(predictions: logits.argmax(squeezingAxis: 1), truths: y)
epochLoss += loss
batchCount += 1
}
epochAccuracy /= Float(batchCount)
epochLoss /= Float(batchCount)
trainAccuracyResults.append(epochAccuracy)
trainLossResults.append(epochLoss)
if epoch % 50 == 0 {
print("Epoch \(epoch): Loss: \(epochLoss), Accuracy: \(epochAccuracy)")
}
}
func lossAndGradient(for input: Tensor<Float>,
using model: IrisParameters,
labels: Tensor<Int32>) -> (Float, IrisParameters) {
let (loss, (_, modelGrad)) =
#valueAndGradient(loss(for:using:labels:), wrt: .0, .1)(input, model, labels)
return (loss, modelGrad)
}
Try using #adjoint(*)
directly.
afraid not.
Updated project.
I have this working on the specified toolchain - and so all is well. Unfortunately - the code for adjoint has changed in latest toolchain released yesterday. fyi - https://github.com/apple/swift/pull/19216
Suggestion by rxwei - This regression is introduced in https://github.com/apple/swift/pull/19201 and is known to break things. You can try to use #gradient instead of manually chaining adjoints.
(it's only the two adjoints that are defining Tensor. that break.. )(
let (_, d3) = #adjoint(Tensor.*)(
let (_, d5) = #adjoint(Tensor
as this code is so bleeding edge - there's not so much documentation / reference code to turn to. If you find time, perhaps you can look into this? I'll keep digging in either case.