Conform `Array` to `Module` and `Layer`

[dan-zheng]

Add a conditional conformance: extension Array: Layer where Element: Layer.

An Array of Layers is a Layer that applies each element layer in sequence to an input to produce an output. The callAsFunction method calls differentiableReduce, with the input as the initial value.

I think this definition is correct, but I haven't tested it:

import TensorFlow

extension Array: Module where Element: Layer, Element.Input == Element.Output {
    public typealias Input = Element.Input
    public typealias Output = Element.Output

    @differentiable
    public func callAsFunction(_ input: Input) -> Output {
        differentiableReduce(input) { $1($0) }
    }
}

extension Array: Layer where Element: Layer, Element.Input == Element.Output {}

Please add this to the library and add tests! Motivating mailing list thread here.

If you'd like to work on this, please comment and I'll add you as the assignee.

[dan-zheng]

Comment: it's pretty slick that you can make Array a Layer in just 9 lines. The power of differentiable programming + protocol-oriented programming!

[SumanSudhir]

@dan-zheng can I take up this issue as I need a similar module for one of my models?

[RahulBhalley]

Oh s---! @SumanSudhir bro I wanted this! You been too quick to get assigned! 🤪

Just kidding. Looking towards your work. 😃

[SumanSudhir]

Sorry, my exam was going on, so I didn't spend time on it that day

[saeta]

Reassigning to @dabrahams who had some thoughts on design here. Thanks!

[dabrahams]

As noted in https://github.com/tensorflow/swift-apis/pull/673#discussion_r394652982 I don't think the specific feature requested in this issue is quite right and there are alternatives that would be more powerful/useful. Back to @dan-zheng to reply or repurpose this issue or close it and open a new one.

[austinjones]

I ran into this issue - would be great to add to the library. It took a bunch of Github sluething to find a way to run an input through many identical layers.

Or, if an array extension doesn't work, maybe a SequentialArray type could wrap this behavior, so it'd be discoverable in the API docs?

[RahulBhalley]

Any updates on this one?

[saeta]

Hi @rahulbhalley ! Thanks for checking in on this one. @dabrahams and @dan-zheng have updated https://github.com/tensorflow/swift-apis/pull/708 to reflect our current thinking. Feedback on that PR is welcome!

[RahulBhalley]

Somehow the mailing list discussion post link is now changed.

[dan-zheng]

I'm closing this issue, since directly conforming Array to Layer is no longer the direction we're pursuing.

A fresh issue https://github.com/tensorflow/swift-apis/issues/1021 tracks the new layer wrapper type approach and provides rationale for it.

---

To users: if you need to use an Array: Layer conformance in the short-term before #1021 is done, feel free to add it locally to your codebase. A sample conformance definition is available here.