proposal: spec: multidimensional slices

[dadkins]

As per Rob and Andrew's request:
https://groups.google.com/forum/#!topic/golang-nuts/Q7lwBDPmQh4

[kisielk]

Comment 1:

Some more discussion on gonum-dev:
https://groups.google.com/d/topic/gonum-dev/WnptzWjqhmk/discussion

[gopherbot]

Comment 2 by nagle@animats.com:

There's been some discussion about this.  Main points so far:
1. The general goal is to make it easy to do Matlab/Octave like work in Go.  Most
   engineering math today is prototyped in one of those languages, then translated
   to something faster for production use.   
2. It should be possible to easily convert standard libraries from "Numerical Methods" 
   (the book series, "Numerical Methods in Fortran, ... in C, in C++ etc.") to Go.
   That would get Go a needed set of standard numerical libraries.  And it should
   be reasonably easy to translate Matlab code to Go code. 
3. It's helpful to have only one standard way to represent matrices.  Otherwise,
   you get math libraries where a matrix coming out of one library won't go into
   another.  (This is one of Matlab's selling points.)
4. Performance is a major issue.  These constructs should go fast, approaching
   or exceeding C/FORTRAN performance.  
Language issues:
1. In general, multidimensional array/slice access syntax would look like "arr[i,j]".
2. Multidimensional slices would be supported, derived from multidimensional arrays.
3. While there is interest in generics and overloading, that's probably too radical.
   However, extending "+", "-", and "*" to vectors and matrices might be reasonable.
   Many machines have hardware that can speed up such operations (MMX, etc.) so
   that's something a compiler can do that a library alone cannot.
4. There's interest in "reshaping", where a subarray or sub-slice is derived from
   an array.  This can easily get overcomplicated (it did in FORTRAN 77) and needs
   to be carefully designed.  Further discussion and use cases are needed for this.
The machine learning community within Google might be consulted on what they want in
this area.  The machine learning theorists write in Matlab, and then someone has to
make that work in production code. 
It would be helpful to get agreement on needed functionality before putting too much
effort into syntax, so there are few specific syntax suggestions here yet.

[adg]

Comment 3:

Thanks for filing the issue.

Labels changed: added feature, languagechange, removed priority-triage.

Status changed to Thinking.

[dadkins]

Comment 5:

My desires are modest. I'd like to be able to write something like
// LUPDecompose performs an in-place LU factorization of a square
// matrix A. It returns a permutation vector P, such that PA = LU.
// On completion, A[i][j] = L[i][j] if i > j, U[i][j] if i <= j.
func LUPDecompose(A [][]float64) (P []float64, err error)
When writing this kind of code in C, I used to define a macro
#define A_(i,j) (A[(i)*rowsep + (j)])
That is precisely the kind of thing I wish the compiler would do for me.

[gopherbot]

Comment 6 by gexarcha1:

This document contains a collection of different proposals described in different levels
of detail:
https://docs.google.com/document/d/1gejfouITT25k29eHYTgdvi4cCLVt5dhYiJVvF46gp78/edit
It also contains links to any proposal or information relevant to the topic that we
found interesting. Feel free to edit the Document to clarify any questions relevant to
implementation.
I think most of the people in need of multi dimensional arrays in Go are not used to
writing proposals for programming language changes. If you have any input that would
make the process less awkward it would be highly appreciated.

[gopherbot]

Comment 7 by nagle@animats.com:

There's been considerable discussion of this subject on "comp.lang.go.general".
Areas of agreement:
   There seems to be a consensus that multidimensional arrays and slices in Go would be useful for numerical work.
   The array element access syntax generally proposed is 
       arr[n,m]     // 2D
       arr[n,m,o]   // 3D
etc.  Existing arrays of arrays would be retained in Go. Arrays of arrays can be
"ragged",
(rows of different length) but multidimensional arrays are always of uniform length in
each dimension, allowing FORTRAN-type optimizations. 
   Both fixed-size multidimensional array types defined at compile time and arrays sized
at run time have use cases.  The former are widely used for graphics, and the latter are
widely used for general numerical work.  It should be possible to pass both as parameters
to functions, although not necessarily the same functions.
Areas of disagreement:
   There's much disagreement over what facilities for reslicing and reshaping should
be provided. 
   The minimum capability discussed is the ability to pass an entire array or 
multidimensional slice object to a function.  Creating a slice which refers
to a portion of an array, though, is controversial. Syntactically, this refers
to expressions of the form
         arr[a0:a1, b0:b1]
The problem is that values of b0 and b1 which are smaller than the array bounds
result in the need to represent a kind of sparse array, where the memory
distance between adjacent elements is not the same as the size of an element.
This complicates the internal representation of slices; even the lowest dimension
must have a "stride" value.  There's a performance penalty for this feature when it
not being used.  But it is convenient to have.  
   The other big issue is the ability to "grow" multidimensional slices. 
Should "append" be allowed for multidimensional arrays, and if so, how
general should it be?  Is growth allowed in all dimensions?  It's a useful
capability but complicates the implementation considerably. 
   Those are the two biggest issues being discussed.
   Lesser issues include whether operators "+", "-", and "*" should be provided
as built-in operations for multidimensional arrays and slices.  Providing them
permits easy transliteration for math libraries in other languages (C++, Matlab
especially).  But they're not essential; NumPy uses "A.dot(B)" for matrix multiply.
There are a lot of cases to handle for "*" - vector*scalar, vector*matrix, 
matrix*matrix...  "+" and "-" are somewhat simpler.
   That's roughly where the discussions are.  Once the slice and growth issues are
decided, the rest should fall into place.

[adg]

Comment 8:

The discussion referred to by nagle@ is on golang-nuts ("comp.lang.go.general" is just a
name made up by gmane.org):
https://groups.google.com/forum/#!topic/golang-nuts/Q7lwBDPmQh4

[gopherbot]

Comment 9 by norman.yarvin:

My proposal, at
https://docs.google.com/document/d/1xHyOK3hSxwMtyMH-pbXSXxo7Td9A_Teoj6cbr9ECeLM/edit?pli=1#
has now reached a reasonably mature state.  I'd appreciate if people had a look at it. 
It's a rather long proposal, but that's because this is not a small addition to the
language and I've tried to be thorough.  The proposal is purely about the basics of
multidimensional slices; math operations on whole slices are not part of it.  (It does
not preclude later adding them, but does not set up a situation where they'd be
necessary, either.)
As regards comments that people have made so far on the document, please keep in mind
that most of them were made against earlier versions, in which some things were not
explained as well.  Thus if any appear to be foolish that should not be held against the
commenter, or at least not too much.  (Though some comments have been marked as
resolved, I haven't marked them that way just because I disagree and have improved my
explanation, as that seems too dictatorial; I figure people can retire their own
comments if their objections have truly been addressed.)
As a personal comment, it is rather rare to have an opportunity to add a major feature
to a language so cleanly.  Usually one either has to break compatibility or introduce
some serious ugliness or severe compromise.  But here it's almost like Go was designed
for this feature to just be plugged in and almost seamlessly fill a gap in its
capabilities.  Or so I believe; go ahead, hammer on the proposal and prove me wrong.

[gopherbot]

Comment 10 by A.Vansteenkiste:

In my opinion, instead of changing the language, it might be useful if the standard
library simply provided a standard matrix format. That should guide the creation of
idiomatic, compatible matrix libraries. Cfr. database/sql, which guided the creation of
excellent database drivers.
The image package already provides nice guidance on how to handle 2D matrices. It uses
contiguous underlying storage which is needed when passing to external libraries. I
modelled a matrix (storage) package after image, this is how it looks like:
http://godoc.org/github.com/barnex/mat. However, many people wrote such kind of packages
with slightly different, incompatible, conventions. Hence it might be nice to have on a
standard convention in the standard library, once and for all.

[rsc]

Comment 11:

Labels changed: added go1.3maybe.

[rsc]

Comment 12:

Labels changed: removed feature.

[rsc]

Comment 13:

Labels changed: added release-none, removed go1.3maybe.

[rsc]

Comment 14:

Labels changed: added repo-main.

[btracey]

Comment 15:

About a month ago, I put together a proposal for this. Putting this comment as a
reference.
Here is a reduced proposal for multi-dimensional arrays
https://docs.google.com/document/d/1eHm7KqfKP9_s4vR1zToxq-FBazdUQ9ZYi-YhcEtdfR0/edit
Golang-nuts discussion thread
https://groups.google.com/forum/#!searchin/golang-nuts/tables$20in$20go/golang-nuts/osTLUEmB5Gk/3-f9_UKfE9MJ
In the thread, Dan Kortschak proposed a nice syntax for range which could be an addition
to the proposal (https://groups.google.com/d/msg/golang-nuts/osTLUEmB5Gk)/-A15bJpuTzsJ

[MichaelTJones]

Comment 16:

Related discussion in my go-nuts post:
https://groups.google.com/forum/#!topic/golang-nuts/ScFRRxqHTkY

[btracey]

Comment 17:

My proposal has been updated and now includes a proposal for range syntax.

[adg]

@nigeltao's response to the proposal: https://groups.google.com/d/msg/golang-dev/T2oH4MK5kj8/Kwpe8NfD45IJ

We appreciate the amount of work and care that went into this proposal, but we're sorry that it's not likely to be adopted, at least for Go 1.

The problem isn't with the proposal itself. The reason is that the language is stable: the bar for new language features is high, and is only getting higher. The "Changes to the language" sections of http://golang.org/doc/go1.1, http://golang.org/doc/go1.2 and http://golang.org/doc/go1.3 are getting smaller, not bigger. The changes that did happen were also minor compared to introducing tables, which touch make, literals, indexing, slices, len/cap, copy, range, reflect and the type system in general.

We don't doubt that adding tables to the language has benefits. Any reasonable language feature has benefits, and the gonum-dev mailing list's existence clearly shows that it would make some people very happy. Yet even if tables were part of the language, it's hard to see any packages in the standard library, or even in the code.google.com/p/go.foo sub-repositories, that would use them. There is a chicken-and-egg factor here, but it's still not encouraging for tables.

The only candidate seems to be the image-related packages, but even then, we can't change, for instance, the image.RGBA type in the Go 1.x time frame, and even for a hypothetical Go 2.0, it's not clear that changing the design of the image package is a win. One of the motivations for the current design is that, when decoding from or encoding to formats like GIF or PNG, it's useful to linearize the rectangle of pixels as a []byte, as spoken by general-purpose compressors like LZW and ZLIB. Another deliberate design decision, based on Plan 9 GUI experience, was that the top-left of an image isn't necessarily at (0, 0).

In any case, debating the proposal's benefits is secondary. To repeat the main point, we value API and language stability very highly. Yes, the proposal is backwards-compatible, but it's a feature request, not a bug fix, and we err on the side of making no changes.

As an alternative, one could define a computational language a la halide-lang, and write a program that worked with "go generate". This program would parse the specialized code and generate Go 1.x code (which possibly uses package unsafe for pointer arithmetic), or generate C code, or generate 6a-compatible assembly code, or generate GPU-specific code. Of course, this still requires finding someone to do the work, but that person or group of people don't have to be familiar with the runtime and compilers, blocked on Go's release cycles, or bound by the Go 1 compatibility promise.

[btracey]

Design document uploaded https://go-review.googlesource.com/16801

[ianlancetaylor]

A somewhat different suggestion at #13253.

[rsc]

For formatting: https://golang.org/design/6282-table-data.

[btracey]

Thanks Russ

[btracey]

The proposal was modified to allow tables of any dimension. Everything extended naturally, except make now accepts [N]int for consistency with len and cap.

[sbinet]

perhaps it should also mention https://github.com/golang/proposal/blob/master/design/12416-cgo-pointers.md ? (especially for the part where we talk about compatibility/interoperability with C libraries)

[btracey]

Do you have something specific in mind? Part of the design of that proposal was to make cgo safer while expressly permitting our current use case. Assuming we can somehow deconstruct the table (which may need a stride built-in, I'm not sure), we should be able to call c just like now as far as I understand.

[sbinet]

Yes, of course. I was merely pointing out that it would perhaps be beneficial for this section (reflect)

as there exists a large body of C libraries written for numerical work, with lots of time
spent in getting the codes to run efficiently. Being able to call these codes directly is 
a huge benefit for doing numerical work in Go (for example, the gonum and biogo
matrix libraries have options for calling a third party BLAS libraries for tuned matrix
math implementations)

to mention the new cgo-rules, even just to clearly show this proposal is aware of them.

[btracey]

Could the title of this be changed to "multidimensional slices"? The language already has multidimensional arrays

[ianlancetaylor]

@btracey Done.

[adg]

@griesemer @rsc @ianlancetaylor and others: this proposal was prepared a long time ago, but it is yet to be seriously addressed (apart from @ianlancetaylor's counterproposal) .

We need to make an explicit decision as to whether this proposal will be seriously considered for Go 1.

[gopherbot]

CL https://golang.org/cl/24271 mentions this issue.

[griesemer]

I believe this is a pretty important topic and good support here might allow Go to break into new application domains. The proposal is well written and pretty well thought out. I'll write some more feedback tomorrow.

[griesemer]

An initial design doc has been committed, with some additional feedback in the respective code review: https://github.com/golang/proposal/blob/master/design/6282-table-data.md . Work in progress.

[kokes]

Excellent, this is very nice. I have just one question.

I see the number of dimensions is set in make(), but then can be changed using reshape. But the way dimensions are passed, in an array, precludes any runtime decisions about the number of dimensions, is that right?

Say I have a 16-element matrix in 4x4 and at runtime I'd like to reshape it into 28 or 22*4, depending on some other inputs. Since the reshaping dimension description is an array, not a slice, I cannot readily do that.

I presume this is due to the fact that the number of dimensions needs to be known at compile time to provide access index checks (just checking the number of dimensions would need to be shifted from compile time to runtime) and other checks and optimisations, I just wanted to know if I inferred it correctly.

Thank you!

[btracey]

I don't fully understand the question, but in this proposal every slice has a fixed dimension size. One cannot write a type of [,*](or something) that is an unknown dimensional slice.

You are correct that you cannot reshape a [4][4]T into a [,]T or [,,]T of arbitrary size. You can, however, reshape a [16]T into those sizes.

A [4][4]T could be reshaped just fine, but

var a [16][16]T
b := a[:4, :4]

could not be correctly reshaped. I don't see how to simply distinguish between those two cases.

[gopherbot]

CL https://golang.org/cl/25180 mentions this issue.

[griesemer]

@kokes You are correct. The number of dimensions is always a compile-time constant since it is given by the multi-dim. slice type, and that type is fixed (cannot change during runtime).

[griesemer]

Regarding the status of this proposal: I have just reviewed the latest version of the design doc and provided some more feedback. As far as I can tell, and ignoring details of the design doc wording and some syntax specifics (which just need to be decided), the proposal seems fairly clean, consistent, and implementable.

The proposal is also backward compatible with Go as it is now - that is, the generalization of slices into multi-dimensional slices could be done w/o affecting existing Go code.

However, this backward-compatibility also comes at a high price: It makes it impossible for an n-dimensional slice to be (sub-)sliced in arbitrary dimensions. Indexing must always be done from left-to-right (outer- to innermost dimension) so that a 1-dimensional slice gracefully degrades into an existing Go slice. The problem is that existing slices don't store an explicit 'stride' as it is always 1. For instance, given a 2-dim. matrix, I can "slice" a row vector, but I cannot "slice" a column vector (since the stride of that column would not be 1 in general).

Without paying a runtime cost, I don't see how backward-compatibility can be retained without the above-mentioned restriction.

I think this is the crucial question here: Is this restriction too restrictive?

[btracey]

I just pushed up a new version a new version of the proposal to the review. It contains a detailed description on why I think the restriction is not too restrictive. I encourage others to comment.

[attaboonie]

Is it possible to get the "best of both worlds" where the slice (in reflect etc) always theoretically has a stride, but for the standard 1 dimension case this extra word of memory footprint is eliminated via a compiler optimisation?

[attaboonie]

At risk of appearing even more foolish ... As length and capacity are currently both ints in the API, yet in practice can only be positive, there are in effect two spare bits available in the current slice header if these were needed to indicate multiple variants.

[griesemer]

@attaboonie Yes, something like this would be possible but it would require an additional runtime check for for regular slices, since those now might have a stride that is not 1 (I am hinting at that "runtime cost" in my comment above).

Maybe there's some clever ways to work around this to make it almost zero-cost in the common case (e.g. by storing a negative length and checking when the bounds-check triggers, or something along those lines), but it's going to be very tricky.

[dr2chase]

Supposing we start with the stride-1 restriction to see if it bites, and also do some experiments to see if it can be relaxed w/o harming performance? There are some SSA-ish optimizations that might help if we got around to implementing them (and if we have multidimensional strides, we'll be motivated to implement them for all the other dimensions that can have non-unit stride).

[yiyus]

First, I would like to thank Brendan Tracey and the gonum guys for their work in this proposal, and also the Go team for considering taking steps towards making Go more suitable for scientific computing. I know language changes at this stage are a very big deal.

However, I do not think this is the best approach. In my opinion, not allowing 1D strided slices from the beginning would be a huge mistake. It would be difficult to explain such a limitation to someone coming from Python, Fortran or Matlab, for example. I think the consistency you lose when you treat rows and columns differently is much bigger than the one you get from normal slices being a particular case of tables.

The proposal argues that having strided slices and normal slices would be harmful to the language, but I do not agree with this claim. It is not clear at all that it would lead to duplication of code for strided and non-strided slices, specially when other scientific languages only support strided slices without this being a problem. I think the choice for one or another would be obvious: a strided slice for anything that can be considered a vector (ie. for anything that could be eventually passed to a foreign Fortran function), and a non-strided slice for everything else (everywhere they are being used now). I imagine the conversion from a non-strided to a strided slice would be a very cheap operation, so you would only write numeric functions that take strided slices arguments (even non-strided slices could be accepted where a strided slice is required, eventually). Additionally, 1D strided slices would simplify the proposal, removing some special cases, in particular in range.

It also needs to be taken into account that not having a strided slice may lead to the abuse of [,]T types for vectors. This may be considered bad practice, but I can imagine myself doing it when porting something from other language where strided slices are supported. Then we would lose type safety, and we would still have all the problems attributed to strided slices (you will need to chose between using a []T, a [,]T with len[0] equal to 1, or both).

IIUC, @dr2chase suggests to start with the limitation and leave open the option to implement strides later, but I think that, then, a considerable amount of scientific code expecting a normal slice (or a [,]T "column table") would be written, and introducing a new type at a later stage would be problematic.

As an additional comment, I do not think it is necessary to modify make or implement literals, at least in a first step. I would be happy enough with reshape (maybe taking a variadic argument) and some help from gofmt.

[griesemer]

@yiyus Thanks for your comments. I have spoken with several people at Google about this proposal, some of which work on machine-learning, and they expressed similar concerns. Personally, I also think that the 1d stride-1 restriction is too limiting.

The example I've brought up repeatedly (in the design doc) is that it makes perfect sense to want to extract a row and column vector from a matrix, and that both these vectors should be 1d vectors so that they can be supplied to a dot-product function that takes two 1d vectors.

(At some point in the future one might want to supply more fancy "slicing" mechanisms, for instance one might want to select the diagonal of a matrix. If multi-dimensional slices/tables have strides in all directions, any such "linear" sub-section of a table can be represented w/o having to change the internal representation.)

I think the good news is that the proposal clarified these two implementations aspects (generalized slices that degrade into regular Go slices but have a restriction, and a more general multi-dimensional data structure "table" as this was the original name).

I'm going to review the latest draft update for the current version in the next few days and at that point it might be good to determine the next steps forward; e.g., should one explore an unrestricted proposal in detail.

[sbinet]

@griesemer: I suppose people having dealt with the sawzall -> lingo migration would also have some insights into data science use cases. (I suppose you may have facilities to tap into their brains :P)

before Go, I was into C++ and (C)python. There, numpy is king and builds on top of the "buffer protocol" to provide data interop' (between C-land and CPython, but also between various CPython extension modules, e.g. a SQL module and numpy, numpy and PIL/pillow (python image libraries)) and cheap slice'n'dice operations.

having a ndim-slice like the buffer protocol (so, with strides), without any allocation possible (so w/o cap) but with reshape would fit the bill in image processing, ML, high energy physics and, probably, all science-y data crunching applications.

having said that, between having this proposal implemented and the current status quo, I would go with this proposal (even if to somehow repeat the mistakes of the std::valarray<T> from C++).

[btracey]

Thanks for the comments @yiyus . It's a good conversation to have.

It would be difficult to explain such a limitation to someone coming from Python, Fortran or Matlab, for example.

In Matlab, such slicing creates a copy, so the behavior in Go is going to be surprising no matter what.

Could you give some examples in Fortran? I've only worked in old Fortran codebases. In BLAS and LAPACK, for instance, data arrays are contiguous, as in this proposal. Many of the operations do work on column vectors, of course, but that is implemented by explicitly passing the stride and doing the strided indexing manually. See the implementation of Dasum https://github.com/gonum/blas/blob/master/native/level1double.go#L91 or http://www.netlib.org/lapack/explore-html/de/d05/dasum_8f_source.html

Python does have the behavior you propose. Do you know how they deal with it? I suspect that they have to make a copy of the array before passing it to Lapack. I tried to read the source, but I when I got to the call for _makearray, I get lost in the c code for the array function and in wrap = getattr(a, "__array_prepare__", new.__array_wrap__)

I ask because the goals of Go are different than those of python/matlab/julia. Go will never have the syntactic flexibility of these languages. The benefit of Go is having a consistent language across users, and having a language that is possible to implement efficiently. In Matlab, the tradeoff is to allow really easy access to columns, even if that means a silent copy. In python, it's likely okay to sacrifice performance opportunities for syntactic ease -- python is not a fast language in general. Just because it's the right choice for those languages doesn't mean it's the right choice for Go -- the definition of "not a problem" is different.

---

I agree that we want to be able to perform operations on the data in columns, and I also agree that in some cases we want to do that copy-free. There is a problem with my proposal at the moment, one cannot extract a single column of a table. A few of us offline have been discussing this. The solution is to also add an unshape function that can take a table, and return the linear data.

func unshape([,*]T) ([]T, [N-1]int)

where [,*]T is shorthand for a table of dimension N, and the returned values are the underlying linear data of the table, and the strides in the dimension. With this function, one can extract a specific column of data, copy-free. Updating gonum/matrix, for instance, we would have

type Dense [,]float64

type Vector struct {
    data []float64
    stride int
}

func (d Dense) ColView(i int) Vector{
      data, stride := unshape(d)
      return Vector {
           data: data[i:],
           stride: stride,
     }  
}

The returned vector is a copy-free view on the column.

Personally, I think this meaningfully meets your desirata. You can use a Vector anywhere you "need something that is a vector", and you can use a []T anywhere else.

---

There are two costs to strided slices. The first is having two types that are basically the same thing. The second is the lost of optimization opportunities. It seems possible to me that strided slices are much harder to optimize than contiguous ones, especially in the area of SIMD. If SIMD optimizations are too difficult to implement, then Go is never going to be competitive with C and Fortran. It seems like there could still be this tension with where to use strided slices and contiguous slices.

I guess I'm playing the role of champion against strided slices. In that role, what I want to see is:

1) An argument for why the Vector type is insufficient, making strided slices necessary. Such an argument should keep in mind a) Assuming inlining, accessing through a Vector is just as expensive as accessing a strided slice b) As far as I understand, the storage costs for a strided slice are identical to the storage costs of a Vector

2) An explanation why a future Go compiler will still be able to implement SIMD operations, even understanding the goals of Go for fast compile times and small binaries

3) Rules of thumb on when to use a strided slice and when to use a contiguous one. Should the entirety of gonum change? Thus, anytime I would have had a []float64, I should instead have a [']float64 (or whatever the syntax is)? If not... clearly it should be type Vector [']float64. Should gonum/floats use [']float64 or []float64? Should there also be a duplicate "stridedfloats" package? What about the weights of a categorical random variable (https://github.com/gonum/stat/blob/master/distuv/categorical.go#L15)?

---

@griesemer Your post came in the middle of me writing mine. Should I update the proposal to add unshape operation? I think it's necessary for the proposal to be useful.

[sbinet]

@btracey no, numpy.array has all the informations to pass to lapack to hand it a pointer to C-array of floats with strides, row/col-major and rank informations. that's the buffer protocol I was talking about. (it also has knobs for r/w access and ref-counting but that's only relevant to the CPython implementation)

see: https://docs.python.org/3/c-api/buffer.html (the numpy.array was the champion for this protocol (during the python2 era) that it, of course, also implements in python3)

wrt SIMD, even with the current proposal we might have some issues with alignment (especially when you sub-slice an n-dim slice (strided or not)). SIMD has support for scatter/gather operations anyways. They are of course less efficient than in the case of completely adjacent data, but for the alignment issues evocated above, I'd say you would want to copy your data anyways to be sure to use the SIMD instructions. (in effect, that's what you would do to tap into GPUs and their remote device memory)

[yiyus]

@btracey Answering your question, since Fortran 90 it is possible to take rows from a matrix with a similar syntax as the one used by Python, as for example in: row = matrix(1,:). Fortran uses column major ordering, so col = matrix(:,1) would just be a sub-array of stride 1 (every array has a stride in Fortran).

Also, I do not think that supporting strided slices would suppose a problem for optimization, on the contrary. If they are not supported, some workaround will be needed to extract columns from a matrix, but this will disallow applying optimizations with SIMD instructions that support strided loads.

[btracey]

@sbinet it has the information to pass to Lapack, but I'm pretty sure it has to copy the data before passing to Lapack if the inner stride is not 1. From http://docs.scipy.org/doc/numpy/user/c-info.how-to-extend.html "Most third-party libraries expect contiguous arrays. But, often it is not difficult to support general-purpose striding. I encourage you to use the striding information in your own code whenever possible, and reserve single-segment requirements for wrapping third-party code. Using the striding information provided with the ndarray rather than requiring a contiguous striding reduces copying that otherwise must be made."

[yiyus]

To add to the discussion, I have cleaned up a bit my old proposal for "shaped slices": https://docs.google.com/document/d/1IvvkX60AMObA11CB6Gcc3xxG14VayGqBOjfpwg3qIRY/edit?usp=sharing

It is not really a counter proposal (it is far less mature than the table proposal, and the syntax chosen or other details may not be the best options, or even possible). I just want to show how I think that everything essential to implement multidimensional slices in what I consider a useful form is a new type, a reshape operation and multi-dimensional slicing/indexing.

Please, if you have any specific comment, add them directly in the document, to avoid hijacking this issue.

[btracey]

I updated my proposal with the unpack built-in described above, and added some code examples of operations that are very similar to strided slices. I reworded the criticism of strided-slice based proposals.

Current PR still at https://go-review.googlesource.com/#/c/25180/