data type interoperability

[vitiral]

I just had an excellent conversation with the developer of rulinalg and he mentioned the following:

Both ndarray and rulinalg already store the elements of their arrays/matrices in a single long contiguous vector. This means that you can already convert an ndarray Array2 into a rulinalg matrix essentially for free**

This is pretty essential information that I think should be included somewhere and your site seems like the best place.

Maybe there needs to be a library that specifically supports data type interop between libraries or something -- but it should definitely be documented.

[anowell]

Capturing interop between different libraries definitely fits into the scope of this site - especially as it starts to show some coordination between efforts.

Currently ndarray lives in data_structures and rulinalg in scientific. So perhaps the easiest thing is add (or otherwise cleanup and include) a note in the summary of one or both of those pages. I also wouldn't mind some sort of per-crate notes to better explain how they fit into the rest of the ML ecosystem.

I'd certainly accept PRs to either end if you're up for it. Otherwise, I'll try and find some time this weekend to capture that (and perhaps more generally update the summaries on each page).

[vitiral]

I think I want to try and have a conversation with the relevant crate authors/maintainers and see if we can get an agreed upon basic data structure and API (Trait) for converting to/from the different types with zero* cost. This issue seems like a good enough forum as any -- would it be fine if we converted it to that purpose?

Once we get such a crate, I think that would be the way to document it on this site. I'm hoping such an effort will be easy. I think telling the story of why there are different crates is really important because a lot of people expect rust to behave like numpy. If you can explain that the conversions are basically free and allow for useful features (like array multiplication with *) I think there will be a lot more support.

* zero meaning no copying or moving (in memory) of the arrays, dimensions etc might be lost/gained.

[anowell]

This issue seems like a good enough forum as any -- would it be fine if we converted it to that purpose?

sounds good to me!

a lot of people expect rust to behave like numpy

I really like the idea of using AWLY to help find the full functionality that a numpy user might expect while correcting expectations of finding that in a single crate!

[vitiral]

closing in favor of #14