ScientificD

[kyllingstad]

I have created the ScientificD "organisation" here on GitHub. I hope that, in time, it will become a nexus for D's scientific programming community, and a natural place to host code related to numerics and science – like D-Programming-Language is for the core language and D-Programming-Deimos is for C library bindings.

I don't have a clear opinion on how best to organise this, but I think it is a good idea if each library gets its own repository. That way someone can associate their library with ScientificD, yet have full control over the code that gets committed to it. Personally, I intend to split the current SciD into several libraries (one for quadrature, one for differentiation, one for nonlinear equations, etc.) and host them all at ScientificD.

Some functionality will most likely be useful for several libraries, yet still too niche for Phobos. For those things, a "core" repository should be made, upon which the other libraries can depend. (Of course, the maintainer(s) of such a library should be very careful about keeping backwards compatibility as much as possible, so they don't break a bunch of other libraries with a commit.)

Opinions, anyone? Ideas about organisation? Is the "one repo per library" idea a good one? If so, should there still be a common build system?

[andralex]

Sounds great! How do I follow an organization?

[kyllingstad]

I don't think you can (although that might be worth an enhancement request to the GitHub devs). You'll have to follow the organisation's repositories and/or members. Anyway, not much is happening there yet, I've only created it to reserve the name.

[dsimcha]

Hi Lars,

I remember you created the ScientifcD group a while ago. I appreciate your efforts at organizing this. I forgot about it because I got sidetracked from these high-level organization issues by lower-level stuff like fixing bugs in and improving Cristi's GSoC fork of SciD and my Dstats lib.

On 2/19/2012 3:37 PM, Lars T. Kyllingstad wrote:

I have created the ScientificD "organisation" here on GitHub. I hope that, in time, it will become a nexus for D's scientific programming community, and a natural place to host code related to numerics and science – like D-Programming-Language is for the core language and D-Programming-Deimos is for C library bindings.

I don't have a clear opinion on how best to organise this, but I think it is a good idea if each library gets its own repository. That way someone can associate their library with ScientificD, yet have full control over the code that gets committed to it. Personally, I intend to split the current SciD into several libraries (one for quadrature, one for differentiation, one for nonlinear equations, etc.) and host them all at ScientificD.

I'm somewhat skeptical of the benefits of splitting things up too much.
That would require dependency tracking, etc. I actually had the opposite goal: To merge my statistics stuff with Cristi's matrix stuff and your quadrature and differentiation stuff into a one-stop shop, kind of like Numpy/SciPy. With regard to the core below, maybe a good middle ground would be to have a "core/satellite" model, where core includes everything that's stable and the "satellite" libraries are supposed to depend only on core but not each other and are more beta quality.

Some functionality will most likely be useful for several libraries, yet still too niche for Phobos. For those things, a "core" repository should be made, upon which the other libraries can depend. (Of course, the maintainer(s) of such a library should be very careful about keeping backwards compatibility as much as possible, so they don't break a bunch of other libraries with a commit.) I agree that long term, if we split things like this, we'll probably want a core library. The main thing we probably want in core is matrices and I don't think the matrix library is stable enough yet to be constrained by backwards compatibility.

Opinions, anyone? Ideas about organisation? Is the "one repo per library" idea a good one? If so, should there still be a common build system?

I would like to standardize on something for build systems, but I have no idea what. Please, no make files. I'm open to any other suggestion as long as it's easy to use. I actually liked your rdmd build system for SciD and we kept it, with modifications, in Cristi's fork.

[MaksimZh]

The first part I'd like to see a separate project is the regionallocator Is its interface stable enough for that?

I hate managing workspaces and this package just makes all the troubles go away :) I really like it!

[kyllingstad]

@dsimcha:

I appreciate your efforts at organizing this. I forgot about it [...]

Being busy with a lot of other stuff, I forgot about it too, until @MaksimZh reminded me. ;-)

[...] maybe a good middle ground would be to have a "core/satellite" model, where core includes everything that's stable and the "satellite" libraries are supposed to depend only on core but not each other and are more beta quality.

That's actually a pretty good idea. The ScientificD organisation could both act as the home of the core library and as an incubator for new libraries. As the various "satellites" mature and stabilise, they'd be included in the core (after a review process sort of like the one we have going in Phobos land).

Even with this approach, however, I would probably still want to split up the current SciD codebase, and only keep a few things in core. There's a lot of code in there which I would not be proud of calling "stable".

I would like to standardize on something for build systems, but I have no idea what. Please, no make files. I'm open to any other suggestion as long as it's easy to use. I actually liked your rdmd build system for SciD and we kept it, with modifications, in Cristi's fork.

Yeah, I hate makefiles too. That's one of the great things about D: Its expressiveness allowed me to write that build script with only minor effort, and since then the only maintenance I've done on it is to keep up with a few changes in DMD and Phobos. I guess it would be a fun project to extend it to cover a multi-library setup. :-)

[MaksimZh]

New age is coming: https://github.com/cristicbz/scid/wiki/Draft-GSoC-2012-Proposal :-) Should we comment on new design here or create another issue for that?

[WebDrake]

If ScientificD is to be a modular project as you describe, are there any licensing constraints that library developers should adhere to? Or any policies on dependencies?

For example, one good contribution would be an arbitrary-precision arithmetic library. One way to do this might be to build D wrappers to GMP. This is LGPL-licensed, so wouldn't place any constraints on the licensing of the D code or of code written by downstream users, but it's readily possible to imagine other GPL-licensed examples. Alternatively, is there a desire to entirely avoid non-D dependencies regardless of licensing issues?

[kyllingstad]

I don't think that the ScientificD organisation should place any constraints on licensing (beyond the fact that the projects should be open source, of course). However, I think the "core" library should be licensed under the Boost license (or something equally or more permissive), and anyone who wants to see their library in core should conform to that.

By the same token, I think library authors should be free to have their libraries hosted under the ScientificD organisation even if they depend on third-party libraries. It is, IMO, a bad idea for the core library to have too many dependencies, but this should be considered individually for each case. For instance, I think that a dependency on BLAS and LAPACK is OK, and even preferable to writing one's own linear algebra routines. At the very least, any core library dependency should be available for both POSIX and Windows platforms.

[MaksimZh]

What about Cristi's proposal? Is it supported? I hope it is not postponed for next summer. Anyway I really want to use such linear algebra library and will do my best to help with it.

[kyllingstad]

Well, this never went anywhere, but DlangScience seems to have gotten off to a nice start. Closing this discussion now.