JuliaInv / MUMPSjInv.jl

MUMPS interface for Julia
18 stars 15 forks source link

Is this package maintained and updated? #21

Open dpo opened 5 years ago

dpo commented 5 years ago

The most recent commits that add functionality to this package go back to June 2018, and before that, to June 2017. I would just like to ask whether this package is still maintained and someone intends to bring it up to date.

If not, I would like to suggest merging this package and https://github.com/JuliaSmoothOptimizers/MUMPS.jl. I've been maintaining the latter and keeping it up to date. It builds with MUMPS 5.2.0, julia 1.0 and 1.1, and supports MPI. It's missing a few functionalities, that we could relatively easily add.

If that doesn't seem agreeable, I would like to ask if this package could be renamed so I can register mine under the name MUMPS.jl as I believe it sports a more modern interface, and is actively maintained and supported. @Pbellive tells me that this package only supports sequential MUMPS. Perhaps it could at least be renamed to reflect that fact (there's an even more dated https://github.com/JuliaSparse/MUMPSseq.jl that I don't believe is registered).

@lruthotto @Pbellive

Pbellive commented 5 years ago

Taking a bit more of a look through the JSO MUMPS package, I'm in favour of getting it registered under the name MUMPS. It seems to me like renaming this JuliaSparse MUMPS package would be a better option than merging the two packages just because this one can be installed without MPI and without having to separately download and install MUMPS. If the JSO MUMPS could be made to be installable without MPI then I would be in favour of merging the packages. I don't have time to contribute to that unfortunately.

The only other feature that JuliaSparse MUMPS has that JSO MUMPS does not is the ability to use sparse right hand sides. I should be able to help get that into JSO MUMPS in a few weeks time but not before that. Thoughts @lruthotto ?

lruthotto commented 5 years ago

I'd be fine renaming this package and use the other one as default. Looking at JSO's package it seems much better maintained than this one. In addition to the sparse right-hand sides, I'd need to check if JSO's package allows one to keep factorizations at remote workers. This is a key feature for many of our codes. I'm sure though this can be added and am happy to contribute to JSO's version.

lruthotto commented 5 years ago

One thing I'm not sure about yet (need to catch up on it) is how package registration works these days in julia. Any thoughts?

dpo commented 5 years ago

Thank @lruthotto. I think that with the new package system, packages are identified by their UUID and not by their name. So you should be able to simply rename this repository, make appropriate changes to the code, and register a new version. It would be great is someone could confirm.

The downside is that every piece of code out there that has using MUMPS will break, and will refer to JSO MUMPS once it's registered. I'm not sure how to avoid that.

And yes, it would be great to add sparse right-hand sides and factors on remote workers to JSO MUMPS. We don't have those features yet.

wrs28 commented 5 years ago

Hi all.

I think in a given registry the packages are accessed by name, and can be disambiguated by UUID, at least according to this comment in the Pkg documentation:

In case there are multiple registries named General installed you have to disambiguate with the uuid, just as when manipulating packages, e.g.

Also I only just figured this out, and it appears to be not in the documentation, but I think this GitHub app is the easiest way to register packages at the moment.

I don't want to step on any toes, but I developed a third MUMPS-Julia interface. My original need a few months ago was for the Schur-complement functionality and sparse rhs, using @dpo's brew tap for MUMPS. What I ended up with sort of "accidentally" also included most if not all of the other MUMPS functionality.

Personally I'd like to get this on the general registry so that I can easily add it as a dependency to other packages I'm working on, but I don't really care under what name it is registered. A few days ago I tried to register it under the terrible name MUMPS3 (again, trying to stay out of the way), but the registry manager gave some feedback suggesting I check in here, which is what I'm doing.

I want to confer with @lruthotto and @dpo about good ways to proceed, and perhaps encourage taking a look at MUMPS3, in case you see an easy way to merge/combine functionality. I've looked over all three packages, and it's not so clear to me how best to do that.

@dpo makes a good point that replacing JuliaSparse/MUMPS will probably break anyone's code who is relying on it. Probably you could introduce the new MUMPS (whatever it is) as a breaking release. For current users to continue using the previous code they would just ]pin MUMPS v... For users who don't yet use MUMPS, they'll never even know the previous version of MUMPS was very different.

Again, sorry to make a crowded namespace even more crowded :(

abelsiqueira commented 5 years ago

Maybe this can work:

In METADATA, and therefore in Julia before 0.7, JuliaSparse/MUMPS will be used;

The newer Julia versions will use JSO/MUMPS or possibly JuliaSparse/MUMPS-something.

abelsiqueira commented 5 years ago

Marking @fredrikekre to check if the strategy above is viable.

Regarding MUMPS3, I'm not very close to the code of MUMPS, but I know it's complicated enough that three versions in Julia exist.

Asking from the JSO perspective, @wrs28 why did you feel the need to create a new API, as opposed to implementing the things you needed in an existing package? Our code didn't work? Was too ugly? Clashed philosophically? The code started with early Julia, so it may need a revamp.

For the betterment of MUMPS use in Julia: can we find common ground? Who among us (and others - ping @lrsantos11) accepts maintaining MUMPS in Julia code in the future? Who is using MUMPS in Julia?

dpo commented 5 years ago

I maintain the MUMPS Homebrew formula for macOS and Linux, and I maintain and use JSO/MUMPS. I'm very open to pull requests that improve the interface and make long-term maintenance easy.

wrs28 commented 5 years ago

@abelsiqueira, Re: new API: I don't recall anymore specifically why I started working outside an existing package, but I am certain it boiled down to expediency, and had nothing to do with aesthetics or philosophy. I started by trying to work from within JSO/MUMPS, but I had trouble connecting the MUMPS documentation with what I saw inside JSO/MUMPS, and therefore it wasn't clear to me how to get what I needed. As I mentioned, I actually did not mean to create a new API, just to access a few features, I believe none of which were implemented in JSO: Schur-complement, sparse rhs, inverse of select entries. From reading the MUMPS documentation and the Julia documentation on C-wrappers, it seemed that the easiest way for me to get what I needed was to use the C-interface of MUMPS. It was something of a side-effect that this potentially gave full access to all the features as documented in the MUMPS manual.

wrs28 commented 5 years ago

@abelsiqueira @dpo @lrsantos11

Re: MUMPS usage: I use MUMPS (and/or Pardiso) for two features: first is to accelerate and reduce memory usage (both over UMFPACK) in repeated solves, often with sparse RHS, and the second is to use the Schur-complement calculation in transport calculations. In both cases the matrices come from finite difference calculations.

Re: common ground: At the risk of stating the obvious, I propose that the ideal solution is one in which:

  1. as much of MUMPS is exposed to users as possible in a way that aligns with the MUMPS documentation, such that the Julia interface requires next-to-no knowledge to use beyond knowing MUMPS itself (this is for the "expert" users)
  2. many convenience wrappers are provided for non-expert users who just want to take advantage of the MUMPS speed and memory advantages without needing to know anything detailed about MUMPS
  3. it is easy to adapt to new versions of MUMPS, and is obvious how to extend from reading the MUMPS documentation
  4. the majority of the current users of MUMPS have few or no changes to make in order to maintain their current code.
  5. a minimal amount of work will suffice to get there. We've all spent time developing and using our own solutions, and probably no one wants to sink even more time into it.

Re: maintenance: @dpo has been very active in maintaining both homebrew and JSO: thank you!

dpo commented 5 years ago

@wrs28 Perhaps it wasn't so easy to use JSO/MUMPS because there's an intermediate C interface called mumps_simple that I figured would also make it easy to interface MUMPS with other languages. Strangely enough, your wrapper also links with mumps_simple?! However, I like the idea of interfacing the C struct directly but I wonder how to do away with the GC_haven structure. It seems your API has substantial overlap with ours. Would it make sense to try and merge your improvements into JSO/MUMPS? If we do it gradually, it should be minimum effort. In my research group, we can pretty much guarantee continued support for the interface.

lruthotto commented 5 years ago

Hi y'all. How are we moving forward to resolve these issues? I've gone through and updated this package so it can be used with the current julia. Many of my other codes require the setup in this package (remote factorization, sparse rhs) and so it's more time efficient for me to keep updating it.

I think though that this package is not ready/useful for mainstream applications. I'm wondering if we should transfer the ownership to our JuliaInv group and rename the package as jInvMUMPS, since it is geared toward our jInv package. This way, MUMPS.jl could refer to JSO's package or someone else's. What do y'all think?

dpo commented 5 years ago

Sounds good to me.

lruthotto commented 5 years ago

OK, I did the proposed change. @dpo: You should be able to register your package under MUMPS.jl. The name of our package has now been changed.

@JuliaRegistrator register()

dpo commented 5 years ago

Still no luck: https://github.com/JuliaSmoothOptimizers/MUMPS.jl/commit/35ac4c77322b819ea360311137a6afe9abf2dd1c#commitcomment-34191000

dpo commented 5 years ago

MUMPS is still in the registry: https://github.com/JuliaRegistries/General/tree/master/M/MUMPS

abelsiqueira commented 5 years ago

I've created a pull request for the url and name change: https://github.com/JuliaRegistries/General/pull/1798

abelsiqueira commented 5 years ago

URL and name changes are merged. Stefan left the following relevant message:

Note that any new versions of this package that are released after the move will be strictly incompatible so the next release should be a major one and you need to make sure you cap all packages that depend on this at the old version when that's done.