search

chapel-lang / chapel

a Productive Parallel Programming Language
https://chapel-lang.org
Other
1.77k stars 416 forks source link

Mason Improvements #7106

Open Spartee opened 7 years ago

Spartee commented 7 years ago

This issue is to gather all of the ideas to improve chapel's package manager Mason. If there is something you would like to add please comment it below.

Commands to add

Features (general, others listed by command)

Mason New

Mason Update

Mason Build

Mason Run

Mason Modify(add/rm)

Mason External (Spack integration)

Mason System (pkg-config integration)

Mason Env

Mason Test

Mason Doc

Mason Search

Mason Example

Mason Help

Mason-Registry

Documentation

Tests

Build

Release

Interaction with TOML library (See #7104)

Extras

Miscellaneous

Spartee commented 7 years ago

In reference to one of the features listed in this issue, there has been discussion of how mason will handle multiple versions of chapel. So far, the idea that is the most favorable amoung team members who have discussed this is to add a field to that project mason.toml that indicates chapel version. 


[brick]
name = "MyPackage"
version = "0.1.0"
chplVersion = "1.16.0"

We have also proposed the idea of using chapel range syntax to indicate multiple compatible versions as follows


[brick]
name = "MyPackage"
version = "0.1.0"
chplVersion = "1.13.0..1.16.0"

We are welcoming all ideas and thoughts on this topic.

ben-albrecht commented 6 years ago

Extend mason doc to document dependencies

I'm wondering if this is really that useful of a feature to support..

benharsh commented 6 years ago

There should be an item for building mason with an appropriate chplenv. For example, CHPL_COMM should be set to 'none' so that users don't need to run mason -nl 1 to do something. CHPL_LAUNCHER should also be set to 'none' because mason shouldn't be running on an actual node. We ought to verify that the module build supports such a configuration.

Edit: we probably don't want to build with hugepages either, but let's try the others first.

nspark commented 6 years ago

I've only tried Mason a bit, but here are some comments on / suggestions for new features:

  1. Build on last modified

    Total 👍 Mason should be able to replace a basic Makefile w.r.t. rebuilding on changes.

  2. Related to (1), I think that mason run --build should not run the executable if the build fails.

  3. Currently, Mason discovers the name of the executable based on the project directory. I was surprised it did not use the name of the brick itself. I discovered this when I created a project and renamed the directory; mason run wouldn't run the proper binary it would build, but also wouldn't report an error that the subprocess spawned didn't properly execute.

  4. From (3), if mason run fails to run the executable (for some reason), it should report an error.

buddha314 commented 6 years ago

To help with the discussion around #8771 I thought I would point to how we are using Make and Makefiles instead of mason build right now. Here is one from Numsuch and one from Relch You can see we have some boilerplate. Our internal projects are a bit more complicated, but face some of the same issues.

ben-albrecht commented 5 years ago

I've added:

Allow excluding tests like .notest files from start_test.

in response to a user inquiry on the mailing list (@tjstavenger-pnnl)

tjstavenger-pnnl commented 5 years ago

I have an email on the developer mailing list, but it looks like "exit with an error code (ideally by returning from main) when the command failed - this is important for integration with scripts and Makefiles" doesn't happen in 1.19.0 on mason test (I'm assuming that the checkmark means it is done?). 

$mason test
...
--- Summary:  23 tests run ---
-----> 15 Passed
-----> 8 Failed
$ echo $?
0

Also, it would be nice to see the working directory used on mason test standardized too. You get the working directory of wherever you execute mason from, so if you do a mason test from the root of your project, then that is your working directory, but it is different if you run mason test from your tests directory. If your tests use relative paths, they can break.

ankushbhardwxj commented 4 years ago

Move registry off of github (centralized registry with cached packages)

I think having the registry on github is good because we can enable CI tests and would make it easier for package moderators to assess a package. However, I did think that Mason should have its own website, where the data regarding the packages can be displayed using the Github API. Also there are other ways to accomplish this. A website could have functionalities like :

@ben-albrecht What are your views on this ?

ben-albrecht commented 4 years ago

Move registry off of github (centralized registry with cached packages)

I think having the registry on github is good because we can enable CI tests and would make it easier for package moderators to assess a package.

I think the real goal is to centralize the packages, such that a package author cannot break / remove an existing working version, breaking anyone's code who relies on it, ultimately avoiding problems like the npm left-pad incident. I think we can consider this a separate feature request from where the registry is maintained. I will update the list in the description to reflect that.

The goal to move away from GitHub eventually is to reduce the burden for users to publish packages. That said, this feature request is still an open discussion. Maybe the GitHub-hosted main registry model will work well for us in the long term.

I did think that Mason should have its own website, where the data regarding the packages can be displayed using the Github API.

Agreed. I think your proposes features sound like a good first step. I encourage you to split off a new issue for us to explore the specifics further.