Closed ghost closed 5 years ago
Dune is able to deal with several opam packages, with different metadata, so I'm not sure that dune-project
is the best place to handle this (for single-opam projects, it's OK of course).
For multiple package we can simply group things inside (package ...)
stanzas:
(package foo
(depends ...))
(package bar
(depends ...)
(description ...))
Missing fields would be inherited from the global ones.
I think that putting this information in dune-project
is quite important, since it means that we can adopt a different generation strategy for other non-opam packaging formats. For example, creating deb
packages would require partitioning out the bytecode and native code files separately, and creating a different "cut" of packages (e.g. lwt subpackages might be folded in).
In the longer term, I can see dune-project being the place where larger projects can package up codebases in a way that is suitable for different upstream requirements. So it might be interesting to also write down a syntax for one other format (perhaps esy
or rpm
or deb
) to see what the metadata differences look like. We could immediately generate Ubuntu PPAs if we generated Deb's easily, and interoperate with system packages...
My understanding of the current proposal is that the opam build
entry is hardcoded to essentially "dune" "build" "-p" name "-j" jobs
, right? Is that intended? I don't know what fraction of packages can get away with just that, but it's certainly not all of them.
There are indeed a few packages that need to run a manual configure phase first. I'm pretty sure we can do away with those in a future revision of Dune, but it's a non-zero number at the moment.
That's intended yes. When these steps are necessary, it tends to break the composability aspect of Dune, which isn't good. I also believe that we can extend Dune so that such extra steps are not necessary. it just means that in the meantime such packages won't be able to use (using opam 1.0)
in their dune-project
file. If that's a problem we can always add a mechanism to keep a few fields under the control of the user.
So it might be interesting to also write down a syntax for one other format (perhaps esy or rpm or deb) to see what the metadata differences look like.
In the context of Debian, it seems difficult to use the same metadata, at least for official packages. The version numbers might be different, there might be a distinction between source and binary packages, etc. Probably this will be handled by the distribution itself using dh_ocaml
or similar tools.
I think that putting this information in dune-project is quite important, since it means that we can adopt a different generation strategy for other non-opam packaging formats.
I do agree. But I think it is slightly better to have a build-system agnostic configuration at the top level, but one that can be extended by build systems and any other tool imaginable. Just as an example, we have package.json
at the top of our projects, but we can store any fields there that build systems or dev tools (like merlin) could look for. I wouldn't want to call that file a merlin-project.json
, or dune.json
- it made more sense for it to be called package.json
and I would have rather it been even more generic (project.json
). Is the dune-project
proposed here kind of like opam
but in sexp? If so would it make more sense to think of it as sexp_of_opam
?
Just as an example, we have package.json at the top of our projects, but we can store any fields there that build systems or dev tools (like merlin) could look for.
That's basically the idea of dune-project
: personally, I want dune-project
to be the only configuration file I have to write and then let dune generate everything else. For instance I was thinking of adding support for generating a package.json
file so that we can use dune projects with esy out of the box.
For instance I was thinking of adding support for generating a package.json file so that we can use dune projects with esy out of the box.
The common workflow with esy
and dune
is to define dune
package as a dependency and then make dune
available in the esy environment. This means dune
isn't available w/o esy
in such case.
I think it makes more sense to add to esy
an ability to read project metadata directly from dune-project
files (similarly how it does read it from package.json
or *.opam
files).
I think it makes more sense to add to esy an ability to read project metadata directly from dune-project files (similarly how it does read it from package.json or *.opam files).
That would be awesome! Dune can provide an API to easily read dune-project
files
Yeah, those are cool ideas too. I would mention though that the ideal project descriptor that we encourage people to use would not sound as if it's specific to any one technology. I liked that npm chose the package.json
name because technically it's not only used by the package manager but also build tooling to configure things. That's one the reasons (not all the reasons) why we encourage a package.json file. If you wanted something similar, I would suggest naming it something more generic like project.sexp
(even npm
doesn't include the word "npm" in package.json
). Sad thing about node_modules
though!
Agree about choice of the name.
Also it would be nice if dependencies declared there can be fully qualified as from which source they come:
(depends
(opam lwt)
(opam lwt_react >= 1.0)
(github user/repo beta)
...
)
I'm not sure the above belongs to dune project files. Specifying the dependencies which a dev used to build a package on his own box is a property of the workspace rather than the project. Users of the project are quite likely to want to fetch dependencies from different sources. The resolution of dependencies isn't something that we should hard code in a project file.
It's not resolution of dependencies but an ability to specify dependency which is hosted on github. Instead of github
it could be (npm reason >=3.0.0)
and so on.
My point was that it shouldn't be opam specific if dune-package
is going to support various packaging systems as stated in the motivation for this issue.
The idea is that the names we write in the dune-project
file are abstract package names. These abstract names can them be mapped to real packages by using one or more package managers such as opam, npm, debian, ...
For instance, if package lwt
exists in both opam and npm, is there any reason to prefer one of the two? If not, it seems to me that the precise source of the package shouldn't appear in the dune-project
file.
That said, it would indeed be nice to allow to provide a precise source for projects that are not yet released.
This will be released in 1.10. There are still some features missing, such as validating the depends
fields, but the proposal is mostly implemented.
This ticket describe a proposal to let dune generate the opam files. They are several goals for such a project:
While 3 is feasible, it is a lot of work and can be considered as a long term goal. This proposal only covers 1 and 2, which is already an improvement, especially as many fields of opam fields are now always the same.
Example
Let's take the current opam file of utop:
and
utop.descr
file:The goal is to have only this
dune-project
file instead:Then, because of
(using opam 1.0)
, dune would automatically generate theutop.opam
andutop.descr
files in the source tree, and possibly even commit them.Notes
A couple of the fields in the
dune-project
file above such aslicense
anddescription
are pure informative fields, however for all the other ones, dune could reuse the information. For instance the constraint onocaml
could be used to automatically decide what opam switches to test against when a special command line flag is passed. The name of the project, homepage, ... would be derived from(github_project ocaml-community/utop)
.