Closed Simn closed 6 years ago
The Haxe compiler should not have to call a separate tool to obtain the information required for compilation.
The Haxe compiler should not have to call a separate tool to obtain the information required for compilation.
That means the compiler should be able to transform -lib
tags into non-lib
tags on its own, with the resolving logic being simple yet generic.
Criteria:
NPM isn't enough ?
The user should be able to use an alternative package manager (without making change to or replacing the haxe exe).
The compiler should expose an interface for frontends like CLIs, IDEs, vscode-style language servers, and so on.
Whatever ends up as standard mechanism to invoke the compiler for one-shot compilations must be very fast, there should be no noticeable difference to invoking the compiler directly (in other words start-up times like that of nodejs or even the JVM are out of the question)
The standard mechanism to invoke the compiler will be a first point of contact for newcomers to the language. As such it should provide an outstanding user experience and documentation can't be an afterthought.
We shouldn't change the current workflow and so not introduce any steps between changing a hxml and running it/doing a haxe command by hand.
A migration plan is required:
Beyond building projects the standard mechanism to invoke the compiler should provide a standardized way to run other common tasks like tests, benchmarks, updating dependencies, etc. (much in the spirit of cargo).
haxelib.json should be extended, proposed new fields (non exhaustive):
That means the compiler should be able to transform -lib tags into non-lib tags on its own, with the resolving logic being simple yet generic.
Because -lib
is the only thing that cannot be "consumed directly" by the compiler.
The following fields should be removed from haxelib.json (or its replacements) and dealt on the library repository website:
since it doesn't make sense to update those by making new releases.
Interesting, what are the arguments against tool test
, tool bench
, tool update
.. ?
I know @damoebius' suggestion to use npm has been firmly downvoted but it isn't that silly - alternatively it would be interesting to have a npm-compatible haxelib API so we could be really interesting to have.
Configuration must be thoroughly documented and transparent. On-disk storage such as hidden .dev, and .current files should be discouraged and instead replaced by a documented configuration file.
The final goal is to make it so that alternative library managers can use/manipulate the same library database without having to know the details of the previous manager.
Interesting, what are the arguments against tool test, tool bench, tool update .. ?
Out of scope (except updates, but that's obvious). This is about a package manager, not a complete build tool, which would be one level higher.
The final goal is to make it so that alternative library managers can use/manipulate the same library database without having to know the details of the previous manager.
That's too restrictive. While some package managers might agree on one database layout, others might not. A package manager using e.g. Nix as a backend will require a different layout than one based on a git repo layout, both of which would be reasonable choices to make.
If what you mean is a common layout for a staging area, as well as for ex- and importing a package database to allow migration between package managers, OTOH, that'd be a good idea.
Interesting, what are the arguments against tool test, tool bench, tool update .. ?
Out of scope (except updates, but that's obvious). This is about a package manager, not a complete build tool, which would be one level higher.
I was under the impression we're talking about all the (future) components required for the workflows that haxelib is currently involved with, not just its role as a package manager.
If this is about package management only, I think we should look at the more principled approaches - so basically read A modular package manager architecture, conclude that we're probably not going to come up with something better, and hence "do whatever opam does".
The final goal is to make it so that alternative library managers can use/manipulate the same library database without having to know the details of the previous manager.
That's too restrictive. While some package managers might agree on one database layout, others might not. A package manager using e.g. Nix as a backend will require a different layout than one based on a git repo layout, both of which would be reasonable choices to make.
It is and it isn't. While I respect the decision of writers of the extended or alternative package managers to create the solutions they think most appropriate, I believe that the canonical package manager (the one delivered as part of the Haxe toolset) must set an open standard that other tools can inter-operate with.
@sangohan @jcward: I removed your comments because they are way too wordy/not actionable. I've pasted them here so they are not lost: https://gist.github.com/Simn/2b13809d830cfd16dd5ba2f9c97cac6b
licenses
sub method to output all dependency licenses much like PHP's composer licenses
.I recently discovered https://lib.haxe.org/p/hmm library which does a pretty nice job on that part with current limitations.
For me, the current haxelib version isn't really far from what I need, but here is my wishlist :
It should be possible to open a directory that a project is in, and when I type haxe build.hxml
it automatically uses the configured version of Haxe for the project, and the right set of haxelibs/dependencies.
(This one might be controversial but I thought I'd throw it in!)
If I install a new dependency, upgrade a dependency, or change my dependencies in any way, this should show up in a way that is tracked in a file in my version control, and can be used to restore the same state on a different computer.
(Similar to a yarn.lock
or haxe_libraries/*.hxml
in haxeshim/lix)
I think we should deprecate haxelib run
to avoid depending on neko.
I think we should deprecate haxelib run to avoid depending on neko.
Don't have to deprecate it, but it shouldn't require a run.n. It could, by convention, make haxe interpret a Run.hx script instead.
(Similar to a yarn.lock or haxe_libraries/*.hxml in haxeshim/lix)
Can't put that in version control since it contains computer specific path into it. EDIT: not actually the case, see @jasononeil comment below
As an alternative to a new file (like yarn.lock) maybe lock the versions directly in the hxml?
@ibilon here is how lix deals with it, no absolute paths:
# @install: lix --silent download "haxelib:hxnodejs#4.0.9" into hxnodejs/4.0.9/haxelib
-D hxnodejs=4.0.9
-cp ${HAXESHIM_LIBCACHE}/hxnodejs/4.0.9/haxelib/src
-D nodejs
--macro allowPackage('sys')
--macro _hxnodejs.VersionWarning.include()
Here is how yarn.lock deals with it (basically, it doesn't, it just uses relative paths):
angular-cookies@^1.6.4:
version "1.6.4"
resolved "https://registry.yarnpkg.com/angular-cookies/-/angular-cookies-1.6.4.tgz#c28f3f6aac7a9826c1e45f1d6807240036e5b26d"
The package manager should use secure protocols (HTTPS vs. HTTP) for downloading packages (I found this to place limitations on the choice of a suitable runtime).
From Haxe code it should still be possible to know which libs are used. so doing #if (libname > 4)
. It would also be useful to be able to know which classes come which library (in macros/ documentation xmls/dependency dumps etc)
I have many feelings on haxelib, but quickly l want to share some "wish list" items:
-lib lime:3.1.*
)Multiple remote repositories would allow companies to generate their own internal versions and distribute more easily, or enable "beta" or "dev" channel repositories.
Maintaining a single class path directory would not be reasonable for dealing with source files. Similarly, it would be nice to be able to add multiple custom library paths -- similar to how class paths are resolved.
Support for wildcard versions would really, really help (especially if semver is respected). You can request library version 1.0.*
, allowing for patch releases. This is a great middle-solution between "Give me version 1.0.4 exactly" or "Give me any installed version", and a frequently requested feature for OpenFL
As a cherry on top, it would be nice if libraries could install command aliases, such as lime
for haxelib run lime
or as3hx
for haxelib run as3hx
The best package manager in the world will need updates. I would like to see a library that is self-updated when you run a command similar to haxelib update
@clemos haxelib run
isn't a problem per se, but it should be extended to other runtimes, and be configurable in haxelib.json
.
Caching libs is good, using symlinks is bad: someone could mistakenly modify a file and it would affect the cache / other projects. Libs should be copied locally in the project (unless optional CI optimisation to use cache).
@Simn @elsassph I agree it's not so much about about deprecating haxelib run
itself, but rather about deprecating run.n
in favor of Run.hx
;)
The package manager should allow custom fields in haxelib.json It would be very usefull for librairies which are tied to a specific framework or runtime. i.e : For externs, we could provide a link to the original library and infos about how to integrate it in your haxe project i.e : a web framework which loads plugins (haxelibs) that have various assets ( like models, controllers, templates and translation files )
I would like to point out that it is important to consider how to handle "native code" (could be compiled ndll, C/C++/JS/... compiled libraries or source files).
One reason is that it is wasteful to download binaries of all architectures. e.g. Lime's ndll
folder is 108M, but if you're only using Windows, you actually only need the 5.1M ndll/Windows
folder.
We should also have some guidelines regarding to how to store the info of those "native" things, e.g. where is the original source, what param is used when building (use of -PIC/-O3/debug info), any additional dependencies that is not bundled etc.
Build tools (hxcpp,hxjava,hxcs) should not rely on haxelib too. They can be invoked by Context.onAfterGenerate or haxe --run
Stage 1 has concluded, I'll prepare the discussion part for Monday.
Foreword
Issues in this repositories tend to get bogged down in long-winded discussions without much of a result. We would like to try a slightly different approach which is outlined here: https://github.com/HaxeFoundation/haxe-evolution/wiki/Haxe-Projects
We are in Stage 1 here, which means that comments should be short and to the point as mentioned in the document. If you want to discuss the procedure, please open a separate issue.
Now on to the actual project.
Haxelib replacement
We are looking to replace haxelib with a better package manager solution.
Please share your ideas and concerns in a concise manner. Please also +1/-1 individual comments that you agree/disagree with.