trigger upgrade warning for older backends

[wilzbach]

I am pretty sure people have come up with the idea to be able to upgrade D compilers via dub before, but I couldn't find an issue. Giving it similar logic to @MartinNowak's famous install.sh would be nice because updating to a new release will hopefully become a lot easier. Of course then the binaries are downloaded into the user folder. I imagine something like the following should work for the user:

dub
dub set ldc.version 1.0.0-beta2 // set the global default version
dub --compiler=ldc-1.0.0-beta2 // build with a specific version, download it if not available

Maybe it also makes sense for a package description to be able to specify their "working" compilers, s.t. dub can pick one and download it if none is available.

Btw if we already talk about it - wouldn't it be nice if dub can upgrade itself? Other package like NPM have this self-bootstrapping too ;-)

npm install -g npm

[mihails-strasuns]

Can we please create new tool, call it something like dhipster and remove from dub everything that is not related to source dependency management? And move all trendy crap there? Nothing personal but you keep consistently proposing all the worst features of scripting language ecosystem which were always among major reasons I stayed miles away from them.

I have kind of accepted that developers now are usually excited about crap like that - it is a lost cause. What I won't accept is that tools I have to use myself for my own job get polluted as a result to the point they have to be forked to remain usable. Though separation of concerns and UNIX way seem to be out of fashionable trend too.

[rikkimax]

This issue should be closed. We have a tool that already handles dmd (dvm) but should be extended to support ldc and gdc as well.

[mihails-strasuns]

(worth noting that https://github.com/Cybershadow/digger has similar functionality too)

[wilzbach]

Can we please create new tool, call it something like dhipster and remove from dub everything that is not related to source dependency management? And move all trendy crap there? Nothing personal but you keep consistently proposing all the worst features of scripting language ecosystem which were always among major reasons I stayed miles away from them.

Fair enough - my point was actually that there are many packages out there which assume that you have got at least frontend of X, because we all just use the latest, greatest version of dmd. However there those unlucky people that are either in a corporate setup, Windows or Ubuntu/Debian that usually don't have a recent frontend. Hence in such cases dub will throw you ugly errors and I strongly assume that this is a very unpleasant experience - especially someone new to D. Here's one example of such an issue: https://github.com/Hackerpilot/Dscanner/issues/354

So maybe as a solution we could introduce a new field "minFrontendVersion" (or similar) that is automatically set on dub init (otherwise people will forget) and checked when the dependency is installed. A simple warning like "Hey your dmd compiler seems to be outdated and not supported for this package. If you run into troubles, please upgrade via http://dlang.org/download.html before you report a bug" could already be a simple solution?

[s-ludwig]

Since we plan to integrate CI into the registry, we could pursue our earlier idea to fully automate this. A manually edited field is almost guaranteed to get outdated anyway for all but the most carefully maintained packages.

BTW, I could have sworn that there was a ticket for this already, but can't find it. There was also the idea to integrate DVM support for automatically switching DMD versions. But IMO it would be good to have LDC/GDC support in DVM for that.

[dejlek]

I humbly believe it would not hurt if dub could install D compiler (into local, user's environment) if it is missing.

[MartinNowak]

I humbly think that properly installing a compiler is more complicated than it sounds and integrating that into dub will add another huge maintenance/support drag. Also having dub use a different compiler opens a possibility for differences in configurations, which is a great source for confusion and reproducibility issues. Given the amount of bugs, usability, and stability issues we already have with dub, we should really KISS.

Not a fan of dvm since it literally never works when I try it, mostly due to relying on the hardly maintainable mambo library. It's also yet another point that requires backwards compatibility for any release/download/versioning changes, since we can't update dvm as easily as install.sh.

BTW, I'm not aware of any other language (bundler, npm, sbt) that tries to integrate compiler installation, it's always sth. that the user does, there are too many choices.

That said, a min frontend version required by a package might be a useful addition to dub.sdl/dub.json to give better error messages, such as your selected dependency vibe-d-0.8.0 requires D >= 2.075.0, but the used compiler is 2.069.1. If it could warn about gdc, ldc requirements all the better. It could even point people to some landing page (e.g. http://dlang.org/download or a wiki page).

Such a feature would have a bit of forward compatible issues if we ever decide to split dmd and phobos releases and versioning.

[jacob-carlborg]

Not a fan of dvm since it literally never works when I try it

If you don't file any issue is not very likely that it will be fixed. I don't know what's not working for you. As always, pull requests and contributions are welcome to fix any issues.

mostly due to relying on the hardly maintainable mambo library

Yeah, I could do a better job there, perhaps remove it completely. Although, all the constant regressions don't help. But I'm not sure that I would like to replace the parts of Tango used in DVM with the corresponding functionality from Phobos. It's mostly the HTTP package, used to download the compilers. The HTTP package in Tango doesn't have any external dependencies while the corresponding functionality in Phobos requires libcurl.

It's also yet another point that requires backwards compatibility for any release/download/versioning changes, since we can't update dvm as easily as install.sh

Please, feel free to submit any pull requests. Note that one of the goals of DVM is to have very few runtime dependencies. Removing the ZIP archives and only provide the tar.xz would require some library to be able to extract those archives. I would really like to avoid requiring to execute tar as a shell command.

BTW, I'm not aware of any other language (bundler, npm, sbt) that tries to integrate compiler installation, it's always sth. that the user does, there are too many choices.

For Bundler, it's possible to specify the version of Ruby in the Gemfile (corresponds to dub.sdl/json). But it won't install Ruby.

I think it would be nice to have some way to indicating a minimum version of D required, or a specific compiler.

[PetarKirov]

And still, install.sh does a better job than both on posix because it works out of the box and supports dmd, dmd-beta, dmd-version, dmd-nightly, dmd-branch, ldc, ldc-beta, dub, and can update itself :P

If it only supported gdc and worked on Windows outside WSL/LXSS it would be perfect!

[jacob-carlborg]

I think I filed a segfault in DVM like 5 years ago that was still there since I last tried using it.

@CyberShadow I found one bug reported by you [1], which is not a segfault.

FWIW here it is (no public version control yet, sorry): https://dump.thecybershadow.net/4ef904f2074e97a0a5969090f074fbb1/dver.d Supports Windows, Linux, Beta releases, 32/64 bits, downloads things as needed, supports every version back to D 1.00, running Windows versions under Wine on Linux, has a sane command line, does not segfault, all under 160 lines.

Unlike DVM, this one:

• Does not work on all the official D platforms (macOS and FreeBSD are missing)
• Hard coded to your specific system
• Requires external tools like aria2c
• Seems like it's not setting executable permission for all executables

About the the 160 lines, I see that you forgot to count the lines of the external modules, ae.util.* and ae.sys.* and the external tool aria2c.

I've used DVM since 2008 to install every DMD compiler I ever used. I use it on every new system, virtual machine or Docker container. It works perfectly fine and does not segfault, at least not the way I use it.

[1] https://github.com/jacob-carlborg/dvm/issues/18

[jacob-carlborg]

And still, install.sh does a better job than both on posix because it works out of the box

@ZombineDev out of the box of what?

install.sh unfortunately, as you mentioned, doesn't work on Windows. It also heavily depend on external tools, make assumptions of the current system and, finally, is written in Bash 😱.

[PetarKirov]

Well it is a bash script after all, how do you expect it to not depend on external tools - that's whole point of writing scripts :D Though, except for the requirement on curl, you would hard-pressed to find a Posix system where it wouldn't work out of the box. Also given how PITA is maintaining dependencies such as OpenSSL, I think it's much better to leave the task of downloading to the system. (Downloading over unencrypted protocol is a big NO.) On Windows both batch and PowerShell scripts can download files out-of-the-box. So as much as I hate ugly scripting languages, I think using them is more pragmatic than requiring the user to download a D binary, which may or may not work with TLS/SSL.

[MartinNowak]

Let's get back to the topic, any good argument for dealing with the install complexity in dub itself? We already have https://github.com/dlang/dub/issues/108 for the compiler specification.

On Windows both batch and PowerShell scripts can download files out-of-the-box. So as much as I hate ugly scripting languages, I think using them is more pragmatic than requiring the user to download a D binary, which may or may not work with TLS/SSL.

Yes, I'm still trying to find someone writing the PowerShell version of that script. http://forum.dlang.org/post/56592679.3010604@dawg.eu http://forum.dlang.org/post/gcuwwuhbbrmgltnoidsa@forum.dlang.org Could use

(new-object Net.WebClient).DownloadString("https://dlang.org/install.ps1") | iex

I'm not aware of download mechanism in batch suitable for copy+paste though. But using PowerShell shouldn't be asking for too much nowadays.

If it only supported gdc and worked on Windows outside WSL/LXSS it would be perfect!

It does support gdc.

It also heavily depend on external tools

Effectively tar and curl (and grep for a tiny use-case). It also has a friendly check_tools function to inform ppl. about missing tools.

make assumptions of the current system

Which ones

and, finally, is written in Bash 😱.

Just because it's not D doesn't make it a bad tool 😄.

Writing a tool that's supposed to be a simple D compiler installer in D kind of misses the point, downloading an executable to download an executable similarily, in particular for a first-time D user. I might remind you of the disaster to propose DVM to Travis-CI https://github.com/travis-ci/travis-ci/issues/730#issuecomment-30732299, ending with the poor guy trying to compile dsss :o. Requirements for a tool to switch compilers are a bit different from a website install script though, not sure different enough to mandate a separate tool though.

NB: Before the argument comes up, it is safe to pipe a script from curl into bash, the whole script is wrapped into a function to avoid executing only half of it (see here). If you don't trust dlang.org's certificate, you can download install.sh and install.sh.sig separately, audit the script, and verify it against our gpg keys.

The HTTP package in Tango doesn't have any external dependencies while the corresponding functionality in Phobos requires libcurl.

Not a liability, libcurl is everywhere. We're loading any possible libcurl version at runtime to avoid previous issues with versioned symbols (libcurl-gnutls vs. libcurl-openssl).

Also, FWIW, Digger may be an alternative if there is interest in going in that direction.

Digger has a great track record of working (at least for me), but also has the bootstrap issue.

[CyberShadow]

Let's get back to the topic,

Sorry for the deviation. I've deleted my off-topic comments.

Digger has a great track record of working (at least for me), but also has the bootstrap issue.

Sorry, what do you mean? If you mean that it's written in D, Digger has binary releases; and it can bootstrap building compilers from either C++ source or using a published binary release of D.

[jacob-carlborg]

Let's get back to the topic, any good argument for dealing with the install complexity in dub itself?

No. #108 looks like a good idea.

Just because it's not D doesn't make it a bad tool

No, not in my opinion, but in everyone else opinions in the D community 😉. I'm not against it not being implemented in D, I'm against it being implemented in Bash. As far as I know, the only option to create a cross-platform tool, that works both on Windows and all Posix out of the box, is to build a native binary. If you want to write it in C, C++, Go, Rust or something else, I won't stop you.

I might remind you of the disaster to propose DVM to Travis-CI

DVM didn't support 64bit at that time, just as the install script doesn't support Windows.

Not a liability, libcurl is everywhere.

Not on Windows.

[Geod24]

I'm going to go ahead and close this.

Reasons:

• A significant portion of the core team is against it (disclaimer: me included);
• We already have many ways to install D compilers (dvm, digger, install.sh, distribution packages, snap packages...);
• Each way have their own issues (versioning, compatibility with other tools, parallel installation of version, etc...);
• No one has worked on it for 3 1/2 years;

Now if someone opens a PR with a cross platform, comprehensive solution that works 100% of the time, I'm sure we'll all be happy, but I hope we can all agree that we have enough highly controversial and insanely time consuming enhancement requests opens as it is.