dlang / dub

Package and build management system for D
MIT License
678 stars 227 forks source link

GNU/Linux [GDC/DMD] RangeError in path.d #601

Closed s-daveb closed 9 years ago

s-daveb commented 9 years ago

Similar to issue #137, my dub binaries throw RangeErrors when they are built with GDC or DMD. When built with LDC2, everything works perfectly.

Even when building dub with dub, this error manifests...

~/.../git/dlang/dub $  dub --compiler=dmd
....
iterating dir /home/bittwiddler/.dub/packages/ entry inifiled-0.0.3
Collecting dependencies for dub
Missing dependency vibe-d 0.7.23 of dub
Checking for missing dependencies.
Downloading metadata for memutils
Getting from http://code.dlang.org/packages/memutils.json
Getting http://code.dlang.org/packages/memutils.json...
Downloading metadata for libevent
Getting from http://code.dlang.org/packages/libevent.json
Getting http://code.dlang.org/packages/libevent.json...
Downloading metadata for vibe-d
Getting from http://code.dlang.org/packages/vibe-d.json
Getting http://code.dlang.org/packages/vibe-d.json...
Downloading metadata for openssl
Getting from http://code.dlang.org/packages/openssl.json
Getting http://code.dlang.org/packages/openssl.json...
Downloading metadata for libev
Getting from http://code.dlang.org/packages/libev.json
Getting http://code.dlang.org/packages/libev.json...
Downloading metadata for libasync
Getting from http://code.dlang.org/packages/libasync.json
Getting http://code.dlang.org/packages/libasync.json...
Configurations used for dependency resolution:
Interation (ci=-1)
Refreshing local packages (refresh existing: false)...
Looking for local package map at /var/lib/dub/packages/local-packages.json
Looking for local package map at /home/bittwiddler/.dub/packages/local-packages.json
iterating dir /home/bittwiddler/.dub/packages/
iterating dir /home/bittwiddler/.dub/packages/ entry libdparse-0.2.0
iterating dir /home/bittwiddler/.dub/packages/ entry inifiled-0.0.3
Collecting dependencies for dub
Missing dependency vibe-d 0.7.23 of dub
Checking for upgrades.
Using cached upgrade results...
Add config dub application
Add config dub library
Add config dub library-nonet
NON-PRIMARY:
Eliminating config library for dub
NON-PRIMARY:
Eliminating config library-nonet for dub
Using configuration 'application' for dub
Generating using build
Creating build generator.
Add config dub application
Add config dub library
Add config dub library-nonet
NON-PRIMARY:
Eliminating config library for dub
NON-PRIMARY:
Eliminating config library-nonet for dub
Using configuration 'application' for dub
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/app.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/commandline.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/compilers/buildsettings.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/compilers/compiler.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/compilers/dmd.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/compilers/gdc.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/compilers/ldc.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/dependency.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/dependencyresolver.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/dub.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/generators/build.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/generators/cmake.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/generators/generator.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/generators/sublimetext.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/generators/visuald.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/init.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/libInputVisitor.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/sdlang/ast.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/sdlang/exception.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/sdlang/lexer.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/sdlang/package.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/sdlang/parser.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/sdlang/symbol.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/sdlang/token.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/sdlang/util.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/utils.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/vibecompat/core/file.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/vibecompat/core/log.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/vibecompat/data/json.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/vibecompat/data/utils.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/vibecompat/inet/path.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/vibecompat/inet/url.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/package_.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/packagemanager.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/packagesupplier.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/platform.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/project.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/recipe/json.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/recipe/packagerecipe.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/recipe/sdl.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/semver.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/version_.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ bin
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/app.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/app.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/commandline.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/compilers/buildsettings.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/compilers/compiler.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/compilers/dmd.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/compilers/gdc.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/compilers/ldc.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/dependency.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/dependencyresolver.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/dub.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/generators/build.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/generators/cmake.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/generators/generator.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/generators/sublimetext.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/generators/visuald.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/init.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/libInputVisitor.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/sdlang/ast.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/sdlang/exception.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/sdlang/lexer.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/sdlang/package.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/sdlang/parser.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/sdlang/symbol.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/sdlang/token.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/sdlang/util.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/utils.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/vibecompat/core/file.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/vibecompat/core/log.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/vibecompat/data/json.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/vibecompat/data/utils.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/vibecompat/inet/path.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/internal/vibecompat/inet/url.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/package_.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/packagemanager.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/packagesupplier.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/platform.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/project.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/recipe/json.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/recipe/packagerecipe.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/recipe/sdl.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/semver.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/dub/version_.d
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ bin
Fixing relative path: /home/bittwiddler/Downloads/source/git/dlang/dub/ ~ source/app.d
Generate target dub (executable /home/bittwiddler/Downloads/source/git/dlang/dub/bin dub)
dub: ["dub"]
Error executing command run:
Range violation
Full exception: core.exception.RangeError@source/dub/internal/vibecompat/inet/path.d(169): Range violation
----------------
$ dmd --version
DMD64 D Compiler v2.067.1
Copyright (c) 1999-2014 by Digital Mars written by Walter Bright

$ gdc --version 
gdc (Gentoo 4.8.4 p1.6, pie-0.6.1) 4.8.4
Copyright (C) 2013 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

the code in question is in the relativeTo function

  Path ret = Path(null, false);
                ret.m_endsWithSlash = true;
                foreach( i; 0 .. nup ) ret ~= "..";
                /* -> */ ret ~= Path(m_nodes[parentPath.length-nup .. $], false);
                ret.m_endsWithSlash = this.m_endsWithSlash;
                return ret;
        }

I'm still learning D, so I'm not quite sure how to fix it. My GDB debugging skills are rusty and I don't know how to make it catch Dlang exceptions.

Kelet commented 9 years ago

Having the same problem with a dub compiled by GDC v2.066.1_gcc4.9

Kelet commented 9 years ago

m_nodes.length is 0 for me in this code, which is causing the problem

Geod24 commented 9 years ago

Ping @ibuclaw

ibuclaw commented 9 years ago

I'll happily check a reduced test. :-)

Kelet commented 9 years ago

Here, in the relativeTo function of a Path struct, a new path is constructed like so:

Path ret = Path(null, false);

Looking at the constructor:

this(immutable(PathEntry)[] nodes, bool absolute)
{
    m_nodes = nodes;
    m_absolute = absolute;
}

So null is being passed in for immutable(PathEntry)[] nodes. After the constructor is executed for ret variable. Suddenly, m_nodes in the original struct instance loses its value and becomes empty.

So to summarize:

I'll investigate this more when I have time, and of course I might be making a mistake with my analysis. I think I'm missing something, because obviously this doesn't make sense.

Kelet commented 9 years ago

If anyone else experiences this problem, you can temporarily fix it by removing these lines and recompiling dub. Seems like the problematic code only functions to shorten the command line, so it seems fine to remove (unless your dub compilation command line gets too huge for your shell, I guess).

s-daveb commented 9 years ago

I can confirm this patch makes the issue go away. I'm glad it was reproducible and was able to reach someone who could fix it! :+1: Out of curiosity, what are some examples of shells that will barf with extremely long commandlines?

grogancolin commented 9 years ago

I know windows Cmd.exe is limited to 2k characters. Maybe thats different on new windows, not sure. It's easily attainable to hit 2k with a command with many paths anyway, so we should prob.try to fix this bug rather than get around it.

s-ludwig commented 9 years ago

I haven't been able to reproduce this on Ubuntu 14.04 x86-64 with either the latest DMD or GDC (x64). Is there anything else that may influence this?

s-ludwig commented 9 years ago

I really need to be able to reproduce this, so if anyone who can has more information about the environment that would be great!

s-daveb commented 9 years ago

Don't know what else I could tell you about the environment besides version numbers.

It's happened to me on two different installs on two different processors. (AMD FX 9590 / Intel i5 4690K)

The only thing that fixed it was compiling dub with ldc2.

$ dmd --version
DMD64 D Compiler v2.067.1
Copyright (c) 1999-2014 by Digital Mars written by Walter Bright

$ gdc --version 
gdc (Gentoo 4.8.4 p1.6, pie-0.6.1) 4.8.4
Copyright (C) 2013 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ emerge --info dev-libs/dub 
dev-util/dub-0.9.23::dlang was built with the following:
USE="gdc-4_8_4 -debug -dmd-2_064 -dmd-2_065 -dmd-2_066 -dmd-2_067 -gdc-4_8_3 -ldc2-0_13 -ldc2-0_14 -ldc2-0_15" ABI_X86="64"

$ emerge --info dev-libs/glibc 
sys-libs/glibc-2.20-r2::gentoo was built with the following:
USE="debug (multilib) -gd (-hardened) -nscd -profile (-selinux) -suid -systemtap -vanilla" ABI_X86="64"
CFLAGS="-march=core-avx2 -pipe -O2 -fno-strict-aliasing -fno-stack-protector"
CXXFLAGS="-march=core-avx2 -pipe -O2 -fno-strict-aliasing -fno-stack-protector"

$ emerge --info gcc 
sys-devel/gcc-4.8.4::dlang was built with the following:
USE="cxx d (multilib) nls nptl openmp sanitize (-altivec) (-awt) -debug -doc (-fixed-point) -fortran -gcj -go -graphite (-hardened) (-libssp) -mudflap (-multislot) -nopie -nossp -objc -objc++ -objc-gc -regression-test -vanilla" ABI_X86="64"

If Ubuntu is using a different/patched version o glibc/gcc it could explain why you don't see the issue.

@Kelet, can you add anything to this?

s-daveb commented 9 years ago

Here's a backtrace.

(gdb) bt
#0  dub.internal.vibecompat.inet.path.Path.relativeTo() (this=..., 
    parentPath=...) at source/dub/internal/vibecompat/inet/path.d:169
#1  0x0000000000462925 in makeRelative (this=0x7fffffffa340, path=...)
    at source/dub/generators/build.d:99
#2  0x0000000000462499 in dub.generators.build.BuildGenerator.buildTarget() (
    this=..., settings=..., buildsettings=..., pack=..., config=..., 
    packages=..., additional_dep_files=...)
    at source/dub/generators/build.d:100
#3  0x0000000000462011 in buildTargetRec (this=0x7fffffffb230, target=...)
    at source/dub/generators/build.d:69
#4  0x00000000004618d7 in dub.generators.build.BuildGenerator.generateTargets()
    (this=..., settings=..., targets=...) at source/dub/generators/build.d:78
#5  0x000000000046b839 in dub.generators.generator.ProjectGenerator.generate()
    (this=..., settings=...) at source/dub/generators/generator.d:105
#6  0x000000000042204d in dub.dub.Dub.testProject() (this=..., settings=..., 
    config=..., custom_main_file=...) at source/dub/dub.d:407
#7  0x000000000040cbb0 in dub.commandline.TestCommand.execute() (this=..., 
    dub=..., free_args=..., app_args=...) at source/dub/commandline.d:743
#8  0x0000000000407ee6 in dub.commandline.runDubCommandLine() (args=...)
    at source/dub/commandline.d:211
#9  0x0000000000406160 in D main (args=...) at source/app.d:15
#10 0x00000000005f0baf in rt.dmain2._d_run_main() ()
#11 0x00000000005f0cdf in rt.dmain2._d_run_main() ()
#12 0x00000000005f10a9 in rt.dmain2._d_run_main() ()
#13 0x00000000005f0cdf in rt.dmain2._d_run_main() ()
#14 0x00000000005f0e70 in _d_run_main ()
#15 0x0000000000406132 in main (argc=2, argv=0x7fffffffdbb8)
    at /usr/lib64/gcc/x86_64-pc-linux-gnu/4.8.4/include/d/__entrypoint.di:62
(gdb) c
Continuing.
Error executing command test:
Range violation
[Inferior 1 (process 24449) exited with code 02]
s-daveb commented 9 years ago

From the dlang documentation...

The second way is to cast data to immutable. When doing so, it is up to the programmer to ensure that no other mutable references to the same data exist.

char[] s = ...;
immutable(char)[] p = cast(immutable)s;     // undefined behavior
immutable(char)[] p = cast(immutable)s.dup; // ok, unique reference

The .idup property is a convenient way to create an immutable copy of an array:

auto p = s.idup; p[0] = ...; // error, p[] is immutable

I'm noticing a few m_nodes = cast(immutable)splitPath(pathstr); instead of (splitPath).idup in the code... Just a shot in the dark, but it could have something to do with this problem?

Kelet commented 9 years ago

I'm on

I have not tried w/DMD, only GDC.

s-daveb commented 9 years ago

@Kelet, let us know if this fixes the problem for you.