installation / usage issues

[armandosano]

Thanks for sharing SciD, it looks like exactly what I would need! Unfortunately, I have been unable to compile/link a simple test example. I am not sure I got it right from installation to compilation/linking, though. I didn't find detailed instructions for those. Perhaps I did something wrong - this is what I did:

1. upgraded to dmd v2.066.1
2. built and installed https://github.com/9il/cblas, i.e. 2.1 dub build => libcblas.a 2.2. mv libcblas.a /usr/local/lib/ (where libphobos2.a lives for me)
3. built and installed scid, i.e. 3.1 unzip, then cd source/scid/bindings/blas 3.2 symlink to cblas.d: ln -s path/to/cblas-master/source/cblas/cblas.d 3.3 rdmd build -O [switch -inline made compilation fail with errors] => generated/libscid.a => generated/headers/... (including source/scid/bindings/blas/cblas.di, so I am guessing cblas worked) 3.4 mv generated/headers/source/scid /usr/local/include/dmd2/ (where phobos & druntime live) mv generated/libscid.a /usr/local/lib/ (where libphobos2.a lives) 3.5 added flags "-I/usr/local/include/dmd2" and "-L/usr/local/lib" to DFLAGS in dmd.conf so that dmd finds /usr/local/include/dmd2/scid and the libraries.
4. In a source file main.d, used: import scid.linalg; import scid.matrix; ... auto m = matrix!double(2, 2); // filling it m[0,0]=1.0; etc. auto d = det(m);
5. compiled with: dmd -L-lscid -L-llapack main.d

But while dmd (or rather ld) clearly finds most symbols in libscid.a and liblapack it fails to find the following one: Undefined symbols for architecture x86_64: "_D4scid6linalg35T4det_TdVE4scid6matrix7Storagei0Z4det_FS4scid6matrix68T10MatrixViewTdVE4scid6matrix7Storagei0VE4scid6matrix8Trianglei85Z10MatrixViewZ3mulMFdZv", referenced from: _D4scid6linalg35T4det_TdVE4scid6matrix7Storagei0Z4det_FS4scid6matrix68T10MatrixViewTdVE4scid6matrix7Storagei0VE4scid6matrix8Trianglei85Z10MatrixViewZd in main.o

In case that matters, I am on Mac OS X 10.9.5. Many thanks for your help!

[9il]

Hi!

1. cblas is a header (!). You need it if and only if you want to call BLAS functions. BLAS is not required by scid anymore.
2. You may want to use DUB (it's works on my OS X)
   1. Install DUB registry
   2. Read about DUB
   3. Read about DUB package format
   4. To use scid, put the following dependency into your project's dub.json into the dependencies section:

{
    ...
    "dependencies": {
        "scid": "~>0.2.0"
    }
}

I hope DUB can help you forget about linking.

[9il]

_D4scid6linalg35T4det_TdVE4scid6matrix7Storagei0Z4det_FS4scid6matrix68T10MatrixViewTdVE4scid6matrix7Storagei0VE4scid6matrix8Trianglei85Z10MatrixViewZ3mulMFdZv

is inner function in det_

[armandosano]

Thanks for your replies 9il!

Indeed, just typing 'dub build' works (yay!). But... it redownloads and reinstalls the whole scid package master folder under ~/.dub/packages/scid-0.2.0/ (without even telling very explicitly). I had tried 'dub build' and it had failed yesterday, presumably because it was re-installing scid and it would have worked had I tried a second time?

Anyway, at least I have now a workable version. Still, I would like to understand and have more control over when/where packages are installed. Do you have any idea about the following:

1. is it possible to tell dub to use the scid I have installed manually instead? If so, how? Presumably I could install using --local but that still doesn't tell me how dub knows in subsequent runs where the package has been installed. (presumably something called local package map, or systemDependencies in dub.json, but these seem undocumented).
2. the scid-master folder is copied under ~/.dub including all the source files (.d not .di). The library libscid.a is automatically compiled and found in the root master folder as well. Why are all the source files kept instead of the more conventional 'header' (.di) + compiled lib (.a) that one can put into a centralised location (e.g. for use by multiple users)?
3. I must say I was confused with cblas indeed. I did compile it with 'dub build' and it produced a libcblas.a and no header file, which I assumed meant there was more than just a wrapper header (why is dub doing this then?). So how would you recommend installing cblas.d if I wanted to call blas directly? Just copy the source file in e.g. /usr/local/include/dmd2/?
4. of course, dub makes appropriate calls to dmd to compile the executable, and dmd calls ld to link with the libraries. So there is a way to call dmd that makes my program compile... It doesn't seem possible to know how dub calls the compiler, using -v (verbose) doesn't tell how.

dub looks like a great idea, but all these things happening in the background are not transparent. For example, it seems that upon each build, dub looks whether the dependencies have been upgraded (by default at least) and it would download and install upgrades automatically. This is in many cases not wanted when working on long projects.

[armandosano]

ok, about the last point, one can specify a package by 'path' in dub.json, instead of a 'version' field: this will prevent any spurious downlads. It will compile the library within the master (root) folder and will not duplicate it in ~/.dub. That folder could be put into a centralised location (e.g. /usr/local/lib or /usr/local/include - well it contains both the library and header files...).

[9il]

1_

$ dub --help
 ...
  Package management
  ------------------
  fetch <name>          Manually retrieves and caches a package
  remove <name>         Removes a cached package
  upgrade [<package>]   Forces an upgrade of all dependencies
  add-path <path>       Adds a default package search path
  remove-path <path>    Removes a package search path
  add-local <path> [<version>]
                        Adds a local package directory (e.g. a git repository)
  remove-local <path>   Removes a local package directory
  list                  Prints a list of all local packages dub is aware of
  add-override <package> <version-spec> <target-path/target-version>
                        Adds a new package override.
  remove-override <package> <version-spec>
                        Removes an existing package override.
  list-overrides        Prints a list of all local package overrides

2 .di extension isn't a part of D language. scid is generic library. Size of .di files is the same. 3 cblas is a header itself. If you want have it like a header just include it into you project using dub configuration or copy-past it. 4_ Dub is very flexible. You may want to read it's spec again.

{
    "dependencies": {
        "simple_matrix": "==0.3.1+g42a2f29",  // <------- 42a2f29 - fixed local commit
    }
}

See also: http://semver.org.

[armandosano]

Thanks 9il, didn't see this part of dub help - it would be nice if they would say something more about this in the online doc though... the one-liners in dub help are suggestive, but minimalistic :) (maybe I am missing sth)

.di are part of D: D interface files ('header'), see e.g. man dmd. Sure, when using templates, you can't pre-compile, this is why in this case the whole code comes into .di and there is no difference between .d and .di. If you build scid with 'rdmd build' etc following the instructions in build.d, it does generate libscid.a and header files .di. I suspect some functions are not templated and so can be pre-compiled into libscid.a.

It's still mysterious how dub calls dmd. I can see (spying with ps) dub build calling dmd like: dmd -of.dub/.../app .dub/.../app.o path/to/scid-master/libscid.a -L-llapack -L-lblas -g and then calls to /Applications/Xcode.app/.../clang -m64 ...-llapack -lblas -Lpath/to/scid-master/libscid.a -lphobos2 -lpthread -lm -L/usr/lib -L/usr/local/lib ... but doing the dmd call explicitly as above fails with the same errors as my original post. I don't see the whole picture though, it's just picking up system calls at specific times by chance... e.g. I don't see how dmd is invoked to produce the .o file (dmd -c presumably, but then the next call fails)

[armandosano]

oops dub build --vverbose does show how dmd is called (not --verbose), sorry :)

I managed to compile app.d using dmd directly with:

dmd -L-llapack -L-lscid -Ipath/to/scid-master/source app.d

but not with dmd -L-llapack -L-lscid -I/usr/local/include/dmd2/scid app.d (=>"Error: undefined identifier GC.BlkAttr")

where /usr/local/include/dmd/scid/ contained the di files generated by compiling the scid library following the instructions in the file build.d in scid-master, as in my original post.

However, the switch -L-lscid uses my scid library in /usr/local/lib/libscid.a (compiled along with the .di files), it fails without this switch.

Bottom line: in point 3.4 of my original post: do instead: cp -R path/to/scid-master/source/scid /usr/local/include/dmd2/

and to compile with dmd, use (after modifying dmd.conf adequately): dmd -L-lscid -L-llapack app.d

Thanks again for your help 9il, and for the explanations concerning dub!

[kyllingstad]

Just as a tip, if you don't want to use Dub, compiling SciD manually is really easy. From the source directory, simply pass all source files to DMD:

dmd -release -inline -O -lib -oflibscid.a `find source | grep '\.d$'`

Then, copy libscid.a to your library directory and source/scid to your import path. Compiling any application that uses SciD should then be as simple as:

dmd -I/path/to/imports -L-lscid -L-llapack -L-lblas app.d

I am planning to remove build.d for the simple reason that there are currently three build systems for SciD, which is at least one too many. It is simply too unpractical to keep them all up to date. Almost everyone seems to use Dub, and the makefile is used for D-APT, so build.d must go.