Autolinking

[galchinsky]

• No explicit clay -lsmth needed to make module work:

clay/test/lang/modules/imports/library$ cat main.clay
import "curl";
external curl_easy_init(): OpaquePointer;
main() {
    curl_easy_init();
}
clay/test/lang/modules/imports/library$ clay main.clay
clay/test/lang/modules/imports/library$ 

Found the idea in Doug Gregor's presentation on slide 67

• Second commit fixes a bug on my system after upgrade: llvm-jit-linker needs smth.so instead of smth without an extension. I tried to revert to the first patch where linkLibraries appeared, but it has the same error. It didn't have, I don't know why. Please test something like clay -lsmth foo.clay, if it is only my system's trouble or not.

[jckarter]

This is cool. You could also potentially support importing Clang modules directly and automatically generate the necessary external bindings so that clay-bindgen doesn't need to be a separate tool. Instead of creating a new "import" form, what do you think about making it a module attribute, so you'd say in my.module (LinkLibrary["smth"]); ?

[galchinsky]

I forgot about module attributes. LinkLibrary["smth"] doesn't work because the analyzer wants to evaluate it and can't find LinkLibrary symbol. I made an addition to verifyAttibutes so that in my.module (Foo : "smth", Bar : "+zmth", Foo : "blah" ++ "blah", Bar : "-zmth"); will be added to a "multimap"named module::attrParameters (Foo = [smth, blahblah], Bar = [+zmth, -zmth]). I think that it is quite universal. LinkLibrary entry is passed to linker as it was by `in my.module (LinkLibrary :"smth");

As I can see in clang/test/Modules it is not well implemented yet to support it. Compiler could automatically run bindgen when import .h files. Something like import libc.smth in "smth.h". What do you think?

[jckarter]

You would have to add LinkLibrary as a primitive symbol, similar to RecordWithProperties. But now that you mention it, using keyword pairs makes perfect sense and is a lot cleaner, so I think that's definitely the way to go. I think that it'd also be an improvement to use keywords for external function attributes, since external (asmName:"_foo$BAR") foo() is more descriptive than the current external ("_foo$BAR") foo().

It'd definitely be cool to build bindgen into the compiler as a stopgap until Clang modules mature and start to get used by libraries. I'm not sure what the best design is though, since I don't think you want to duplicate a bunch of common C definitions from common headers into different Clay modules. I had in mind a design in which importing C headers dumped symbols into a __c__ module, and you'd then public import the symbols you wanted to expose through your API module, for instance:

external import "<stdio.h>";
public import __c__.(FILE, fopen, fclose, fread, fwrite);

[galchinsky]

What do you mean by "build into"? Bindgen is written in Clay, so it needs the compiler to work. I could call the bindgen through a system(...) in loadDependents. Dynamically linked shared lib is an alternative, but it will be platform dependent. About the syntax, I think external import "<stdio.h>" as libc.stdio; is more concise than magic __c__ name.

external import "<stdio.h>" as libc.stdio;
extrenal import "<stdlib.h>" as stdlib;
public import  libc.stdio.(FILE, fopen, fclose, fread, fwrite);
public import stdlib.*;

[jckarter]

The bindgen code would have to be rewritten in C++ to embed it in the compiler. Even if the external import form can import symbols into a named module, the symbols have to actually live in a __c__ module or equivalent, because headers need to be able to reference each other's definitions, and you don't want different distinct definitions of things because a header is shared in common with two other headers.

[galchinsky]

What is wrong?

$ cat hello.c
int x;
$ cat hello.h
extern int x;
$ cat another.clay
external x : CInt;
$ cat main.clay
import printer.(println);
import another;

external x : CInt;

main(){
   x = 100;
   println(x);
   println(another.x);
}
$ gcc -c hello.c -o hello.o && ar rcs libhello.a hello.o
$ clay main.clay -lhello -L.
/tmp/clayobj-073e64ab.obj: In function `main':
main.clay:(.text+0x16c): undefined reference to `x1'
clang: error: linker command failed with exit code 1 (use -v to see invocation)

[jckarter]

Looks like a bug. It's probably trying to create separate LLVM globals for both external x declarations, and the second global's name gets uniqued to x1. It needs to reuse the existing LLVM global declaration like it does with external functions.

On Sat, Dec 1, 2012 at 9:41 AM, Dmitry Galchinsky notifications@github.comwrote:

What is wrong?

$ cat hello.c int x; $ cat hello.h extern int x; $ cat another.clay external x : CInt; $ cat main.clay import printer.(println); import another;

external x : CInt;

main(){ x = 100; println(x); println(another.x); } $ gcc -c hello.c -o hello.o && ar rcs libhello.a hello.o $ clay main.clay -lhello -L. /tmp/clayobj-073e64ab.obj: In function main': main.clay:(.text+0x16c): undefined reference tox1' clang: error: linker command failed with exit code 1 (use -v to see invocation)

— Reply to this email directly or view it on GitHubhttps://github.com/jckarter/clay/pull/450#issuecomment-10919678.

[galchinsky]

Fixed the bug with twice-meeted-externals.

I think adding "automatically generate the necessary external bindings" is finished, but it calls clay-bindgen through a pipe. Calling to internal version can be done in makeModuleFromCHeaders in loader.

External ImportMembers and importStar are supported, but importStar will obviously pollute namespace with everything imported:

external "smth.h" import smth;
external "smth2.h" import smth2.*;
//in fact is
import __c__;
import __c__.*;

But anyway I think that things like external "smth.h" import smth(foo, bar); are useful and star import for externals should not be recommended. If anyone wants to use it, he should create new module and public import from it. It should be nicer with clang modules.

Didn't test piping in windows, but MSVS CRT has _popen and _pclose, I don't think there would be a problem.

Also added LibSearchPath module attribute so one can write in main (LinkLibrary : "hello", LibSearchPath : "."); which is passed to clang as -L.

[jckarter]

I'm not a fan of shelling out to bindgen like that. Your other patches look good though.

[galchinsky]

Me too. It is a scaffold to test the idea and replace with internal bindgen someday, so I tried to localize it. As I said the other "fast" approach is to load bindgen.so with dlopen/loadlibrary, but I don't think that it is much better to move text. Internal bindgen could create AST straightforward (or why else is it needed?) but it'll take a time to rewrite it to C++ and replace println with AST construction.

[galchinsky]

I'll close till have enough time to finish