Error compiling

[olafura]

make[1]: Entering directory '/home/olafura/dev/expostal'
cc -g -O3 -ansi -pedantic -Wall -Wextra -Wno-unused-parameter -I/usr/local/lib/erlang/erts-9.0/include -I/usr/local/include -I/usr/include -L/usr/local/lib -L/usr/lib -lpostal -std=gnu99 -Wno-unused-function -fPIC -shared  -o priv/parser.so src/parser.c
src/parser.c: In function ‘parse_address’:
src/parser.c:30:13: warning: pointer targets in passing argument 1 of ‘strncpy’ differ in signedness [-Wpointer-sign]
     strncpy(component_term_bin, component, strlen(component));
             ^~~~~~~~~~~~~~~~~~
In file included from /usr/include/features.h:364:0,
                 from /usr/include/stdio.h:27,
                 from /usr/local/include/libpostal/libpostal.h:8,
                 from src/parser.c:1:
/usr/include/x86_64-linux-gnu/bits/string3.h:123:1: note: expected ‘char * restrict’ but argument is of type ‘unsigned char *’
 __NTH (strncpy (char *__restrict __dest, const char *__restrict __src,
 ^
src/parser.c: At top level:
src/parser.c:45:5: warning: missing initializer for field ‘flags’ of ‘ErlNifFunc {aka struct enif_func_t}’ [-Wmissing-field-initializers]
     {"parse_address", 1, parse_address}};
     ^
In file included from src/parser.c:2:0:
/usr/local/lib/erlang/erts-9.0/include/erl_nif.h:104:14: note: ‘flags’ declared here
     unsigned flags;
              ^~~~~
In file included from src/parser.c:6:0:
src/helper.c:16:12: warning: ‘is_libpostal_language_classifier_setup’ defined but not used [-Wunused-variable]
 static int is_libpostal_language_classifier_setup = 0;
            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
make[1]: Leaving directory '/home/olafura/dev/expostal'
make[1]: Entering directory '/home/olafura/dev/expostal'
cc -g -O3 -ansi -pedantic -Wall -Wextra -Wno-unused-parameter -I/usr/local/lib/erlang/erts-9.0/include -I/usr/local/include -I/usr/include -L/usr/local/lib -L/usr/lib -lpostal -std=gnu99 -Wno-unused-function -fPIC -shared  -o priv/expand.so src/expand.c
src/expand.c: In function ‘expand_address’:
src/expand.c:23:17: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (i = 0; i < num_expansions; i++)
                 ^
src/expand.c:28:13: warning: pointer targets in passing argument 1 of ‘strncpy’ differ in signedness [-Wpointer-sign]
     strncpy(expansion_term_bin, expansion, strlen(expansion));
             ^~~~~~~~~~~~~~~~~~
In file included from /usr/include/features.h:364:0,
                 from /usr/include/stdio.h:27,
                 from /usr/local/include/libpostal/libpostal.h:8,
                 from src/expand.c:1:
/usr/include/x86_64-linux-gnu/bits/string3.h:123:1: note: expected ‘char * restrict’ but argument is of type ‘unsigned char *’
 __NTH (strncpy (char *__restrict __dest, const char *__restrict __src,
 ^
src/expand.c: At top level:
src/expand.c:40:3: warning: missing initializer for field ‘flags’ of ‘ErlNifFunc {aka struct enif_func_t}’ [-Wmissing-field-initializers]
   { "expand_address", 1, expand_address }
   ^
In file included from src/expand.c:2:0:
/usr/local/lib/erlang/erts-9.0/include/erl_nif.h:104:14: note: ‘flags’ declared here
     unsigned flags;
              ^~~~~
make[1]: Leaving directory '/home/olafura/dev/expostal'
Compiling 2 files (.ex)
Generated expostal app

17:30:05.198 [error] Process #PID<0.151.0> raised an exception
** (MatchError) no match of right hand side value: {:error, {:load_failed, 'Failed to load NIF library: \'/home/olafura/dev/expostal/_build/dev/lib/expostal/priv/parser.so: undefined symbol: libpostal_teardown_parser\''}}
    lib/parser.ex:12: Expostal.Parser.init/0
    (kernel) code_server.erl:1333: anonymous fn/1 in :code_server.handle_on_load/5

17:30:05.199 [error] Process #PID<0.152.0> raised an exception
** (MatchError) no match of right hand side value: {:error, {:load_failed, 'Failed to load NIF library: \'/home/olafura/dev/expostal/_build/dev/lib/expostal/priv/expand.so: undefined symbol: libpostal_expand_address\''}}
    lib/expand.ex:12: Expostal.Expand.init/0
    (kernel) code_server.erl:1333: anonymous fn/1 in :code_server.handle_on_load/5

17:30:05.200 [warn]  The on_load function for module Elixir.Expostal.Parser returned:
{{:badmatch,
  {:error,
   {:load_failed,
    'Failed to load NIF library: \'/home/olafura/dev/expostal/_build/dev/lib/expostal/priv/parser.so: undefined symbol: libpostal_teardown_parser\''}}},
 [{Expostal.Parser, :init, 0, [file: 'lib/parser.ex', line: 12]},
  {:code_server, :"-handle_on_load/5-fun-0-", 1,
   [file: 'code_server.erl', line: 1333]}]}

17:30:05.201 [warn]  The on_load function for module Elixir.Expostal.Expand returned:
{{:badmatch,
  {:error,
   {:load_failed,
    'Failed to load NIF library: \'/home/olafura/dev/expostal/_build/dev/lib/expostal/priv/expand.so: undefined symbol: libpostal_expand_address\''}}},
 [{Expostal.Expand, :init, 0, [file: 'lib/expand.ex', line: 12]},
  {:code_server, :"-handle_on_load/5-fun-0-", 1,
   [file: 'code_server.erl', line: 1333]}]}

[olafura]

I'm using OTP master and Elixir master

[olafura]

Maybe this might be a good idea possible to get the libpostal dep: https://github.com/ninenines/erlang.mk/blob/master/doc/src/guide/deps.asciidoc

[xiamx]

which version of libpostal and c compiler are you using?

[xiamx]

Regarding including libpostal as dep, I raised similar suggestion with the official Node.JS binding https://github.com/openvenues/node-postal/issues/10 . @thatdatabaseguy responded with the reason for not including libpostal as a direct dependency. I feel like the situation is the same with Elixir binding. Though this may change cuz I'm not very familiar with Elixir 😄

[olafura]

I'm using libpostal: 0cb8c61fb0619a47ab19ab97e847bea8af3a7e44 gcc: 6.3.0 20170406

libportal seems to be hardcoded to gcc so I don't need to post the clang version ;)

[xiamx]

Cool I'm going to try reproducing it. In TravisCI, I never got it to compile with gcc, so I opted to use clang instead.

[xiamx]

Wasn't able to reproduce it with libpostal on 0cb8c61fb.. though I'm not sure how to get master branch of Elixir and OTP on TravisCI

[xiamx]

https://travis-ci.org/SweetIQ/expostal/builds/242498961

[jdemaris]

I hit the same issue with my build process. Using expostal 0.1.0 from Hex, Elixir 1.4.4 and OTP19.

It seems to me that the issue is coming from the use of :code.priv_dir/1 (e.g. https://github.com/SweetIQ/expostal/blob/master/lib/parser.ex#L11), which is returning a path for the :expostal app as app/_build/dev/lib/expostal/priv for the parser.so. That .so is actually sitting in deps/expostal/priv and _build/dev/lib/expostal/priv is symlinked to that folder at some point during the build process - but it seems like not early enough.

If I do a clean start (erase _build and deps folders) and then run mix deps.get followed by mix deps.compile, I consistently see the issue. After it runs, I can look at _build/dev/lib/expostal and there is no symlink there named priv.

However, if I run mix deps.compile again immediately after, it actually works! And looking into _build/dev/lib/expostal at that point shows that the priv symlink has been created.

[jdemaris]

From my first mix deps.compile:

==> expostal
Compiling 2 files (.ex)

== Compilation error on file lib/parser.ex ==
** (MatchError) no match of right hand side value: {:error, :on_load_failure}
    (stdlib) erl_eval.erl:670: :erl_eval.do_apply/6

could not compile dependency :expostal, "mix compile" failed. You can recompile this dependency with "mix deps.compile expostal", update it with "mix deps.update expostal" or clean it with "mix deps.clean expostal"

22:03:11.291 [error] Process #PID<0.3949.0> raised an exception
** (MatchError) no match of right hand side value: {:error, {:load_failed, 'Failed to load NIF library: \'/app/_build/dev/lib/expostal/priv/parser.so: cannot open shared object file: No such file or directory\''}}
    lib/parser.ex:12: Expostal.Parser.init/0
    (kernel) code_server.erl:1347: anonymous fn/1 in :code_server.handle_on_load/5

22:03:11.330 [warn]  The on_load function for module Elixir.Expostal.Parser returned {{:badmatch,
  {:error,
   {:load_failed,
    'Failed to load NIF library: \'/app/_build/dev/lib/expostal/priv/parser.so: cannot open shared object file: No such file or directory\''}}},
 [{Expostal.Parser, :init, 0, [file: 'lib/parser.ex', line: 12]},
  {:code_server, :"-handle_on_load/5-fun-0-", 1,
   [file: 'code_server.erl', line: 1347]}]}

22:03:11.330 [error] Process #PID<0.3951.0> raised an exception
** (MatchError) no match of right hand side value: {:error, {:load_failed, 'Failed to load NIF library: \'/app/_build/dev/lib/expostal/priv/expand.so: cannot open shared object file: No such file or directory\''}}
    lib/expand.ex:12: Expostal.Expand.init/0
    (kernel) code_server.erl:1347: anonymous fn/1 in :code_server.handle_on_load/5

22:03:11.334 [warn]  The on_load function for module Elixir.Expostal.Expand returned {{:badmatch,
  {:error,
   {:load_failed,
    'Failed to load NIF library: \'/app/_build/dev/lib/expostal/priv/expand.so: cannot open shared object file: No such file or directory\''}}},
 [{Expostal.Expand, :init, 0, [file: 'lib/expand.ex', line: 12]},
  {:code_server, :"-handle_on_load/5-fun-0-", 1,
   [file: 'code_server.erl', line: 1347]}]}

from my second mix deps.compile:

==> expostal
Compiling 2 files (.ex)
Compiling lib/parser.ex (it's taking more than 10s)
Generated expostal app

[jdemaris]

I did get it to work by using mix deps.compile --include-children first

[jdemaris]

--include-children only worked sporadically. I am now pinned to expostal 0.1.3 and this is the compilation output:

+ mix deps.compile expostal --include-children
0s
2
cc -g -O3 -ansi -pedantic -Wall -Wextra -Wno-unused-parameter -I/usr/local/lib/erlang/erts-8.3.1/include -I/usr/local/include -I/usr/include -L/usr/local/lib -L/usr/lib -lpostal -std=gnu99 -Wno-unused-function -fPIC -shared  -o priv/parser.so src/parser.c
2s
3
In file included from /usr/include/string.h:635:0,
2s
4
                 from src/parser.c:4:
2s
5
src/parser.c: In function ‘parse_address’:
2s
6
src/parser.c:29:13: warning: pointer targets in passing argument 1 of ‘__builtin_strncpy’ differ in signedness [-Wpointer-sign]
2s
7
     strncpy(component_term_bin, component, strlen(component));
2s
8
             ^
2s
9
src/parser.c:29:13: note: expected ‘char *’ but argument is of type ‘unsigned char *’
2s
10
src/parser.c: At top level:
2s
11
src/parser.c:44:5: warning: missing initializer for field ‘flags’ of ‘ErlNifFunc’ [-Wmissing-field-initializers]
2s
12
     {"parse_address", 1, parse_address}};
2s
13
     ^
2s
14
In file included from src/parser.c:2:0:
2s
15
/usr/local/lib/erlang/erts-8.3.1/include/erl_nif.h:104:14: note: ‘flags’ declared here
2s
16
     unsigned flags;
2s
17
              ^
2s
18
cc -g -O3 -ansi -pedantic -Wall -Wextra -Wno-unused-parameter -I/usr/local/lib/erlang/erts-8.3.1/include -I/usr/local/include -I/usr/include -L/usr/local/lib -L/usr/lib -lpostal -std=gnu99 -Wno-unused-function -fPIC -shared  -o priv/expand.so src/expand.c
4s
19
src/expand.c: In function ‘expand_address’:
4s
20
src/expand.c:22:17: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
4s
21
   for (i = 0; i < num_expansions; i++)
4s
22
                 ^
4s
23
In file included from /usr/include/string.h:635:0,
4s
24
                 from src/expand.c:4:
4s
25
src/expand.c:27:13: warning: pointer targets in passing argument 1 of ‘__builtin_strncpy’ differ in signedness [-Wpointer-sign]
4s
26
     strncpy(expansion_term_bin, expansion, strlen(expansion));
4s
27
             ^
4s
28
src/expand.c:27:13: note: expected ‘char *’ but argument is of type ‘unsigned char *’
4s
29
src/expand.c: At top level:
4s
30
src/expand.c:39:3: warning: missing initializer for field ‘flags’ of ‘ErlNifFunc’ [-Wmissing-field-initializers]
4s
31
   { "expand_address", 1, expand_address }
4s
32
   ^
4s
33
In file included from src/expand.c:2:0:
4s
34
/usr/local/lib/erlang/erts-8.3.1/include/erl_nif.h:104:14: note: ‘flags’ declared here
4s
35
     unsigned flags;
4s
36
              ^
4s
37
==> expostal
4s
38
Compiling 2 files (.ex)
4s
40
== Compilation error on file lib/parser.ex ==
4s
41
** (MatchError) no match of right hand side value: {:error, :on_load_failure}
4s
42
    (stdlib) erl_eval.erl:670: :erl_eval.do_apply/6
4s
45
14:40:46.793 [error] Process #PID<0.180.0> raised an exception
4s
46
** (MatchError) no match of right hand side value: {:error, {:load_failed, 'Failed to load NIF library: \'/drone/src/github.com/urbint/machina/_build/dev/lib/expostal/priv/parser.so: cannot open shared object file: No such file or directory\''}}
4s
47
    lib/parser.ex:12: Expostal.Parser.init/0
4s
48
    (kernel) code_server.erl:1347: anonymous fn/1 in :code_server.handle_on_load/5
4s
49
could not compile dependency :expostal, "mix compile" failed. You can recompile this dependency with "mix deps.compile expostal", update it with "mix deps.update expostal" or clean it with "mix deps.clean expostal"
4s
51
14:40:46.797 [warn]  The on_load function for module Elixir.Expostal.Parser returned {{:badmatch,
4s
52
  {:error,
4s
53
   {:load_failed,
4s
54
    'Failed to load NIF library: \'/drone/src/github.com/urbint/machina/_build/dev/lib/expostal/priv/parser.so: cannot open shared object file: No such file or directory\''}}},
4s
55
 [{Expostal.Parser, :init, 0, [file: 'lib/parser.ex', line: 12]},
4s
56
  {:code_server, :"-handle_on_load/5-fun-0-", 1,
4s
57
   [file: 'code_server.erl', line: 1347]}]}
4s
60
14:40:46.799 [error] Process #PID<0.182.0> raised an exception
4s
61
** (MatchError) no match of right hand side value: {:error, {:load_failed, 'Failed to load NIF library: \'/drone/src/github.com/urbint/machina/_build/dev/lib/expostal/priv/expand.so: cannot open shared object file: No such file or directory\''}}
4s
62
    lib/expand.ex:12: Expostal.Expand.init/0
4s
63
    (kernel) code_server.erl:1347: anonymous fn/1 in :code_server.handle_on_load/5
4s
65
14:40:46.800 [warn]  The on_load function for module Elixir.Expostal.Expand returned {{:badmatch,
4s
66
  {:error,
4s
67
   {:load_failed,
4s
68
    'Failed to load NIF library: \'/drone/src/github.com/urbint/machina/_build/dev/lib/expostal/priv/expand.so: cannot open shared object file: No such file or directory\''}}},
4s
69
 [{Expostal.Expand, :init, 0, [file: 'lib/expand.ex', line: 12]},
4s
70
  {:code_server, :"-handle_on_load/5-fun-0-", 1,
4s
71
   [file: 'code_server.erl', line: 1347]}]}

[jdemaris]

The patch I just pushed in PR #4 fixed the issue for me consistently

[xiamx]

Thanks @jdemaris 🎉

[xiamx]

@olafura does version v0.1.4 (including @jdemaris 'spatch) fixes the issues for you ?

[olafura]

@xiamx @jdemaris This works for me thanks ;)