erlang / otp

Erlang/OTP
http://erlang.org
Apache License 2.0
11.35k stars 2.95k forks source link

ERL-1208: Android build trouble (socket_util.c) #4163

Closed OTP-Maintainer closed 3 years ago

OTP-Maintainer commented 4 years ago

Original reporter: JIRAUSER14400 Affected version: OTP-23.0-rc2 Component: erts Migrated from: https://bugs.erlang.org/browse/ERL-1208


I tried to build beam for android but I get an error during compilation of socket_util.c. Like this:

...

 CC obj/aarch64-unknown-linux-android/opt/smp/socket_util.o
nifs/common/socket_util.c:1622:10: error: duplicate case value '6'
    case PACKET_FASTROUTE:
         ^
/usr/local/lib/android-ndk-r21/toolchains/llvm/prebuilt/linux-x86_64/bin/../sysroot/usr/include/linux/if_packet.h:44:26: note: expanded from macro 'PACKET_FASTROUTE'
#define PACKET_FASTROUTE 6
                         ^
nifs/common/socket_util.c:1610:10: note: previous case defined here
    case PACKET_USER:
         ^
/usr/local/lib/android-ndk-r21/toolchains/llvm/prebuilt/linux-x86_64/bin/../sysroot/usr/include/linux/if_packet.h:42:21: note: expanded from macro 'PACKET_USER'
#define PACKET_USER 6
                    ^
1 error generated.
aarch64-unknown-linux-android/Makefile:828: recipe for target 'obj/aarch64-unknown-linux-android/opt/smp/socket_util.o' failed

...

I use android-ndk-r21 on Ubuntu 18.04.

Cheers
/Joakim

Full transcript:

 MAKE   opt
make[4]: Går till katalogen ”/home/jocke/vendor/otp-OTP-23.0-rc2/erts/emulator”
 GEN    aarch64-unknown-linux-android/gen_git_version.mk
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_main.o
 CC obj/aarch64-unknown-linux-android/opt/smp/preload.o
 EMU_CC obj/aarch64-unknown-linux-android/opt/smp/beam_emu.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_process.o
 CC obj/aarch64-unknown-linux-android/opt/smp/beam_opcodes.o
 CC obj/aarch64-unknown-linux-android/opt/smp/beam_load.o
 CC obj/aarch64-unknown-linux-android/opt/smp/beam_bif_load.o
 CC obj/aarch64-unknown-linux-android/opt/smp/beam_debug.o
 CC obj/aarch64-unknown-linux-android/opt/smp/beam_bp.o
 CC obj/aarch64-unknown-linux-android/opt/smp/beam_catches.o
 CC obj/aarch64-unknown-linux-android/opt/smp/code_ix.o
 CC obj/aarch64-unknown-linux-android/opt/smp/beam_ranges.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_alloc.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_mtrace.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_alloc_util.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_goodfit_alloc.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_bestfit_alloc.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_afit_alloc.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_init.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_atom_table.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_bif_table.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_bif_ddll.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_bif_guard.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_bif_info.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_bif_op.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_bif_os.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_bif_lists.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_bif_persistent.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_bif_atomics.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_bif_counters.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_bif_trace.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_bif_unique.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_guard_bifs.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_dirty_bif_wrap.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_trace.o
 CC obj/aarch64-unknown-linux-android/opt/smp/copy.o
 CC obj/aarch64-unknown-linux-android/opt/smp/utils.o
 CC obj/aarch64-unknown-linux-android/opt/smp/bif.o
 CC obj/aarch64-unknown-linux-android/opt/smp/io.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_printf_term.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_debug.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_md5.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_message.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_proc_sig_queue.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_process_dict.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_process_lock.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_port_task.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_arith.o
 CC obj/aarch64-unknown-linux-android/opt/smp/time.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_time_sup.o
 CC obj/aarch64-unknown-linux-android/opt/smp/external.o
 CC obj/aarch64-unknown-linux-android/opt/smp/dist.o
 CC obj/aarch64-unknown-linux-android/opt/smp/binary.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_db.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_db_util.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_db_hash.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_db_tree.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_thr_progress.o
 CC obj/aarch64-unknown-linux-android/opt/smp/big.o
 CC obj/aarch64-unknown-linux-android/opt/smp/hash.o
 CC obj/aarch64-unknown-linux-android/opt/smp/index.o
 CC obj/aarch64-unknown-linux-android/opt/smp/atom.o
 CC obj/aarch64-unknown-linux-android/opt/smp/module.o
 CC obj/aarch64-unknown-linux-android/opt/smp/export.o
 CC obj/aarch64-unknown-linux-android/opt/smp/register.o
 CC obj/aarch64-unknown-linux-android/opt/smp/break.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_async.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_lock_check.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_dyn_lock_check.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_gc.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_lock_count.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_posix_str.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_bits.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_math.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_fun.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_bif_port.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_term.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_node_tables.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_monitor_link.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_process_dump.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_hl_timer.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_cpu_topology.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_drv_thread.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_bif_chksum.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_bif_re.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_unicode.o
 CC obj/aarch64-unknown-linux-android/opt/smp/packet_parser.o
 CC obj/aarch64-unknown-linux-android/opt/smp/safe_hash.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_zlib.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_nif.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_bif_binary.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_ao_firstfit_alloc.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_thr_queue.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_sched_spec_pre_alloc.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_ptab.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_map.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_msacc.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_lock_flags.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_io_queue.o
 CC obj/aarch64-unknown-linux-android/opt/smp/erl_db_catree.o
 CC obj/aarch64-unknown-linux-android/opt/smp/socket_dbg.o
 CC obj/aarch64-unknown-linux-android/opt/smp/socket_tarray.o
 CC obj/aarch64-unknown-linux-android/opt/smp/socket_util.o
nifs/common/socket_util.c:1622:10: error: duplicate case value '6'
    case PACKET_FASTROUTE:
         ^
/usr/local/lib/android-ndk-r21/toolchains/llvm/prebuilt/linux-x86_64/bin/../sysroot/usr/include/linux/if_packet.h:44:26: note: expanded from macro 'PACKET_FASTROUTE'
#define PACKET_FASTROUTE 6
                         ^
nifs/common/socket_util.c:1610:10: note: previous case defined here
    case PACKET_USER:
         ^
/usr/local/lib/android-ndk-r21/toolchains/llvm/prebuilt/linux-x86_64/bin/../sysroot/usr/include/linux/if_packet.h:42:21: note: expanded from macro 'PACKET_USER'
#define PACKET_USER 6
                    ^
1 error generated.
aarch64-unknown-linux-android/Makefile:828: recipe for target 'obj/aarch64-unknown-linux-android/opt/smp/socket_util.o' failed
OTP-Maintainer commented 4 years ago

bmk said:

Ouch, both PACKET_FASTROUTE and PACKET_USER has the same value on Android.
Skip one of them on Android maybe?

 
OTP-Maintainer commented 4 years ago

bmk said:

I tried to install the env for this myself on my LInux Mint 19.3.

But I I did not find a *android-ndk*. There was a *android-sdk* which I installed, but it did not
include the same files it seems. Any suggestions? Beside choosing a different machine to 
experiment on?
OTP-Maintainer commented 4 years ago

JIRAUSER14400 said:

I used https://developer.android.com/studio/releases/platform-tools and https://developer.android.com/ndk/downloads

I also tried to uncomment PACKET_FASTROUTE in./erts/emulator/nifs/common/socket_util.c, i.e.

#if defined(PACKET_FASTROUTE)
        /*
    case PACKET_FASTROUTE:
        *ePktType = esock_atom_fastroute;
        break;
        */
#endif

Indeed not the right thing to do :-) but the compilation suceeded, albeit with a fair amount of compiler warnings.

I will verify that the build works on an android device as described in the build intructions...
OTP-Maintainer commented 4 years ago

bmk said:

Can you paste a snippet of the include file where PACKET_FASTROUTE and PACKET_USER
are defined, so I can see if there are any useful comments?
OTP-Maintainer commented 4 years ago

JIRAUSER14400 said:

I suceeded to get an Erlang prompt on my Android device after the ugly uncomment procedure described above, i.e.

jocke@eve:~/vendor$ adb push erlang_23.0_arm /data/localjocke@eve:~/vendor$ adb shell
sargo:/ $ /data/local/tmp/erlang_23.0/bin/erl
Eshell V11.0  (abort with ^G)
1> 

To do this I installed:
https://developer.android.com/studio/releases/platform-tools
https://developer.android.com/ndk/downloads

And followed the instructions in:
https://developer.android.com/studio/run/device

It would be really nice if the TODO entries in
https://github.com/erlang/otp/blob/OTP-23.0-rc2/HOWTO/INSTALL-ANDROID.md at some time could be fixed.

Cheers
OTP-Maintainer commented 4 years ago

JIRAUSER14400 said:

PACKET_FASTROUTE etc seems to be defined in the android-ndk release itself but the build error occur in OTP's socket_util.c.

AFAICS:

jocke@eve:~/vendor/otp-OTP-23.0-rc2$ find . -type f -exec grep PACKET_FASTROUTE {} \; -print
#if defined(PACKET_FASTROUTE)
    case PACKET_FASTROUTE:
./erts/emulator/nifs/common/socket_util.c
#if defined(PACKET_FASTROUTE)
    case PACKET_FASTROUTE:
./erts/emulator/nifs/common/#socket_util.c#
#if defined(PACKET_FASTROUTE)
    case PACKET_FASTROUTE:
./erts/emulator/nifs/common/socket_util.c~

jocke@eve:/usr/local/lib/android-ndk-r21$ find . -type f -exec grep PACKET_FASTROUTE {} \; -print
#define PACKET_FASTROUTE 6
./toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include/linux/if_packet.h
#define PACKET_FASTROUTE 6
./sysroot/usr/include/linux/if_packet.h

I get a bit confused over this...
OTP-Maintainer commented 4 years ago

bmk said:

What does net:getifaddrs()  return?

TODO: Any suggestions would be welcome. Please :)

 
OTP-Maintainer commented 4 years ago

bmk said:

On my machine (Linux Mint 19.3) There is a comment in if_packet.h

 
{code:java}
/* Unused, PACKET_FASTROUTE and PACKET_LOOPBACK are invisible to user space */
#define PACKET_FASTROUTE 6 /* Fastrouted frame *{code}
 

This seems to indicate that maybe we should just remote the PACKET_FASTROUTE stuff.
At least until we can figure out what to do with FASTROUTE.

I will create a branch with this change are merge it as soon as we get it through the tests.
OTP-Maintainer commented 4 years ago

JIRAUSER14400 said:

Sounds good

When it comes to the TODO-list in INSTALL-ANDROID.md, i.e.

<cite>

TODO: Propose a permanent fix for the following issue. Adapt the installation specifically for Android, by replacing manually /bin/sh into /system/bin/sh in the various Erlang/OTP release scripts, such as:

bin/erl
bin/start
bin/start_erl
erts-X.Y.Z/bin/erl
erts-X.Y.Z/bin/erl.src
erts-X.Y.Z/bin/start
erts-X.Y.Z/bin/start_erl.src
erts-X.Y.Z/bin/start.src
etc.
WARNING: adb has issues with symlinks (and java.util.zip too). There is only one symlink for epmd in recent Erlang/OTP releases (20 to master-based 23) so it has to be removed before using adb push, and then recreated manually on the target device itself if needed (or epmd can simply be duplicated instead).

$ # Make sure that the epmd symlink is not present before adb push
$ rm bin/epmd

</cite>

That is something I will have to delve into when bundling my app code into a release (with reltools and systools). Time will tell if more Android problems pop up here.
OTP-Maintainer commented 4 years ago

bmk said:

Could you please create a pull request with your suggested updates to INSTALL-ANDROID.md
OTP-Maintainer commented 4 years ago

bmk said:

I have merged my fix (commenting out handling of PACKET_FASTROUTE) 
into master.
OTP-Maintainer commented 4 years ago

jeromedebretagne said:

@Joakim, for your info, the mentioned TODO entry for Android has been fixed with [https://github.com/erlang/otp/pull/2729] (and the corresponding documentation has been updated).

Cheers,
 Jérôme