lamikr / rocm_sdk_builder

Other
113 stars 8 forks source link

/usr/bin/ld: cannot find -lmsgpackc-cxx: No such file or directory #11

Closed flip111 closed 4 weeks ago

flip111 commented 1 month ago

OS: Ubuntu 24.04 GPU: Sapphire Radeon RX 7900 XTX Nitro+ Vapor-X Aktiv PCIe 4.0 x16

cd /home/flip111/programs/src/rocm_sdk_builder/builddir/035_AMDMIGraphX/src && /usr/bin/cmake -E cmake_link_script CMakeFiles/migraphx.dir/link.txt --verbose=1
/opt/rocm_sdk_611/bin/clang++ -fPIC -O3 -DNDEBUG -L/opt/rocm_sdk_611/lib64 -L/opt/rocm_sdk_611/lib -L/opt/rocm_sdk_611/hsa/lib -L/opt/rocm_sdk_611/rocblas/lib -L/opt/rocm_sdk_611/hcc/lib -shared -Wl,-soname,libmigraphx.so.2009000 -o ../lib/libmigraphx.so.2009000.0.60101 CMakeFiles/migraphx.dir/adjust_allocation.cpp.o CMakeFiles/migraphx.dir/analyze_streams.cpp.o CMakeFiles/migraphx.dir/apply_alpha_beta.cpp.o CMakeFiles/migraphx.dir/argument.cpp.o CMakeFiles/migraphx.dir/autocast_fp8.cpp.o CMakeFiles/migraphx.dir/auto_contiguous.cpp.o CMakeFiles/migraphx.dir/common.cpp.o CMakeFiles/migraphx.dir/common_dims.cpp.o CMakeFiles/migraphx.dir/compile_src.cpp.o CMakeFiles/migraphx.dir/convert_to_json.cpp.o CMakeFiles/migraphx.dir/cpp_generator.cpp.o CMakeFiles/migraphx.dir/dead_code_elimination.cpp.o CMakeFiles/migraphx.dir/dom_info.cpp.o CMakeFiles/migraphx.dir/dynamic_loader.cpp.o CMakeFiles/migraphx.dir/eliminate_allocation.cpp.o CMakeFiles/migraphx.dir/eliminate_common_subexpression.cpp.o CMakeFiles/migraphx.dir/eliminate_concat.cpp.o CMakeFiles/migraphx.dir/eliminate_contiguous.cpp.o CMakeFiles/migraphx.dir/eliminate_convert.cpp.o CMakeFiles/migraphx.dir/eliminate_data_type.cpp.o CMakeFiles/migraphx.dir/eliminate_identity.cpp.o CMakeFiles/migraphx.dir/eliminate_pad.cpp.o CMakeFiles/migraphx.dir/env.cpp.o CMakeFiles/migraphx.dir/file_buffer.cpp.o CMakeFiles/migraphx.dir/fp_to_double.cpp.o CMakeFiles/migraphx.dir/fuse_concat.cpp.o CMakeFiles/migraphx.dir/fuse_pointwise.cpp.o CMakeFiles/migraphx.dir/fuse_reduce.cpp.o CMakeFiles/migraphx.dir/generate.cpp.o CMakeFiles/migraphx.dir/inline_module.cpp.o CMakeFiles/migraphx.dir/insert_pad.cpp.o CMakeFiles/migraphx.dir/instruction.cpp.o CMakeFiles/migraphx.dir/json.cpp.o CMakeFiles/migraphx.dir/layout_nhwc.cpp.o CMakeFiles/migraphx.dir/load_save.cpp.o CMakeFiles/migraphx.dir/make_op.cpp.o CMakeFiles/migraphx.dir/memory_coloring.cpp.o CMakeFiles/migraphx.dir/module.cpp.o CMakeFiles/migraphx.dir/msgpack.cpp.o CMakeFiles/migraphx.dir/normalize_attributes.cpp.o CMakeFiles/migraphx.dir/normalize_ops.cpp.o CMakeFiles/migraphx.dir/op_enums.cpp.o CMakeFiles/migraphx.dir/operation.cpp.o CMakeFiles/migraphx.dir/optimize_module.cpp.o CMakeFiles/migraphx.dir/pad_calc.cpp.o CMakeFiles/migraphx.dir/pass.cpp.o CMakeFiles/migraphx.dir/pass_manager.cpp.o CMakeFiles/migraphx.dir/permutation.cpp.o CMakeFiles/migraphx.dir/preallocate_param.cpp.o CMakeFiles/migraphx.dir/process.cpp.o CMakeFiles/migraphx.dir/program.cpp.o CMakeFiles/migraphx.dir/propagate_constant.cpp.o CMakeFiles/migraphx.dir/promote_literals.cpp.o CMakeFiles/migraphx.dir/quantization.cpp.o CMakeFiles/migraphx.dir/quantize_fp16.cpp.o CMakeFiles/migraphx.dir/quantize_8bits.cpp.o CMakeFiles/migraphx.dir/reduce_dims.cpp.o CMakeFiles/migraphx.dir/register_op.cpp.o CMakeFiles/migraphx.dir/register_target.cpp.o CMakeFiles/migraphx.dir/replace_allocate.cpp.o CMakeFiles/migraphx.dir/rewrite_reduce.cpp.o CMakeFiles/migraphx.dir/simplify_qdq.cpp.o CMakeFiles/migraphx.dir/sqlite.cpp.o CMakeFiles/migraphx.dir/rewrite_gelu.cpp.o CMakeFiles/migraphx.dir/rewrite_pooling.cpp.o CMakeFiles/migraphx.dir/rewrite_quantization.cpp.o CMakeFiles/migraphx.dir/rewrite_rnn.cpp.o CMakeFiles/migraphx.dir/schedule.cpp.o CMakeFiles/migraphx.dir/serialize.cpp.o CMakeFiles/migraphx.dir/shape.cpp.o CMakeFiles/migraphx.dir/simplify_algebra.cpp.o CMakeFiles/migraphx.dir/simplify_dyn_ops.cpp.o CMakeFiles/migraphx.dir/simplify_reshapes.cpp.o CMakeFiles/migraphx.dir/split_single_dyn_dim.cpp.o CMakeFiles/migraphx.dir/target.cpp.o CMakeFiles/migraphx.dir/tmp_dir.cpp.o CMakeFiles/migraphx.dir/value.cpp.o CMakeFiles/migraphx.dir/verify_args.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_abs_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_acosh_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_acos_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_add_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_allocate_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_argmax_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_argmin_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_asinh_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_asin_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_as_shape_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_atanh_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_atan_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_broadcast_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_capture_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_ceil_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_clip_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_concat_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_contiguous_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_convert_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_convolution_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_convolution_backwards_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_cosh_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_cos_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_dequantizelinear_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_dimensions_of_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_div_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_dot_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_elu_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_equal_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_erf_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_exp_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_fill_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_flatten_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_floor_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_fmod_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_gather_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_gathernd_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_get_tuple_elem_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_greater_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_gru_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_identity_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_if_op_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_im2col_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_isinf_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_isnan_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_layout_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_leaky_relu_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_less_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_load_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_log_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_logical_and_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_logical_or_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_logical_xor_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_logsoftmax_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_loop_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_lrn_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_lstm_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_max_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_min_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_mod_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_mul_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_multibroadcast_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_multinomial_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_nearbyint_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_neg_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_nonmaxsuppression_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_nonzero_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_outline_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_pad_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_pointwise_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_pooling_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_pow_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_prefix_scan_sum_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_prelu_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_quant_convolution_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_quant_dot_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_quantizelinear_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_random_uniform_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_random_seed_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_recip_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_reduce_max_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_reduce_mean_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_reduce_min_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_reduce_prod_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_reduce_sum_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_relu_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_reshape_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_reshape_lazy_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_reverse_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_rnn_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_rnn_last_cell_output_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_rnn_last_hs_output_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_rnn_var_sl_last_output_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_roialign_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_rsqrt_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_run_on_target_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_scalar_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_scatter_none_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_scatter_add_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_scatter_mul_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_scatter_min_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_scatter_max_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_scatternd_add_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_scatternd_mul_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_scatternd_none_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_scatternd_max_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_scatternd_min_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_select_module_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_sigmoid_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_sign_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_sinh_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_sin_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_slice_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_softmax_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_sqdiff_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_sqrt_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_squeeze_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_step_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_sub_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_tanh_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_tan_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_topk_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_transpose_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_unary_not_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_undefined_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_unique_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_unknown_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_unsqueeze_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_where_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_op_rnn_variable_seq_lens_hpp.cpp.o CMakeFiles/migraphx.dir/ops/migraphx_builtin_hpp.cpp.o  -Wl,-rpath,/home/flip111/programs/src/rocm_sdk_builder/builddir/035_AMDMIGraphX/lib: -lstdc++fs -ldl /usr/lib/x86_64-linux-gnu/libsqlite3.so -lmsgpackc-cxx
/usr/bin/ld: cannot find -lmsgpackc-cxx: No such file or directory
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [src/CMakeFiles/migraphx.dir/build.make:3426: lib/libmigraphx.so.2009000.0.60101] Error 1
make[2]: Leaving directory '/home/flip111/programs/src/rocm_sdk_builder/builddir/035_AMDMIGraphX'
make[1]: *** [CMakeFiles/Makefile2:5912: src/CMakeFiles/migraphx.dir/all] Error 2
make[1]: Leaving directory '/home/flip111/programs/src/rocm_sdk_builder/builddir/035_AMDMIGraphX'
make: *** [Makefile:166: all] Error 2
build failed: AMDMIGraphX
lamikr commented 1 month ago

Did you found a solution to this one yet? If not, I will try tomorrow install the latest ubuntu and use it instead of 2023.10 version I have used for build testing from Ubuntu. It could be some msgpackc- devel dep file that needs to be installed.

flip111 commented 1 month ago

It might be the missing package libmsgpuck-dev at first i couldn't find it because of puck instead of pack, odd but it's like that. I will install it then recompile again.

Question: is recompiling from the point it stopped last time supported or do i need to wipe the build dir?

EDIT: already quickly found out it's not libmsgpuck-dev and libmsgpack-cxx-dev was already installed

flip111 commented 1 month ago

I think it's due to renaming of the library

https://packages.ubuntu.com/mantic/libmsgpack-cxx-dev https://packages.ubuntu.com/noble/libmsgpack-cxx-dev

https://github.com/msgpack/msgpack-c/wiki/Q&A

lamikr commented 1 month ago

Just checked that on my ubuntu 39, I have libmsgpack-cxx-dev installed. I will setup Ubuntu 40 to try this out.

# apt list --installed | grep msgpack

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

libmsgpack-cxx-dev/mantic,now 4.1.3-1 amd64 [installed,automatic]
libmsgpack-dev/mantic,now 4.0.0-3 amd64 [installed]
libmsgpackc2/mantic,now 4.0.0-3 amd64 [installed,automatic]
lamikr commented 1 month ago

@flip111 Usually the build can just continue from where it failed, unless there are fixes to the older packages. Inside each build folder there are .result_* files and you can remove those if you want to force certain steps to be re-run for some package. (Like "rm -rf builddir/007_02_rocminfo/.result_install" for example)

Once some step has been succesfully done, it creates that file. Python apps are little more problematic as at the moment, they are modifying also the content in source directory. (In pytorch aten and hipify for example modify some cuda-files to amd friendly format in pytorch source directory and at the moment I do not have any clean solution in babs.sh itself for resetting those cleanly)

TheRikke commented 1 month ago

I could resolve the issue by creating a link to the new library name: sudo ln -s /usr/lib/x86_64-linux-gnu/libmsgpackc.so.2.0.0 /usr/lib/x86_64-linux-gnu/libmsgpackc-cxx.so

eLBart0-DTG commented 1 month ago

I could resolve the issue by creating a link to the new library name: sudo ln -s /usr/lib/x86_64-linux-gnu/libmsgpackc.so.2.0.0 /usr/lib/x86_64-linux-gnu/libmsgpackc-cxx.so

I can confirm this works. Of course it's not the best solution, but you can definitely try it, if you don't want to wait for an official work-around.

lamikr commented 1 month ago

Thanks for the findings, I have not had time yet to work on Ubuntu 24.04. WIll plan to do that once last 2 Fedora 40 problems are solved. There are discussion from same kind of thing on

https://github.com/lamikr/rocm_sdk_builder/pull/27

Maybe the project CMakefiles could be changed to optionally try to search both the msgpack and msgpack-cxx options for AMDMIGraphX?

lamikr commented 4 weeks ago

Pushed fix does not require creating symlink for testing. Tested already on Ubuntu 24.04, now doing just other test build on other distro to check it does not break anything