GoogleCloudPlatform / anthos-service-mesh-packages

Packaged configuration for setting up a Kubernetes cluster with Anthos Service Mesh features enabled
https://cloud.google.com/anthos/service-mesh
Apache License 2.0
134 stars 169 forks source link

asmcli install segmentation fault #1182

Open ryanhmasten opened 2 years ago

ryanhmasten commented 2 years ago

I know MacOS is not officially supported but I did see you try and look into issues still when they happen. Trying to install on MacOS 12.3 (x86) and getting the following error:

asmcli: Getting account information... asmcli: Downloading kpt.. % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 669 100 669 0 0 2846 0 --:--:-- --:--:-- --:--:-- 2834 100 12.3M 100 12.3M 0 0 14.2M 0 --:--:-- --:--:-- --:--:-- 14.2M asmcli: Downloading ASM.. % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 41.8M 100 41.8M 0 0 41.0M 0 0:00:01 0:00:01 --:--:-- 41.0M asmcli: Downloading ASM kpt package... [2] 87836 segmentation fault ./asmcli install

zerobfd commented 2 years ago

Thank you for the report! This must have been something fairly recent since I got one internal report a couple of days ago after a long stretch of no MacOS issues. I'll try and get a fix in as soon as I can get a repro case and figure out what changed.

ryanhmasten commented 2 years ago

Thanks. Is there any log/debug information I could provide to assist?

zerobfd commented 2 years ago

There's nothing obvious I can think of. From first looks it seems like the code that's supposed to override the system kpt installation broke somehow--could be a change in the default Mac shell, some GNU/BSD coreutils incompatibility, filesystem change...if you could add set -x at the top of the file and reproduce it the logs would be useful, but there's all sorts of private information that could get included and I don't want to put the onus on anyone to try and scrub logs before posting something on a public GH repo.

So short answer: not really, but thank you for offering :)

nitendra-thakur commented 2 years ago

The issue is with asmcli validate as well. Ran it with -v (verbose) option; and looks like it is stuck at this step:

asmcli: Downloading ASM kpt package...
asmcli: Running: 'kpt pkg get --auto-set=false https://github.com/GoogleCloudPlatform/anthos-service-mesh-packages.git/asm@main asm'
asmcli: -------------
asmcli: Running: 'kpt pkg get --auto-set=false https://github.com/GoogleCloudPlatform/anthos-service-mesh-packages.git/asm@main asm'
asmcli: -------------

That line keeps repeating and eventually fails with "Segmentation Fault: 11"

Interestingly, I can cd to the output_dir and run the same command without any error. So this worked:

$ cd ./out/
$ ./kpt pkg get --auto-set=false https://github.com/GoogleCloudPlatform/anthos-service-mesh-packages.git/asm@main asm
fetching package "/asm" from "https://github.com/GoogleCloudPlatform/anthos-service-mesh-packages" to "asm"
$

Would be nice to have a documentation for manual steps that I can use to setup asm; skipping asmcli.

zerobfd commented 2 years ago

Random question: is this problem cropping up on a Mac?

nitendra-thakur commented 2 years ago

Yes. on MacOS Catalina version 10.15.7

$ echo $0
-bash
$ bash --version
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin19)
Copyright (C) 2007 Free Software Foundation, Inc.

Even tried with this; but no luck

$ /usr/local/bin/bash --version
GNU bash, version 5.1.16(1)-release (x86_64-apple-darwin19.6.0)
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
$
zerobfd commented 2 years ago

I know this isn't a great response, but we don't support MacOS for reasons like this. Apple has a history of changing seemingly random shell-related configs/tools between releases, and we don't have any way of doing any kind of automated testing. You should be able to run it on Cloud Shell, or use a Docker container. There's a Dockerfile in asmcli/ that's used for testing.

If we can get someone with an affected machine to find the issue I'm happy to make a compatibility fix.

phspagiari commented 2 years ago

Installing coreutils on macos resolve any issues with asmcli or migrate_addon. I added a simple fix specifically for kpt here: https://github.com/GoogleCloudPlatform/anthos-service-mesh-packages/pull/1320

For migrate_addon there's also a problem with convert binary.

jojonicho commented 1 year ago

I'm also having this issue on Mac OS

./asmcli install \
--project_id ${PROJECT_ID} \
--cluster_name ${CLUSTER_NAME} \
--cluster_location ${ZONE} \
--fleet_id ${PROJECT_ID} \
--output_dir ${CLUSTER_NAME} \
--enable-all \
--ca mesh_ca

and got the error:

asmcli: Downloading ASM kpt package...
[2] 29878 segmentation fault ./asmcli install --project_id ${PROJECT_ID} --cluster_name ${CLUSTER_NAME}

I've also installed coreutils as suggested by @phspagiari ,

corinz commented 1 year ago

You need to be on a later version of bash, not the apple default to get rid of the segfault. But even then, there's kpt invocation issues when running on macos. Run it in cloud shell