DescribeServices operation is failing with the below error when we perform ECS to EKS conversion

[sendhil-subramani]

Service describe through AWS CLI is working as expected and when I run specctl experiencing the below issues,

specctl -m e2k --ecs_region_name us-east-1 --ecs_cluster_name ecs-cluster1 Traceback (most recent call last): File "/root/specctl/tutorial-env/bin/specctl", line 8, in sys.exit(transform()) File "/root/specctl/tutorial-env/lib64/python3.9/site-packages/click/core.py", line 1157, in call return self.main(args, kwargs) File "/root/specctl/tutorial-env/lib64/python3.9/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/root/specctl/tutorial-env/lib64/python3.9/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, ctx.params) File "/root/specctl/tutorial-env/lib64/python3.9/site-packages/click/core.py", line 783, in invoke return __callback(args, **kwargs) File "/root/specctl/tutorial-env/lib64/python3.9/site-packages/specctl/specctl.py", line 145, in transform e2k_cli_handler(options) File "/root/specctl/tutorial-env/lib64/python3.9/site-packages/specctl/specctl.py", line 69, in e2k_cli_handler ecs_reader_writer(options) File "/root/specctl/tutorial-env/lib64/python3.9/site-packages/specctl/ecs2k8s/ecs_reader_writer.py", line 235, in ecs_reader_writer svc_details = ecs_get_service_details(client, cluster_name, i['serviceArns']) File "/root/specctl/tutorial-env/lib64/python3.9/site-packages/specctl/ecs2k8s/ecs_reader_writer.py", line 173, in ecs_get_service_details response = client.describe_services( File "/root/specctl/tutorial-env/lib64/python3.9/site-packages/botocore/client.py", line 535, in _api_call return self._make_api_call(operation_name, kwargs) File "/root/specctl/tutorial-env/lib64/python3.9/site-packages/botocore/client.py", line 980, in _make_api_call raise error_class(parsed_response, operation_name) botocore.errorfactory.InvalidParameterException: An error occurred (InvalidParameterException) when calling the DescribeServices operation

[arvindsoni80]

Let me take a look. I am assuming that cluster has some ECS service in it.

[sendhil-subramani]

Hi @arvindsoni80 - I understood, Tool is working fine with Fargate ECS and not ECS with EC2. Do you have any plan to release ECS with EC2 to EKS ?

[arvindsoni80]

Likely by end of this year. But if you feel adventurous you can remove this part in the list_services call and see how far you go launchType = "FARGATE",

[sendhil-subramani]

@arvindsoni80, Sure, Thank you so much. I am not great in programming but will give a try and update you.

[sendhil-subramani]

Hi @arvindsoni80 - Thank you so much for your suggestion. We have made changes locally and it works as expected. Thank you once again making our life easy for ECS to kubernetes manifest conversion.

[arvindsoni80]

Oh nice. Glad it helped!

[sumeet-zuora]

are there steps to compile code locally?

[arvindsoni80]

All code is written in python. If you meant to install specctl locally you could do pip install --editable . in the top specctl folder.

[sumeet-zuora]

Already tried that

  Installing build dependencies ... |  done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Requirement already satisfied: PyYAML<7.0,>=6.0 in /Users/schaudhari/.pyenv/versions/3.7.5/lib/python3.7/site-packages (from specctl==0.1.6) (6.0)
Collecting boto3<2.0.0,>=1.26.130 (from specctl==0.1.6)
  Using cached boto3-1.33.5-py3-none-any.whl.metadata (6.7 kB)
Collecting click<9.0.0,>=8.1.3 (from specctl==0.1.6)
  Using cached click-8.1.7-py3-none-any.whl.metadata (3.0 kB)
Collecting kubernetes<27.0.0,>=26.1.0 (from specctl==0.1.6)
  Using cached kubernetes-26.1.0-py2.py3-none-any.whl (1.4 MB)
Collecting pick<3.0.0,>=2.2.0 (from specctl==0.1.6)
  Using cached pick-2.2.0-py3-none-any.whl (4.9 kB)
INFO: pip is looking at multiple versions of specctl to determine which version is compatible with other requirements. This could take a while.
ERROR: Ignored the following versions that require a different python version: 1.0.0 Requires-Python >=3.8
ERROR: Could not find a version that satisfies the requirement python-dotenv<2.0.0,>=1.0.0 (from specctl) (from versions: 0.1.0, 0.1.2, 0.1.3, 0.1.5, 0.2.0, 0.3.0, 0.4.0, 0.5.0, 0.5.1, 0.6.0, 0.6.1, 0.6.2, 0.6.3, 0.6.4, 0.6.5, 0.7.0, 0.7.1, 0.8.0, 0.8.1, 0.8.2, 0.9.0, 0.9.1, 0.10.0, 0.10.1, 0.10.2, 0.10.3, 0.10.4, 0.10.5, 0.11.0, 0.12.0, 0.13.0, 0.14.0, 0.15.0, 0.16.0, 0.17.0, 0.17.1, 0.18.0, 0.19.0, 0.19.1, 0.19.2, 0.20.0, 0.21.0, 0.21.1)
ERROR: No matching distribution found for python-dotenv<2.0.0,>=1.0.0

[arvindsoni80]

What is your python --version ? And you are doing this in a "virtualenv" ya? as in you have done these steps

pip install virtualenv
virtualenv .venv
source .venv/bin/activate

[sumeet-zuora]

Yep, did that

 pip install virtualenv
Requirement already satisfied: virtualenv in /Users/schaudhari/.pyenv/versions/3.7.5/lib/python3.7/site-packages (20.24.7)
Requirement already satisfied: distlib<1,>=0.3.7 in /Users/schaudhari/.pyenv/versions/3.7.5/lib/python3.7/site-packages (from virtualenv) (0.3.7)
Requirement already satisfied: filelock<4,>=3.12.2 in /Users/schaudhari/.pyenv/versions/3.7.5/lib/python3.7/site-packages (from virtualenv) (3.12.2)
Requirement already satisfied: importlib-metadata>=6.6 in /Users/schaudhari/.pyenv/versions/3.7.5/lib/python3.7/site-packages (from virtualenv) (6.7.0)
Requirement already satisfied: platformdirs<5,>=3.9.1 in /Users/schaudhari/.pyenv/versions/3.7.5/lib/python3.7/site-packages (from virtualenv) (4.0.0)
Requirement already satisfied: zipp>=0.5 in /Users/schaudhari/.pyenv/versions/3.7.5/lib/python3.7/site-packages (from importlib-metadata>=6.6->virtualenv) (3.9.0)
Requirement already satisfied: typing-extensions>=3.6.4 in /Users/schaudhari/.pyenv/versions/3.7.5/lib/python3.7/site-packages (from importlib-metadata>=6.6->virtualenv) (4.7.1)

 virtualenv .venv
created virtual environment CPython3.7.5.final.0-64 in 1076ms
  creator CPython3Posix(dest=/Users/schaudhari/ecstoeks/specctl/.venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/schaudhari/Library/Application Support/virtualenv)
    added seed packages: PyYAML==6.0.1, boto3==1.33.2, botocore==1.33.2, cachetools==5.3.2, certifi==2023.11.17, charset_normalizer==3.3.2, click==8.1.7, google_auth==2.23.4, idna==3.6, importlib_metadata==6.7.0, jmespath==1.0.1, kubernetes==26.1.0, oauthlib==3.2.2, pick==2.2.0, pip==23.3.1, pyasn1==0.5.1, pyasn1_modules==0.3.0, python_dateutil==2.8.2, requests==2.31.0, requests_oauthlib==1.3.1, rsa==4.9, s3transfer==0.8.1, setuptools==68.0.0, six==1.16.0, specctl==0.1.5, typing_extensions==4.7.1, urllib3==1.26.18, websocket_client==1.6.1, wheel==0.41.3, zipp==3.15.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
(⎈|ekstools-uw2-na-admin:logging)schaudha-macOS:specctl ⚡️ ekstools-uw2-na-admin $  source .venv/bin/activate

 pip install --editable .
Obtaining file:///Users/schaudhari/ecstoeks/specctl
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Requirement already satisfied: PyYAML<7.0,>=6.0 in ./.venv/lib/python3.7/site-packages (from specctl==0.1.6) (6.0.1)
Requirement already satisfied: boto3<2.0.0,>=1.26.130 in ./.venv/lib/python3.7/site-packages (from specctl==0.1.6) (1.33.2)
Requirement already satisfied: click<9.0.0,>=8.1.3 in ./.venv/lib/python3.7/site-packages (from specctl==0.1.6) (8.1.7)
Requirement already satisfied: kubernetes<27.0.0,>=26.1.0 in ./.venv/lib/python3.7/site-packages (from specctl==0.1.6) (26.1.0)
Requirement already satisfied: pick<3.0.0,>=2.2.0 in ./.venv/lib/python3.7/site-packages (from specctl==0.1.6) (2.2.0)
INFO: pip is looking at multiple versions of specctl to determine which version is compatible with other requirements. This could take a while.
ERROR: Ignored the following versions that require a different python version: 1.0.0 Requires-Python >=3.8
ERROR: Could not find a version that satisfies the requirement python-dotenv<2.0.0,>=1.0.0 (from specctl) (from versions: 0.1.0, 0.1.2, 0.1.3, 0.1.5, 0.2.0, 0.3.0, 0.4.0, 0.5.0, 0.5.1, 0.6.0, 0.6.1, 0.6.2, 0.6.3, 0.6.4, 0.6.5, 0.7.0, 0.7.1, 0.8.0, 0.8.1, 0.8.2, 0.9.0, 0.9.1, 0.10.0, 0.10.1, 0.10.2, 0.10.3, 0.10.4, 0.10.5, 0.11.0, 0.12.0, 0.13.0, 0.14.0, 0.15.0, 0.16.0, 0.17.0, 0.17.1, 0.18.0, 0.19.0, 0.19.1, 0.19.2, 0.20.0, 0.21.0, 0.21.1)
ERROR: No matching distribution found for python-dotenv<2.0.0,>=1.0.0

[arvindsoni80]

Doesn't look like you activated the virtual env

Requirement already satisfied: virtualenv in /Users/schaudhari/.pyenv/versions/3.7.5/lib/python3.7/site-packages (20.24.7)
Requirement already satisfied: distlib<1,>=0.3.7 in /Users/schaudhari/.pyenv/versions/3.7.5/lib/python3.7/site-packages (from virtualenv) (0.3.7)
Requirement already satisfied: filelock<4,>=3.12.2 in /Users/schaudhari/.pyenv/versions/3.7.5/lib/python3.7/site-packages (from virtualenv) (3.12.2)
Requirement already satisfied: importlib-metadata>=6.6 in /Users/schaudhari/.pyenv/versions/3.7.5/lib/python3.7/site-packages (from virtualenv) (6.7.0)
Requirement already satisfied: platformdirs<5,>=3.9.1 in /Users/schaudhari/.pyenv/versions/3.7.5/lib/python3.7/site-packages (from virtualenv) (4.0.0)
Requirement already satisfied: zipp>=0.5 in /Users/schaudhari/.pyenv/versions/3.7.5/lib/python3.7/site-packages (from importlib-metadata>=6.6->virtualenv) (3.9.0)
Requirement already satisfied: typing-extensions>=3.6.4 in /Users/schaudhari/.pyenv/versions/3.7.5/lib/python3.7/site-packages (from importlib-metadata>=6.6->virtualenv) (4.7.1)

 virtualenv .venv
created virtual environment CPython3.7.5.final.0-64 in 1076ms
  creator CPython3Posix(dest=/Users/schaudhari/ecstoeks/specctl/.venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/schaudhari/Library/Application Support/virtualenv)
    added seed packages: PyYAML==6.0.1, boto3==1.33.2, botocore==1.33.2, cachetools==5.3.2, certifi==2023.11.17, charset_normalizer==3.3.2, click==8.1.7, google_auth==2.23.4, idna==3.6, importlib_metadata==6.7.0, jmespath==1.0.1, kubernetes==26.1.0, oauthlib==3.2.2, pick==2.2.0, pip==23.3.1, pyasn1==0.5.1, pyasn1_modules==0.3.0, python_dateutil==2.8.2, requests==2.31.0, requests_oauthlib==1.3.1, rsa==4.9, s3transfer==0.8.1, setuptools==68.0.0, six==1.16.0, specctl==0.1.5, typing_extensions==4.7.1, urllib3==1.26.18, websocket_client==1.6.1, wheel==0.41.3, zipp==3.15.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
(⎈|ekstools-uw2-na-admin:logging)schaudha-macOS:specctl ⚡️ ekstools-uw2-na-admin $  source .venv/bin/activate

<---- ACTIVATE THE VIRTUALENV ----> 
source .venv/bin/activate
<-------------------------------------->

 pip install --editable .

[sumeet-zuora]

Got this

(.venv) (⎈|eks-apps-uw2-na-dev:observability)schaudha-macOS:specctl schaudhari$ pip install --editable .
Obtaining file:///Users/schaudhari/ecstoeks/specctl
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Requirement already satisfied: PyYAML<7.0,>=6.0 in ./.venv/lib/python3.7/site-packages (from specctl==0.1.6) (6.0.1)
Requirement already satisfied: boto3<2.0.0,>=1.26.130 in ./.venv/lib/python3.7/site-packages (from specctl==0.1.6) (1.33.2)
Requirement already satisfied: click<9.0.0,>=8.1.3 in ./.venv/lib/python3.7/site-packages (from specctl==0.1.6) (8.1.7)
Requirement already satisfied: kubernetes<27.0.0,>=26.1.0 in ./.venv/lib/python3.7/site-packages (from specctl==0.1.6) (26.1.0)
Requirement already satisfied: pick<3.0.0,>=2.2.0 in ./.venv/lib/python3.7/site-packages (from specctl==0.1.6) (2.2.0)
INFO: pip is looking at multiple versions of specctl to determine which version is compatible with other requirements. This could take a while.
ERROR: Ignored the following versions that require a different python version: 1.0.0 Requires-Python >=3.8; 1.0.1 Requires-Python >=3.8
ERROR: Could not find a version that satisfies the requirement python-dotenv<2.0.0,>=1.0.0 (from specctl) (from versions: 0.1.0, 0.1.2, 0.1.3, 0.1.5, 0.2.0, 0.3.0, 0.4.0, 0.5.0, 0.5.1, 0.6.0, 0.6.1, 0.6.2, 0.6.3, 0.6.4, 0.6.5, 0.7.0, 0.7.1, 0.8.0, 0.8.1, 0.8.2, 0.9.0, 0.9.1, 0.10.0, 0.10.1, 0.10.2, 0.10.3, 0.10.4, 0.10.5, 0.11.0, 0.12.0, 0.13.0, 0.14.0, 0.15.0, 0.16.0, 0.17.0, 0.17.1, 0.18.0, 0.19.0, 0.19.1, 0.19.2, 0.20.0, 0.21.0, 0.21.1)
ERROR: No matching distribution found for python-dotenv<2.0.0,>=1.0.0

[sumeet-zuora]

got it working,