imageid1 does not exist using Amazon EC2

[Giuseppe1992]

Hello, I am trying to run an experiment with Amazon AWS; Following the instructions, I managed to have the first test with CLOUDSIM working.

Now I want to run an experiment with Amazon AWS. Following the instruction, I set the configuration file as follow. The VM running CBench is in AWS.

ubuntu@ip-172-31-24-168:~$ cat cbtool/cbtool/configs/ubuntu_cloud_definitions.txt
[USER-DEFINED]
# Variable $IP_AUTO attempts to auto-discover IP address
MANAGER_IP = $IP_AUTO
MAIN_UPDATE_FREQUENCY = 5
MAIN_UPDATE_ATTEMPTS = 36
MAIN_NOTIFICATION = $False
MAIN_ATTEMPTS = 3
MAIN_TIMEOUT = 240
# Variable $USER auto-selects current username
MAIN_USERNAME = $USER_AUTO
# This is the username created inside the VMs
MAIN_LOGIN = cbuser
# Variable $PATH_TO_TOOL attempts to auto-selects the appropriate path
MAIN_BASE_DIR = $PATH_TO_TOOL

# Which cloud to pick first when the tool first starts
STARTUP_CLOUD = MYEC2

# List the clouds you want to define
....
#-------------------------------------------------------------------------------
# Elastic Compute Cloud (EC2) requires the following parameters
[USER-DEFINED : CLOUDOPTION_MYEC2]
EC2_ACCESS = xxxxxxxxxxxxxxxxxxxxxx                                                                                     # This is the AWS access key id
EC2_CREDENTIALS = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx                                                          # This is the AWS secret access key
EC2_SECURITY_GROUPS = cloudbench                                                                                      # Make sure that this group exists first
EC2_KEY_NAME = cbtool_rsa                                                                                             # Make sure that this key exists first
EC2_INITIAL_VMCS = eu-central-1                                                                                      # Change "us-east-1" accordingly
EC2_SSH_KEY_NAME = cbtool_rsa                                                                                         # Name of the private/public key pair - locally available - used to login on the VMs
EC2_KEY_NAME = cbtool_rsa                                                                                             # Name of the private key present in the cloud to be injected on VMs
EC2_LOGIN = cbuser                                                                                                    # The username that logins on the VMs
EC2_NETNAME = private                                                                                                 # If your Orchestrator is running inside the cloud, use "private". If external to the cloud, use "public".For more complex networks, refer to the documentation
#-------------------------------------------------------------------------------
....
[VM_TEMPLATES : EC2_CLOUDCONFIG ]
CASSANDRA = size:04-16384, imageid1:ami-0d359437d1756caa8, login:ubuntu
....

I tried with different ami, but I have the same error. This is the error that I get:

ubuntu@ip-172-31-24-168:~/cbtool$ cbtool/cb --hard_reset
Cbtool version is "55c3ae7c"
Parsing "cloud definitions" file..... "/home/ubuntu/cbtool/lib/auxiliary//../..//configs/ubuntu_cloud_definitions.txt" opened and parsed successfully.

Checking "Object Store"..... An Object Store of the kind "Redis" (shared) on node 172.31.24.168, TCP port 6379, database id "0" seems to be running.
Checking "Log Store"..... A Log Store of the kind "rsyslog" (private) on node 172.31.24.168, UDP port 5114 seems to be running.
Checking "Metric Store"..... A Metric Store of the kind "MongoDB" (shared) on node 172.31.24.168, TCP port 27017, database id "metrics" seems to be running.
Checking "File Store"..... A File Store of the kind "rsync" (private) on node 172.31.24.168, TCP port 873 seems to be running.

Executing "hard" reset: (killing all running toolkit processes and flushing stores) before starting the experiment......
Only flush registered cloud: MYEC2
    Killing all processes... done
    Flushing Object Store... done
    Flushing Log Store... done
    Flushing Metric Store... done

Checking for a running API service daemon..... API Service daemon was successfully started. The process id is ['4261'] (http://172.31.24.168:7070).
Checking for a running GUI service daemon..... GUI Service daemon was successfully started. The process id is ['4284', '4285'], listening on port 8080. Full url is "http://172.31.24.168:8080".

 ############################# Executing command "cldattach ec2 MYEC2" (specified on the configuration file)

 status: VPN configuration for this cloud already generated: /home/ubuntu/cbtool/lib/auxiliary//../../configs/generated/MYEC2_server-cb-openvpn.conf

 status: Checking if the ssh key pair "ubuntu_cbtool_rsa" is created on VMC eu-central-1....
 status: Checking if the security group "cloudbench" is created on VMC eu-central-1....
 status: Checking if the imageids associated to each "VM role" are registered on VMC "eu-central-1"....
 status: VMC "eu-central-1" did not pass the connection test." : EC2ResponseError: 400 Bad Request
<?xml version="1.0" encoding="UTF-8"?>
<Response><Errors><Error><Code>InvalidAMIID.NotFound</Code><Message>The image id '[ami-04b9e92b5572fa0d1]' does not exist</Message></Error></Errors><RequestID>2c9a20f0-fdf4-4334-bfb7-0f31edd8331b</RequestID></Response>
 status: Traceback (most recent call last):
 status:   File "/home/ubuntu/cbtool/lib/operations/active_operations.py", line 256, in cldattach
 status:     cld_attr_lst["vmc_defaults"])
 status:   File "/home/ubuntu/cbtool/lib/clouds/ec2_cloud_ops.py", line 148, in test_vmc_connection
 status:     _status, _msg = self.common_messages("VMC", {"name" : vmc_name }, "connected", _status, _fmsg)
 status:   File "/home/ubuntu/cbtool/lib/clouds/shared_functions.py", line 1791, in common_messages
 status:     raise CldOpsException(_msg, status)
 status: lib.clouds.shared_functions.CldOpsException: VMC "eu-central-1" did not pass the connection test." : EC2ResponseError: 400 Bad Request
 status: <?xml version="1.0" encoding="UTF-8"?>
 status: <Response><Errors><Error><Code>InvalidAMIID.NotFound</Code><Message>The image id '[ami-04b9e92b5572fa0d1]' does not exist</Message></Error></Errors><RequestID>2c9a20f0-fdf4-4334-bfb7-0f31edd8331b</RequestID></Response>

The EC2 cloud named "MYEC2" could not be attached to this experiment: VMC "eu-central-1" did not pass the connection test." : EC2ResponseError: 400 Bad Request
<?xml version="1.0" encoding="UTF-8"?>
<Response><Errors><Error><Code>InvalidAMIID.NotFound</Code><Message>The image id '[ami-04b9e92b5572fa0d1]' does not exist</Message></Error></Errors><RequestID>2c9a20f0-fdf4-4334-bfb7-0f31edd8331b</RequestID></Response>

 ############################# Executing command "vmcattach all" (specified on the configuration file)

 status: Traceback (most recent call last):
 status:   File "/home/ubuntu/cbtool/lib/operations/base_operations.py", line 960, in initialize_object
 status:     _time_attr_list = self.osci.get_object(obj_attr_list["cloud_name"], "GLOBAL", False, "time", False)
 status:   File "/home/ubuntu/cbtool/lib/stores/redis_datastore_adapter.py", line 852, in get_object
 status:     raise self.ObjectStoreMgdConnException(str(_msg), 23)
 status: lib.stores.redis_datastore_adapter.RedisMgdConn.ObjectStoreMgdConnException: GLOBAL object time could not be retrieved from object list (FQIN: TEST_ubuntu:MYEC2:GLOBAL).
(MYEC2) exit

Bests Giuseppe

[mraygalaxy]

Your configuration looks good. Are you sure the AMI is available with the credentials that you have supplied? The error returned is The image id '[ami-04b9e92b5572fa0d1]' does not exist

[Giuseppe1992]

Your configuration looks good. Are you sure the AMI is available with the credentials that you have supplied? The error returned is The image id '[ami-04b9e92b5572fa0d1]' does not exist

Yes, I am sure, the AMI is the public ubuntu18.04 image available in Frankfurt.

The problem is that it is using ami-04b9e92b5572fa0d1, while in the Configuration file I set:

....
[VM_TEMPLATES : EC2_CLOUDCONFIG ]
CASSANDRA = size:04-16384, imageid1:ami-0d359437d1756caa8, login:ubuntu
....

Is this the correct way to set the ami, or there is a default value somewhere else?

[Giuseppe1992]

I modified the default _NEST_CONTAINERS_BASEIMAGE

[VM_TEMPLATES : EC2_CLOUDCONFIG ]
NEST_CONTAINERS_BASE_IMAGE = size:NA, imageids:1, imageid1:ami-0d359437d1756caa8, cloudinit_packages:openvpn;bc;jq;docker.io;python;redis-server
TEST = size:02-8192, imageid1:ami-0d359437d1756caa8, login:ubuntu

seems to work now

Bests Giuseppe

[mraygalaxy]

Oh, are you using our nesting support? (Not creating your own base images?) That wasn't clear.