11.07.2014 meeting

[evdokim]

@alberskib we don't have time to speak today, but we will try to answer you questions in issues. So feel free to ask about compota and other stuff...

about AWS account keep using account that @eparejatobes gave you, but be careful with resource management (delete unused tables and autoscaling groups)

[alberskib]

I managed to create ddwrte app (prototype) that will perform distributed writing but when I run it I receive next error:

com.amazonaws.services.sns.model.AuthorizationErrorException: User:
arn:aws:iam::MY_IAM:user/alberskib is not authorized to perform: 
SNS:CreateTopic on resource: arn:aws:sns:eu-west-1:MY_IAM:nispero_-966230106 (Service: AmazonSNS; Status Code: 403;
 Error Code: AuthorizationError; Request ID: 695a0912-0fa2-56a8-af31-4528fec2618c)

It looks like I do not have sufficient privileges.

[eparejatobes]

OK, I didn't know that SNS was needed for this. Should work now.

[alberskib]

Ok. Now I have another error:

com.amazonaws.AmazonServiceException: User: arn:aws:iam::MY_IAM:user/alberskib is not authorized to perform:
 autoscaling:DescribeAutoScalingGroups (Service: AmazonAutoScaling; Status Code: 403; Error Code: AccessDenied; Request ID: 33c99988-09b8-11e4-966b-618448395c6f)

Currently I use default autoscalling groups - I believe that it is ok (I checked that it will run on single instance) to start with - but if you think differently please let me know.

[eparejatobes]

fixed :)

[alberskib]

:) Unfortunately I come back one more time with another exception:

com.amazonaws.AmazonServiceException: Invalid IamInstanceProfile: compota 
(Service: AmazonAutoScaling; Status Code: 400; Error Code: ValidationError;
 Request ID: 7f82610f-09bc-11e4-94e5-2f1374799dc8)
com.amazonaws.AmazonServiceException: Invalid IamInstanceProfile: compota 
(Service: AmazonAutoScaling; Status Code: 400; Error Code: ValidationError; 
Request ID: 7f82610f-09bc-11e4-94e5-2f1374799dc8)

I dig into code and it looks like incorrect default configuration for autoscalling.

[eparejatobes]

mmm I see. Probably it'd be faster if @evdokim would configure this.

[eparejatobes]

I can try to be @evdokim for 10min

[alberskib]

Generally I think that I will be able to handle it

[eparejatobes]

sure, but I meant the part where I grant you the right privileges :)

[alberskib]

I process it futher - now I have next exception but it could be problem on my side:

 com.amazonaws.AmazonServiceException: Access to the resource https://sqs.eu-west-1.amazonaws.com/ is denied. (Service: AmazonSQS; Status Code: 403; Error Code: AccessDenied; Request ID: 4cedae3d-6f8a-5269-8e78-e2062d4b1b59)

[eparejatobes]

I just created the needed compota IAM role in the Bio4j account.

[eparejatobes]

and now you should be able to launch stuff with that role

[alberskib]

Still next problem:

 com.amazonaws.AmazonServiceException: Access to the resource https://sqs.eu-west-1.amazonaws.com/ is denied. (Service: AmazonSQS; Status Code: 403; Error Code: AccessDenied; Request ID: 4cedae3d-6f8a-5269-8e78-e2062d4b1b59)

Sorry for all troubles

[eparejatobes]

from where?

[alberskib]

Full stack trace:

(run-main-0) com.amazonaws.AmazonServiceException: Access to the resource https://sqs.eu-west-1.amazonaws.com/ is denied. (Service: AmazonSQS; Status Code: 403; Error Code: AccessDenied; Request ID: d36357e3-b5a8-59a5-a648-1bd8ff29c690)
com.amazonaws.AmazonServiceException: Access to the resource https://sqs.eu-west-1.amazonaws.com/ is denied. (Service: AmazonSQS; Status Code: 403; Error Code: AccessDenied; Request ID: d36357e3-b5a8-59a5-a648-1bd8ff29c690)
    at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:820)
    at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:439)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:245)
    at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:2221)
    at com.amazonaws.services.sqs.AmazonSQSClient.createQueue(AmazonSQSClient.java:1150)
    at ohnosequences.nisperon.queues.SQSQueue.createQueue(SQSQueue.scala:343)
    at ohnosequences.nisperon.queues.SQSQueue.getWriter(SQSQueue.scala:322)
    at ohnosequences.nisperon.queues.S3Queue.initWrite(S3Queue.scala:127)
    at ddwriter.DynamograpDistributedWriting$.addTasks(configuration.scala:94)
    at ohnosequences.nisperon.Nisperon.main(Nisperon.scala:161)
    at ddwriter.DynamograpDistributedWriting.main(configuration.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)

It occurs after running distributing writing. It looks like problem with s3queue creation

[eparejatobes]

ok but that's coming from something launched with the compota role?

[alberskib]

In the compta/nispero configuartion there is instanceProfile setted to compota:

val defaultInstanceSpecs = InstanceSpecs(
    instanceType = InstanceType.T1Micro,
    amiId = "",
    securityGroups = List("nispero"),
    keyName = "nispero",
    instanceProfile = Some("compota"),
    deviceMapping = Map("/dev/xvdb" -> "ephemeral0")
  )

I do not know whether somewhere else I could configure to the compota role

[eparejatobes]

It's strange, it should be working. I changed the policy, basically making the resource part in SQS unbounded. Let's see if that helps

[alberskib]

Curently I cannot check it. In about 3 hours i will let you know. I know that there is weekend so yo do not have to devote your free time to it.

[eparejatobes]

no problem :)

[alberskib]

Still the same - I will investigate it

[alberskib]

Generally when I enter SQS page https://eu-west-1.console.aws.amazon.com/sqs I am receiving next message:

You don't have permission to use the Amazon SQS Console.
If you need assistance, contact your System Administrator.

[evdokim]

@alberskib I don't know witch account are you using but anyway, it has to have compota role with some specific permissions. The nispero CLI can create compota profile with root permissions with command nispero configure.

But it seems that @eparejatobes fixed problem with profile. So it should work. Just try to launch it from EC2 instance with instance profile compota.