noobaa / noobaa-core

High-performance S3 application gateway to any backend - file / s3-compatible / multi-clouds / caching / replication ...
https://www.noobaa.io
Apache License 2.0
270 stars 78 forks source link

Question: Does Nooba supports Security Token Service ? #6224

Closed anuj812 closed 11 months ago

anuj812 commented 4 years ago

Does S3 compatible storage through Nooba supports Security Token Service ? We tried using aws sts commands on OCS object storage on Openshift 4.5 but didn't get any success. We were looking to migrate from minio to ocs but seeems to be a problem as it doesn't seems to be supporting STS . We tried from AWS Java sdk as well as aws cli as well 'aws sts' commands. Please confirm.

guymguym commented 4 years ago

Thanks for the question @anuj812. Tagging @jeniawhite and @nimrod-becker to provide you with more info.

jindong-ibm commented 4 years ago

@jeniawhite and @nimrod-becker Can you please help provide some guide/info? Thanks much.

jeniawhite commented 4 years ago

@jindong-ibm @anuj812 Hello, We do not have STS. Previously, I've created a POC that implements STS and IAM in NooBaa. The POC allowed configuring Azure's AD with NooBaa and using it (Azure AD) as a web identity provider. Sadly due to other higher priority tasks, we've dropped the POC and didn't advance with it.

I would like to get more input on your use cases and the flows that you are interested in. 1) What operations are you using in the STS (if possible with a deep dive details of the configurations and options as well), do you use SAML, web identity, or other methods? 3) Do you need anything besides STS (IAM for example to manage the rules for the STS?) 4) Who is the identity provider for your STS and how do you configure it? (AWS AD FS, Azure AD, others?) 5) How do you sync NooBaa's IAM roles (if you have any) with the identity providers?

Basically, I'm interested in any info that I can get regarding your plans and current usage. This will allow me to understand the gaps and what are the actual use cases that interest you. Thank you.

anuj812 commented 4 years ago
  1. We create a subuser and then just use the AssumeRole feature to get the session creds which consist of temporary access key, secret key and session token. I can share pseudo code of Java client leverage STS from AWS SDK to confirm the capability we are using. We are not using any web identity so far and usage restricted to assume role only.
  2. No IAM
  3. No identity provider so far, we create subuser directly in atleast in MinIO using add user commands. We use admin user similar to admin@nooba.io in OCS. in nooba, problem is we are not able to create a subuser. We try to leverage the Nooba Management Console to create one account/subuser but we were not able to apply permission at sub-user level and later when tried to assume role operation , getting 403 issue.
  4. Since no IAM so no sync. Pseudo Code for Current Usage.
import com.amazonaws.ClientConfiguration;
import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.auth.policy.Policy;
import com.amazonaws.auth.policy.Statement;
import com.amazonaws.auth.policy.actions.S3Actions;
import com.amazonaws.client.builder.AwsClientBuilder;
import com.amazonaws.services.securitytoken.AWSSecurityTokenService;
import com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClientBuilder;
import com.amazonaws.services.securitytoken.model.AssumeRoleRequest;
import com.amazonaws.services.securitytoken.model.AssumeRoleResult;
import com.amazonaws.services.securitytoken.model.Credentials;
import com.amazonaws.auth.policy.Resource;

public class TestOCSSubUser {

    public static void main(String[] args) {
        // TODO Auto-generated method stub

        Policy policy = new Policy();
        Statement statement = new Statement(Statement.Effect.Allow);
        String resource;

        resource = String.format("arn:aws:s3:::mydata/*");

        statement.withActions(S3Actions.GetObject).withResources(new Resource(resource));
        policy.withStatements(statement);

        AssumeRoleRequest roleRequest = new AssumeRoleRequest()
                .withRoleArn("arn:aws:s3:::mydata/*")
                .withRoleSessionName("opendes")
                .withDurationSeconds(7200)
                .withPolicy(policy.toJson());

         AWSCredentials credentials = new BasicAWSCredentials("ACCESS-KEY-OF-SUBUSER",
                 "SECRET-KEY-OF--SUBUSER"); 
         ClientConfiguration clientConfiguration = new ClientConfiguration();

// Optional based on SSL handshake of selfsigned cert
         System.setProperty("javax.net.ssl.trustStore", "C:\\CEP\\cep-svc\\osdu-bvt-ocs.jks");
         System.setProperty("javax.net.ssl.trustStorePassword", "JKS Password");

         AWSSecurityTokenService stsClient =
                 AWSSecurityTokenServiceClientBuilder.standard()
                 .withEndpointConfiguration(new
                 AwsClientBuilder.EndpointConfiguration("https://"+"s3-ocs-url-route", "us-east-1"))
                 .withClientConfiguration(clientConfiguration) .withCredentials(new
                 AWSStaticCredentialsProvider(credentials)) .build();

         AssumeRoleResult response = stsClient.assumeRole(roleRequest);
         Credentials session_creds = response.getCredentials();

         System.out.println("AccessKey "+session_creds.getAccessKeyId()+"Secret Key "+ session_creds.getSecretAccessKey());

    }

//Application Specific Code.    
     public Policy createPolicy(String srn, String s3BucketName, String s3ObjectKey) {

            Policy policy = new Policy();
            Statement statement = new Statement(Statement.Effect.Allow);
            String resource;

            // Application Buisness Logic

            resource = String.format("arn:aws:s3:::%s/%s/*",s3BucketName,s3ObjectKey);

            statement.withActions(S3Actions.GetObject).withResources(new Resource(resource));
            policy.withStatements(statement);

            return policy;
        }

}

Policy which we apply at sub-user level manually as one time activity

'{
    "Version": "2012-10-17",
    "Statement": [
       {
        "Effect": "Allow",
        "Action": "s3:ListBucket",
        "Resource": "arn:aws:s3:::mydata"
       },
       {
        "Effect": "Allow",
        "Action": [ "s3:GetObject", "s3:PutObject" ],
        "Resource": "arn:aws:s3:::mydata/*"
      }
   ]
}
jindong-ibm commented 4 years ago

@jeniawhite and @nimrod-becker Please check above post ^^ Can you help to give any suggestion/workaround? Team need to evaluate this situation and come up with "plan B" if Nooba can not work around the issue. So I hope we can have a final "yes, can do" or "no, cannot" answer today or tomorrow, to help team make decision. Thanks for your help.

nimrod-becker commented 4 years ago

@jindong-ibm thanks for the info, it does scope what's needed and help us understand he exact request.

It's not available now, but assessing the needed work, there isn't a lot in order to support this. We won't be able to pick this up in the immediate future but can tackle it a little further down the road. On the other hand, we can help if you want to give it a stab and add this support.

jindong-ibm commented 4 years ago

Thanks much for reply @nimrod-becker . We are planning to use Red Hat OCS (openshift container storage) which using noobaa for object storage. Yes, we are happy to give a try when you have it support.

If it is alright, I hope to keep this item open so to track this support and communicate. But if you have any other channel or other way, please let us know, we are happy to follow. Thanks

nimrod-becker commented 3 years ago

We will keep the bug open of course :)

stale[bot] commented 12 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.