awslabs / aws-fluent-plugin-kinesis

Amazon Kinesis output plugin for Fluentd
Apache License 2.0
293 stars 96 forks source link

aws-fluent-plugin-kinesis plugin takes ~8 minutes to install #145

Closed jamesongithub closed 5 years ago

jamesongithub commented 6 years ago

this plugin installs every single ruby aws sdk dependency it looks like, can we parse down the dependencies? seems unnecessary.

/usr/sbin/td-agent-gem install fluent-plugin-kinesis

       "Successfully installed aws-sdk-acm-1.7.0", 
        "Successfully installed aws-sdk-acmpca-1.0.0", 
        "Successfully installed aws-sdk-apigateway-1.10.0", 
        "Successfully installed aws-sdk-alexaforbusiness-1.4.0", 
        "Successfully installed aws-sdk-appstream-1.7.0", 
        "Successfully installed aws-sdk-appsync-1.2.0", 
        "Successfully installed aws-sdk-applicationautoscaling-1.8.0", 
        "Successfully installed aws-sdk-applicationdiscoveryservice-1.1.0", 
        "Successfully installed aws-sdk-athena-1.0.1", 
        "Successfully installed aws-sdk-autoscaling-1.5.0", 
        "Successfully installed aws-sdk-autoscalingplans-1.2.0", 
        "Successfully installed aws-sdk-batch-1.4.0", 
        "Successfully installed aws-sdk-budgets-1.6.0", 
        "Successfully installed aws-sdk-cloud9-1.1.0", 
        "Successfully installed aws-sdk-clouddirectory-1.2.0", 
        "Successfully installed aws-sdk-cloudformation-1.4.0", 
        "Successfully installed aws-sdk-cloudfront-1.2.0", 
        "Successfully installed aws-sdk-cloudhsm-1.3.0", 
        "Successfully installed aws-sdk-cloudhsmv2-1.1.0", 
        "Successfully installed aws-sdk-cloudsearch-1.0.1", 
        "Successfully installed aws-sdk-cloudsearchdomain-1.0.1", 
        "Successfully installed aws-sdk-cloudtrail-1.0.1", 
        "Successfully installed aws-sdk-cloudwatch-1.5.0", 
        "Successfully installed aws-sdk-cloudwatchevents-1.3.0", 
        "Successfully installed aws-sdk-codebuild-1.7.0", 
        "Successfully installed aws-sdk-codecommit-1.3.0", 
        "Successfully installed aws-sdk-codedeploy-1.4.0", 
        "Successfully installed aws-sdk-codepipeline-1.3.0", 
        "Successfully installed aws-sdk-codestar-1.1.0", 
        "Successfully installed aws-sdk-cognitoidentity-1.0.1", 
        "Successfully installed aws-sdk-cognitoidentityprovider-1.3.0", 
        "Successfully installed aws-sdk-cognitosync-1.0.1", 
        "Successfully installed aws-sdk-comprehend-1.0.0", 
        "Successfully installed aws-sdk-configservice-1.9.0", 
        "Successfully installed aws-sdk-connect-1.0.0", 
        "Successfully installed aws-sdk-costexplorer-1.3.0", 
        "Successfully installed aws-sdk-costandusagereportservice-1.0.1", 
        "Successfully installed aws-sdk-dax-1.0.1", 
        "Successfully installed aws-sdk-datapipeline-1.0.1", 
        "Successfully installed aws-sdk-databasemigrationservice-1.5.0", 
        "Successfully installed aws-sdk-devicefarm-1.5.0", 
        "Successfully installed aws-sdk-directconnect-1.1.0", 
        "Successfully installed aws-sdk-directoryservice-1.1.0", 
        "Successfully installed aws-sdk-dynamodb-1.6.0", 
        "Successfully installed aws-sdk-dynamodbstreams-1.0.1", 
        "Successfully installed aws-sdk-ec2-1.32.0", 
        "Successfully installed aws-sdk-ecr-1.3.0", 
        "Successfully installed aws-sdk-ecs-1.12.0", 
        "Successfully installed aws-sdk-efs-1.0.1", 
        "Successfully installed aws-sdk-emr-1.1.0", 
        "Successfully installed aws-sdk-elasticache-1.3.0", 
        "Successfully installed aws-sdk-elasticbeanstalk-1.6.0", 
        "Successfully installed aws-sdk-elasticloadbalancing-1.2.0", 
        "Successfully installed aws-sdk-elasticloadbalancingv2-1.8.0", 
        "Successfully installed aws-sdk-elastictranscoder-1.0.1", 
        "Successfully installed aws-sdk-elasticsearchservice-1.5.0", 
        "Successfully installed aws-sdk-fms-1.0.0", 
        "Successfully installed aws-sdk-firehose-1.2.0", 
        "Successfully installed aws-sdk-gamelift-1.3.0", 
        "Successfully installed aws-sdk-glacier-1.7.0", 
        "Successfully installed aws-sdk-glue-1.7.0", 
        "Successfully installed aws-sdk-greengrass-1.3.0", 
        "Successfully installed aws-sdk-guardduty-1.4.0", 
        "Successfully installed aws-sdk-health-1.0.1", 
        "Successfully installed aws-sdk-iam-1.4.0", 
        "Successfully installed aws-sigv2-1.0.1", 
        "Successfully installed aws-sdk-importexport-1.0.1", 
        "Successfully installed aws-sdk-inspector-1.3.0", 
        "Successfully installed aws-sdk-iot-1.5.0", 
        "Successfully installed aws-sdk-iotanalytics-1.0.0", 
        "Successfully installed aws-sdk-iotdataplane-1.0.1", 
        "Successfully installed aws-sdk-iotjobsdataplane-1.0.0", 
        "Successfully installed aws-sdk-kinesis-1.2.0", 
        "Successfully installed aws-sdk-kinesisanalytics-1.2.0", 
        "Successfully installed aws-sdk-kinesisvideo-1.0.0", 
        "Successfully installed aws-sdk-kinesisvideoarchivedmedia-1.0.0", 
        "Successfully installed aws-sdk-kinesisvideomedia-1.0.0", 
        "Successfully installed aws-sdk-lambda-1.5.0", 
        "Successfully installed aws-sdk-lambdapreview-1.0.1", 
        "Successfully installed aws-sdk-lex-1.3.0", 
        "Successfully installed aws-sdk-lexmodelbuildingservice-1.5.0", 
        "Successfully installed aws-sdk-lightsail-1.4.0", 
        "Successfully installed aws-sdk-mq-1.0.0", 
        "Successfully installed aws-sdk-mturk-1.3.0", 
        "Successfully installed aws-sdk-machinelearning-1.0.1", 
        "Successfully installed aws-sdk-marketplacecommerceanalytics-1.0.1", 
        "Successfully installed aws-sdk-marketplaceentitlementservice-1.0.1", 
        "Successfully installed aws-sdk-marketplacemetering-1.0.1", 
        "Successfully installed aws-sdk-mediaconvert-1.1.0", 
        "Successfully installed aws-sdk-medialive-1.5.0", 
        "Successfully installed aws-sdk-mediapackage-1.1.0", 
        "Successfully installed aws-sdk-mediastore-1.1.0", 
        "Successfully installed aws-sdk-mediastoredata-1.1.0", 
        "Successfully installed aws-sdk-migrationhub-1.1.0", 
        "Successfully installed aws-sdk-mobile-1.0.0", 
        "Successfully installed aws-sdk-opsworks-1.2.0", 
        "Successfully installed aws-sdk-opsworkscm-1.2.0", 
        "Successfully installed aws-sdk-organizations-1.8.0", 
        "Successfully installed aws-sdk-pinpoint-1.4.0", 
        "Successfully installed aws-sdk-polly-1.4.0", 
        "Successfully installed aws-sdk-pricing-1.0.0", 
        "Successfully installed aws-sdk-rds-1.17.0", 
        "Successfully installed aws-sdk-redshift-1.2.0", 
        "Successfully installed aws-sdk-rekognition-1.3.0", 
        "Successfully installed aws-sdk-resourcegroups-1.0.0", 
        "Successfully installed aws-sdk-resourcegroupstaggingapi-1.0.1", 
        "Successfully installed aws-sdk-route53-1.9.0", 
        "Successfully installed aws-sdk-route53domains-1.2.0", 
        "Successfully installed aws-sdk-ses-1.6.0", 
        "Successfully installed aws-sdk-sms-1.0.1", 
        "Successfully installed aws-sdk-sns-1.1.0", 
        "Successfully installed aws-sdk-ssm-1.13.0", 
        "Successfully installed aws-sdk-swf-1.0.1", 
        "Successfully installed aws-sdk-sagemaker-1.8.0", 
        "Successfully installed aws-sdk-sagemakerruntime-1.0.0", 
        "Successfully installed aws-sdk-secretsmanager-1.4.0", 
        "Successfully installed aws-sdk-serverlessapplicationrepository-1.2.0", 
        "Successfully installed aws-sdk-servicecatalog-1.4.0", 
        "Successfully installed aws-sdk-servicediscovery-1.2.0", 
        "Successfully installed aws-sdk-shield-1.1.0", 
        "Successfully installed aws-sdk-simpledb-1.0.1", 
        "Successfully installed aws-sdk-snowball-1.2.0", 
        "Successfully installed aws-sdk-states-1.2.0", 
        "Successfully installed aws-sdk-storagegateway-1.3.0", 
        "Successfully installed aws-sdk-support-1.0.1", 
        "Successfully installed aws-sdk-transcribeservice-1.1.0", 
        "Successfully installed aws-sdk-translate-1.1.0", 
        "Successfully installed aws-sdk-waf-1.4.0", 
        "Successfully installed aws-sdk-wafregional-1.4.0", 
        "Successfully installed aws-sdk-workdocs-1.1.0", 
        "Successfully installed aws-sdk-workmail-1.1.0", 
        "Successfully installed aws-sdk-workspaces-1.2.0", 
        "Successfully installed aws-sdk-xray-1.2.0", 
        "Successfully installed aws-sdk-resources-3.15.0", 
        "Successfully installed aws-sdk-3.0.1", 
        "Successfully installed google-protobuf-3.5.1.2-x86_64-linux", 
        "Successfully installed fluent-plugin-kinesis-2.1.1", 
        "Parsing documentation for aws-sdk-acm-1.7.0", 
        "Installing ri documentation for aws-sdk-acm-1.7.0", 
        "Parsing documentation for aws-sdk-acmpca-1.0.0", 
        "Installing ri documentation for aws-sdk-acmpca-1.0.0", 
        "Parsing documentation for aws-sdk-apigateway-1.10.0", 
        "Installing ri documentation for aws-sdk-apigateway-1.10.0", 
        "Parsing documentation for aws-sdk-alexaforbusiness-1.4.0", 
        "Installing ri documentation for aws-sdk-alexaforbusiness-1.4.0", 
        "Parsing documentation for aws-sdk-appstream-1.7.0", 
        "Installing ri documentation for aws-sdk-appstream-1.7.0", 
        "Parsing documentation for aws-sdk-appsync-1.2.0", 
        "Installing ri documentation for aws-sdk-appsync-1.2.0", 
        "Parsing documentation for aws-sdk-applicationautoscaling-1.8.0", 
        "Installing ri documentation for aws-sdk-applicationautoscaling-1.8.0", 
        "Parsing documentation for aws-sdk-applicationdiscoveryservice-1.1.0", 
        "Installing ri documentation for aws-sdk-applicationdiscoveryservice-1.1.0", 
        "Parsing documentation for aws-sdk-athena-1.0.1", 
        "Installing ri documentation for aws-sdk-athena-1.0.1", 
        "Parsing documentation for aws-sdk-autoscaling-1.5.0", 
        "Installing ri documentation for aws-sdk-autoscaling-1.5.0", 
        "Parsing documentation for aws-sdk-autoscalingplans-1.2.0", 
        "Installing ri documentation for aws-sdk-autoscalingplans-1.2.0", 
        "Parsing documentation for aws-sdk-batch-1.4.0", 
        "Installing ri documentation for aws-sdk-batch-1.4.0", 
        "Parsing documentation for aws-sdk-budgets-1.6.0", 
        "Installing ri documentation for aws-sdk-budgets-1.6.0", 
        "Parsing documentation for aws-sdk-cloud9-1.1.0", 
        "Installing ri documentation for aws-sdk-cloud9-1.1.0", 
        "Parsing documentation for aws-sdk-clouddirectory-1.2.0", 
        "Installing ri documentation for aws-sdk-clouddirectory-1.2.0", 
        "Parsing documentation for aws-sdk-cloudformation-1.4.0", 
        "Installing ri documentation for aws-sdk-cloudformation-1.4.0", 
        "Parsing documentation for aws-sdk-cloudfront-1.2.0", 
        "Installing ri documentation for aws-sdk-cloudfront-1.2.0", 
        "Parsing documentation for aws-sdk-cloudhsm-1.3.0", 
        "Installing ri documentation for aws-sdk-cloudhsm-1.3.0", 
        "Parsing documentation for aws-sdk-cloudhsmv2-1.1.0", 
        "Installing ri documentation for aws-sdk-cloudhsmv2-1.1.0", 
        "Parsing documentation for aws-sdk-cloudsearch-1.0.1", 
        "Installing ri documentation for aws-sdk-cloudsearch-1.0.1", 
        "Parsing documentation for aws-sdk-cloudsearchdomain-1.0.1", 
        "Installing ri documentation for aws-sdk-cloudsearchdomain-1.0.1", 
        "Parsing documentation for aws-sdk-cloudtrail-1.0.1", 
        "Installing ri documentation for aws-sdk-cloudtrail-1.0.1", 
        "Parsing documentation for aws-sdk-cloudwatch-1.5.0", 
        "Installing ri documentation for aws-sdk-cloudwatch-1.5.0", 
        "Parsing documentation for aws-sdk-cloudwatchevents-1.3.0", 
        "Installing ri documentation for aws-sdk-cloudwatchevents-1.3.0", 
        "Parsing documentation for aws-sdk-codebuild-1.7.0", 
        "Installing ri documentation for aws-sdk-codebuild-1.7.0", 
        "Parsing documentation for aws-sdk-codecommit-1.3.0", 
        "Installing ri documentation for aws-sdk-codecommit-1.3.0", 
        "Parsing documentation for aws-sdk-codedeploy-1.4.0", 
        "Installing ri documentation for aws-sdk-codedeploy-1.4.0", 
        "Parsing documentation for aws-sdk-codepipeline-1.3.0", 
        "Installing ri documentation for aws-sdk-codepipeline-1.3.0", 
        "Parsing documentation for aws-sdk-codestar-1.1.0", 
        "Installing ri documentation for aws-sdk-codestar-1.1.0", 
        "Parsing documentation for aws-sdk-cognitoidentity-1.0.1", 
        "Installing ri documentation for aws-sdk-cognitoidentity-1.0.1", 
        "Parsing documentation for aws-sdk-cognitoidentityprovider-1.3.0", 
        "Installing ri documentation for aws-sdk-cognitoidentityprovider-1.3.0", 
        "Parsing documentation for aws-sdk-cognitosync-1.0.1", 
        "Installing ri documentation for aws-sdk-cognitosync-1.0.1", 
        "Parsing documentation for aws-sdk-comprehend-1.0.0", 
        "Installing ri documentation for aws-sdk-comprehend-1.0.0", 
        "Parsing documentation for aws-sdk-configservice-1.9.0", 
        "Installing ri documentation for aws-sdk-configservice-1.9.0", 
        "Parsing documentation for aws-sdk-connect-1.0.0", 
        "Installing ri documentation for aws-sdk-connect-1.0.0", 
        "Parsing documentation for aws-sdk-costexplorer-1.3.0", 
        "Installing ri documentation for aws-sdk-costexplorer-1.3.0", 
        "Parsing documentation for aws-sdk-costandusagereportservice-1.0.1", 
        "Installing ri documentation for aws-sdk-costandusagereportservice-1.0.1", 
        "Parsing documentation for aws-sdk-dax-1.0.1", 
        "Installing ri documentation for aws-sdk-dax-1.0.1", 
        "Parsing documentation for aws-sdk-datapipeline-1.0.1", 
        "Installing ri documentation for aws-sdk-datapipeline-1.0.1", 
        "Parsing documentation for aws-sdk-databasemigrationservice-1.5.0", 
        "Installing ri documentation for aws-sdk-databasemigrationservice-1.5.0", 
        "Parsing documentation for aws-sdk-devicefarm-1.5.0", 
        "Installing ri documentation for aws-sdk-devicefarm-1.5.0", 
        "Parsing documentation for aws-sdk-directconnect-1.1.0", 
        "Installing ri documentation for aws-sdk-directconnect-1.1.0", 
        "Parsing documentation for aws-sdk-directoryservice-1.1.0", 
        "Installing ri documentation for aws-sdk-directoryservice-1.1.0", 
        "Parsing documentation for aws-sdk-dynamodb-1.6.0", 
        "Installing ri documentation for aws-sdk-dynamodb-1.6.0", 
        "Parsing documentation for aws-sdk-dynamodbstreams-1.0.1", 
        "Installing ri documentation for aws-sdk-dynamodbstreams-1.0.1", 
        "Parsing documentation for aws-sdk-ec2-1.32.0", 
        "Installing ri documentation for aws-sdk-ec2-1.32.0", 
        "Parsing documentation for aws-sdk-ecr-1.3.0", 
        "Installing ri documentation for aws-sdk-ecr-1.3.0", 
        "Parsing documentation for aws-sdk-ecs-1.12.0", 
        "Installing ri documentation for aws-sdk-ecs-1.12.0", 
        "Parsing documentation for aws-sdk-efs-1.0.1", 
        "Installing ri documentation for aws-sdk-efs-1.0.1", 
        "Parsing documentation for aws-sdk-emr-1.1.0", 
        "Installing ri documentation for aws-sdk-emr-1.1.0", 
        "Parsing documentation for aws-sdk-elasticache-1.3.0", 
        "Installing ri documentation for aws-sdk-elasticache-1.3.0", 
        "Parsing documentation for aws-sdk-elasticbeanstalk-1.6.0", 
        "Installing ri documentation for aws-sdk-elasticbeanstalk-1.6.0", 
        "Parsing documentation for aws-sdk-elasticloadbalancing-1.2.0", 
        "Installing ri documentation for aws-sdk-elasticloadbalancing-1.2.0", 
        "Parsing documentation for aws-sdk-elasticloadbalancingv2-1.8.0", 
        "Installing ri documentation for aws-sdk-elasticloadbalancingv2-1.8.0", 
        "Parsing documentation for aws-sdk-elastictranscoder-1.0.1", 
        "Installing ri documentation for aws-sdk-elastictranscoder-1.0.1", 
        "Parsing documentation for aws-sdk-elasticsearchservice-1.5.0", 
        "Installing ri documentation for aws-sdk-elasticsearchservice-1.5.0", 
        "Parsing documentation for aws-sdk-fms-1.0.0", 
        "Installing ri documentation for aws-sdk-fms-1.0.0", 
        "Parsing documentation for aws-sdk-firehose-1.2.0", 
        "Installing ri documentation for aws-sdk-firehose-1.2.0", 
        "Parsing documentation for aws-sdk-gamelift-1.3.0", 
        "Installing ri documentation for aws-sdk-gamelift-1.3.0", 
        "Parsing documentation for aws-sdk-glacier-1.7.0", 
        "Installing ri documentation for aws-sdk-glacier-1.7.0", 
        "Parsing documentation for aws-sdk-glue-1.7.0", 
        "Installing ri documentation for aws-sdk-glue-1.7.0", 
        "Parsing documentation for aws-sdk-greengrass-1.3.0", 
        "Installing ri documentation for aws-sdk-greengrass-1.3.0", 
        "Parsing documentation for aws-sdk-guardduty-1.4.0", 
        "Installing ri documentation for aws-sdk-guardduty-1.4.0", 
        "Parsing documentation for aws-sdk-health-1.0.1", 
        "Installing ri documentation for aws-sdk-health-1.0.1", 
        "Parsing documentation for aws-sdk-iam-1.4.0", 
        "Installing ri documentation for aws-sdk-iam-1.4.0", 
        "Parsing documentation for aws-sigv2-1.0.1", 
        "Installing ri documentation for aws-sigv2-1.0.1", 
        "Parsing documentation for aws-sdk-importexport-1.0.1", 
        "Installing ri documentation for aws-sdk-importexport-1.0.1", 
        "Parsing documentation for aws-sdk-inspector-1.3.0", 
        "Installing ri documentation for aws-sdk-inspector-1.3.0", 
        "Parsing documentation for aws-sdk-iot-1.5.0", 
        "Installing ri documentation for aws-sdk-iot-1.5.0", 
        "Parsing documentation for aws-sdk-iotanalytics-1.0.0", 
        "Installing ri documentation for aws-sdk-iotanalytics-1.0.0", 
        "Parsing documentation for aws-sdk-iotdataplane-1.0.1", 
        "Installing ri documentation for aws-sdk-iotdataplane-1.0.1", 
        "Parsing documentation for aws-sdk-iotjobsdataplane-1.0.0", 
        "Installing ri documentation for aws-sdk-iotjobsdataplane-1.0.0", 
        "Parsing documentation for aws-sdk-kinesis-1.2.0", 
        "Installing ri documentation for aws-sdk-kinesis-1.2.0", 
        "Parsing documentation for aws-sdk-kinesisanalytics-1.2.0", 
        "Installing ri documentation for aws-sdk-kinesisanalytics-1.2.0", 
        "Parsing documentation for aws-sdk-kinesisvideo-1.0.0", 
        "Installing ri documentation for aws-sdk-kinesisvideo-1.0.0", 
        "Parsing documentation for aws-sdk-kinesisvideoarchivedmedia-1.0.0", 
        "Installing ri documentation for aws-sdk-kinesisvideoarchivedmedia-1.0.0", 
        "Parsing documentation for aws-sdk-kinesisvideomedia-1.0.0", 
        "Installing ri documentation for aws-sdk-kinesisvideomedia-1.0.0", 
        "Parsing documentation for aws-sdk-lambda-1.5.0", 
        "Installing ri documentation for aws-sdk-lambda-1.5.0", 
        "Parsing documentation for aws-sdk-lambdapreview-1.0.1", 
        "Installing ri documentation for aws-sdk-lambdapreview-1.0.1", 
        "Parsing documentation for aws-sdk-lex-1.3.0", 
        "Installing ri documentation for aws-sdk-lex-1.3.0", 
        "Parsing documentation for aws-sdk-lexmodelbuildingservice-1.5.0", 
        "Installing ri documentation for aws-sdk-lexmodelbuildingservice-1.5.0", 
        "Parsing documentation for aws-sdk-lightsail-1.4.0", 
        "Installing ri documentation for aws-sdk-lightsail-1.4.0", 
        "Parsing documentation for aws-sdk-mq-1.0.0", 
        "Installing ri documentation for aws-sdk-mq-1.0.0", 
        "Parsing documentation for aws-sdk-mturk-1.3.0", 
        "Installing ri documentation for aws-sdk-mturk-1.3.0", 
        "Parsing documentation for aws-sdk-machinelearning-1.0.1", 
        "Installing ri documentation for aws-sdk-machinelearning-1.0.1", 
        "Parsing documentation for aws-sdk-marketplacecommerceanalytics-1.0.1", 
        "Installing ri documentation for aws-sdk-marketplacecommerceanalytics-1.0.1", 
        "Parsing documentation for aws-sdk-marketplaceentitlementservice-1.0.1", 
        "Installing ri documentation for aws-sdk-marketplaceentitlementservice-1.0.1", 
        "Parsing documentation for aws-sdk-marketplacemetering-1.0.1", 
        "Installing ri documentation for aws-sdk-marketplacemetering-1.0.1", 
        "Parsing documentation for aws-sdk-mediaconvert-1.1.0", 
        "Installing ri documentation for aws-sdk-mediaconvert-1.1.0", 
        "Parsing documentation for aws-sdk-medialive-1.5.0", 
        "Installing ri documentation for aws-sdk-medialive-1.5.0", 
        "Parsing documentation for aws-sdk-mediapackage-1.1.0", 
        "Installing ri documentation for aws-sdk-mediapackage-1.1.0", 
        "Parsing documentation for aws-sdk-mediastore-1.1.0", 
        "Installing ri documentation for aws-sdk-mediastore-1.1.0", 
        "Parsing documentation for aws-sdk-mediastoredata-1.1.0", 
        "Installing ri documentation for aws-sdk-mediastoredata-1.1.0", 
        "Parsing documentation for aws-sdk-migrationhub-1.1.0", 
        "Installing ri documentation for aws-sdk-migrationhub-1.1.0", 
        "Parsing documentation for aws-sdk-mobile-1.0.0", 
        "Installing ri documentation for aws-sdk-mobile-1.0.0", 
        "Parsing documentation for aws-sdk-opsworks-1.2.0", 
        "Installing ri documentation for aws-sdk-opsworks-1.2.0", 
        "Parsing documentation for aws-sdk-opsworkscm-1.2.0", 
        "Installing ri documentation for aws-sdk-opsworkscm-1.2.0", 
        "Parsing documentation for aws-sdk-organizations-1.8.0", 
        "Installing ri documentation for aws-sdk-organizations-1.8.0", 
        "Parsing documentation for aws-sdk-pinpoint-1.4.0", 
        "Installing ri documentation for aws-sdk-pinpoint-1.4.0", 
        "Parsing documentation for aws-sdk-polly-1.4.0", 
        "Installing ri documentation for aws-sdk-polly-1.4.0", 
        "Parsing documentation for aws-sdk-pricing-1.0.0", 
        "Installing ri documentation for aws-sdk-pricing-1.0.0", 
        "Parsing documentation for aws-sdk-rds-1.17.0", 
        "Installing ri documentation for aws-sdk-rds-1.17.0", 
        "Parsing documentation for aws-sdk-redshift-1.2.0", 
        "Installing ri documentation for aws-sdk-redshift-1.2.0", 
        "Parsing documentation for aws-sdk-rekognition-1.3.0", 
        "Installing ri documentation for aws-sdk-rekognition-1.3.0", 
        "Parsing documentation for aws-sdk-resourcegroups-1.0.0", 
        "Installing ri documentation for aws-sdk-resourcegroups-1.0.0", 
        "Parsing documentation for aws-sdk-resourcegroupstaggingapi-1.0.1", 
        "Installing ri documentation for aws-sdk-resourcegroupstaggingapi-1.0.1", 
        "Parsing documentation for aws-sdk-route53-1.9.0", 
        "Installing ri documentation for aws-sdk-route53-1.9.0", 
        "Parsing documentation for aws-sdk-route53domains-1.2.0", 
        "Installing ri documentation for aws-sdk-route53domains-1.2.0", 
        "Parsing documentation for aws-sdk-ses-1.6.0", 
        "Installing ri documentation for aws-sdk-ses-1.6.0", 
        "Parsing documentation for aws-sdk-sms-1.0.1", 
        "Installing ri documentation for aws-sdk-sms-1.0.1", 
        "Parsing documentation for aws-sdk-sns-1.1.0", 
        "Installing ri documentation for aws-sdk-sns-1.1.0", 
        "Parsing documentation for aws-sdk-ssm-1.13.0", 
        "Installing ri documentation for aws-sdk-ssm-1.13.0", 
        "Parsing documentation for aws-sdk-swf-1.0.1", 
        "Installing ri documentation for aws-sdk-swf-1.0.1", 
        "Parsing documentation for aws-sdk-sagemaker-1.8.0", 
        "Installing ri documentation for aws-sdk-sagemaker-1.8.0", 
        "Parsing documentation for aws-sdk-sagemakerruntime-1.0.0", 
        "Installing ri documentation for aws-sdk-sagemakerruntime-1.0.0", 
        "Parsing documentation for aws-sdk-secretsmanager-1.4.0", 
        "Installing ri documentation for aws-sdk-secretsmanager-1.4.0", 
        "Parsing documentation for aws-sdk-serverlessapplicationrepository-1.2.0", 
        "Installing ri documentation for aws-sdk-serverlessapplicationrepository-1.2.0", 
        "Parsing documentation for aws-sdk-servicecatalog-1.4.0", 
        "Installing ri documentation for aws-sdk-servicecatalog-1.4.0", 
        "Parsing documentation for aws-sdk-servicediscovery-1.2.0", 
        "Installing ri documentation for aws-sdk-servicediscovery-1.2.0", 
        "Parsing documentation for aws-sdk-shield-1.1.0", 
        "Installing ri documentation for aws-sdk-shield-1.1.0", 
        "Parsing documentation for aws-sdk-simpledb-1.0.1", 
        "Installing ri documentation for aws-sdk-simpledb-1.0.1", 
        "Parsing documentation for aws-sdk-snowball-1.2.0", 
        "Installing ri documentation for aws-sdk-snowball-1.2.0", 
        "Parsing documentation for aws-sdk-states-1.2.0", 
        "Installing ri documentation for aws-sdk-states-1.2.0", 
        "Parsing documentation for aws-sdk-storagegateway-1.3.0", 
        "Installing ri documentation for aws-sdk-storagegateway-1.3.0", 
        "Parsing documentation for aws-sdk-support-1.0.1", 
        "Installing ri documentation for aws-sdk-support-1.0.1", 
        "Parsing documentation for aws-sdk-transcribeservice-1.1.0", 
        "Installing ri documentation for aws-sdk-transcribeservice-1.1.0", 
        "Parsing documentation for aws-sdk-translate-1.1.0", 
        "Installing ri documentation for aws-sdk-translate-1.1.0", 
        "Parsing documentation for aws-sdk-waf-1.4.0", 
        "Installing ri documentation for aws-sdk-waf-1.4.0", 
        "Parsing documentation for aws-sdk-wafregional-1.4.0", 
        "Installing ri documentation for aws-sdk-wafregional-1.4.0", 
        "Parsing documentation for aws-sdk-workdocs-1.1.0", 
        "Installing ri documentation for aws-sdk-workdocs-1.1.0", 
        "Parsing documentation for aws-sdk-workmail-1.1.0", 
        "Installing ri documentation for aws-sdk-workmail-1.1.0", 
        "Parsing documentation for aws-sdk-workspaces-1.2.0", 
        "Installing ri documentation for aws-sdk-workspaces-1.2.0", 
        "Parsing documentation for aws-sdk-xray-1.2.0", 
        "Installing ri documentation for aws-sdk-xray-1.2.0", 
        "Parsing documentation for aws-sdk-resources-3.15.0", 
        "Installing ri documentation for aws-sdk-resources-3.15.0", 
        "Parsing documentation for aws-sdk-3.0.1", 
        "Installing ri documentation for aws-sdk-3.0.1", 
        "Parsing documentation for google-protobuf-3.5.1.2-x86_64-linux", 
        "Installing ri documentation for google-protobuf-3.5.1.2-x86_64-linux", 
        "Parsing documentation for fluent-plugin-kinesis-2.1.1", 
        "Installing ri documentation for fluent-plugin-kinesis-2.1.1", 
        "Done installing documentation for aws-sdk-acm, aws-sdk-acmpca, aws-sdk-apigateway, aws-sdk-alexaforbusiness, aws-sdk-appstream, aws-sdk-appsync, aws-sdk-applicationautoscaling, aws-sdk-applicationdiscoveryservice, aws-sdk-athena, aws-sdk-autoscaling, aws-sdk-autoscalingplans, aws-sdk-batch, aws-sdk-budgets, aws-sdk-cloud9, aws-sdk-clouddirectory, aws-sdk-cloudformation, aws-sdk-cloudfront, aws-sdk-cloudhsm, aws-sdk-cloudhsmv2, aws-sdk-cloudsearch, aws-sdk-cloudsearchdomain, aws-sdk-cloudtrail, aws-sdk-cloudwatch, aws-sdk-cloudwatchevents, aws-sdk-codebuild, aws-sdk-codecommit, aws-sdk-codedeploy, aws-sdk-codepipeline, aws-sdk-codestar, aws-sdk-cognitoidentity, aws-sdk-cognitoidentityprovider, aws-sdk-cognitosync, aws-sdk-comprehend, aws-sdk-configservice, aws-sdk-connect, aws-sdk-costexplorer, aws-sdk-costandusagereportservice, aws-sdk-dax, aws-sdk-datapipeline, aws-sdk-databasemigrationservice, aws-sdk-devicefarm, aws-sdk-directconnect, aws-sdk-directoryservice, aws-sdk-dynamodb, aws-sdk-dynamodbstreams, aws-sdk-ec2, aws-sdk-ecr, aws-sdk-ecs, aws-sdk-efs, aws-sdk-emr, aws-sdk-elasticache, aws-sdk-elasticbeanstalk, aws-sdk-elasticloadbalancing, aws-sdk-elasticloadbalancingv2, aws-sdk-elastictranscoder, aws-sdk-elasticsearchservice, aws-sdk-fms, aws-sdk-firehose, aws-sdk-gamelift, aws-sdk-glacier, aws-sdk-glue, aws-sdk-greengrass, aws-sdk-guardduty, aws-sdk-health, aws-sdk-iam, aws-sigv2, aws-sdk-importexport, aws-sdk-inspector, aws-sdk-iot, aws-sdk-iotanalytics, aws-sdk-iotdataplane, aws-sdk-iotjobsdataplane, aws-sdk-kinesis, aws-sdk-kinesisanalytics, aws-sdk-kinesisvideo, aws-sdk-kinesisvideoarchivedmedia, aws-sdk-kinesisvideomedia, aws-sdk-lambda, aws-sdk-lambdapreview, aws-sdk-lex, aws-sdk-lexmodelbuildingservice, aws-sdk-lightsail, aws-sdk-mq, aws-sdk-mturk, aws-sdk-machinelearning, aws-sdk-marketplacecommerceanalytics, aws-sdk-marketplaceentitlementservice, aws-sdk-marketplacemetering, aws-sdk-mediaconvert, aws-sdk-medialive, aws-sdk-mediapackage, aws-sdk-mediastore, aws-sdk-mediastoredata, aws-sdk-migrationhub, aws-sdk-mobile, aws-sdk-opsworks, aws-sdk-opsworkscm, aws-sdk-organizations, aws-sdk-pinpoint, aws-sdk-polly, aws-sdk-pricing, aws-sdk-rds, aws-sdk-redshift, aws-sdk-rekognition, aws-sdk-resourcegroups, aws-sdk-resourcegroupstaggingapi, aws-sdk-route53, aws-sdk-route53domains, aws-sdk-ses, aws-sdk-sms, aws-sdk-sns, aws-sdk-ssm, aws-sdk-swf, aws-sdk-sagemaker, aws-sdk-sagemakerruntime, aws-sdk-secretsmanager, aws-sdk-serverlessapplicationrepository, aws-sdk-servicecatalog, aws-sdk-servicediscovery, aws-sdk-shield, aws-sdk-simpledb, aws-sdk-snowball, aws-sdk-states, aws-sdk-storagegateway, aws-sdk-support, aws-sdk-transcribeservice, aws-sdk-translate, aws-sdk-waf, aws-sdk-wafregional, aws-sdk-workdocs, aws-sdk-workmail, aws-sdk-workspaces, aws-sdk-xray, aws-sdk-resources, aws-sdk, google-protobuf, fluent-plugin-kinesis after 391 seconds", 
        "137 gems installed"
darknighthunder commented 6 years ago

@jamesongithub could you try installing the gem without any ruby documentation. It was much faster for me.

td-agent-gem install fluent-plugin-kinesis --no-document

jamesongithub commented 6 years ago

@darknighthunder hm seems to be the same for me

time /usr/sbin/td-agent-gem install fluent-plugin-kinesis --no-document  --backtrace
Successfully installed fluent-plugin-kinesis-2.1.1
1 gem installed
196.05user 28.39system 8:23.90elapsed 44%CPU (0avgtext+0avgdata 353176maxresident)k
240inputs+248outputs (1major+120560minor)pagefaults 0swaps
adammw commented 6 years ago

Should be faster with the 3.0 alpha version since it no longer requires the entire aws sdk just the parts it needs.

jamesongithub commented 6 years ago

great! is the 3.0 version backward compatible?

simukappu commented 6 years ago

Configuration is comparible, but plugin dependency is not compatible. Which version of fluentd or td-agent are you using?

jamesongithub commented 6 years ago

v1.0 (td-agent3)

whatever version install-ubuntu-trusty-td-agent3.sh installs seems to change w/ releases

simukappu commented 6 years ago

Plugin v3.0 supports fluentd v0.14+, so you can try 3.0.0.alpha.2.0 if you have test environment. Please give us feedback if you have any trouble on that. Thanks!

uklg commented 6 years ago

The Alpha plugin release works well with very speedy install. Any timeline for it being a non alpha release? Will the config likely change?

darknighthunder commented 6 years ago

@simukappu I have tested with latest alpha release and it was very fast with installation, is there a plan to make the gem as official release, would be interested to test that.

riywo commented 5 years ago

Today, we've released v3.0.0 in rubygems. https://rubygems.org/gems/fluent-plugin-kinesis/versions/3.0.0