armandcismaru
commented
3 years ago I selected a few ETL Tools that are popular in the market right now. Listed few features and specific details about each one. (more to be added)
1. AWS Glue | Product documentation
- Automatic schema discovery
- This ETL tool automatically generates the code to extract, transform, and load your data.
- AWS Glue jobs allow you to invoke on a schedule, on-demand, or based on a specific event.
- Serverless. There is no infrastructure to provision or manage.
- AWS Glue will generate Apache Spark ETL code in Scala or Python.
-
Pricing: -$0.44 per DPU-Hour, billed per second, with a 1-minute minimum (Glue version 2.0) or 10-minute minimum (Glue version 0.9/1.0) for each ETL job of type Apache Spark. -$0.44 per DPU-Hour, billed per second, with a 1-minute minimum for each ETL job of type Python shell. -$0.44 per DPU-Hour, billed per second, with a 10-minute minimum for each provisioned development endpoint.
2. Informatica PowerCenter | Product documentation
- It has a centralized error logging system which facilitates logging errors and rejecting data into relational tables
- Build-in Intelligence to improve performance
- Limit the Session Log
- Ability to Scale-up Data Integration
- Foundation for Data Architecture Modernization
- Better designs with enforced best practices on code development
- Code integration with external Software Configuration tools
- Synchronization amongst geographically distributed team members.
- Supports AWS and Microsoft Azure
- Web-Based, Cloud, SaaS deployment
- Pricing: Starting from $3.50/hr or from $24,528/yr
3. Talend Open Studio for Data Integration | Product documentation
- It is the first commercial open source software vendor for data integration.
- Over 900 inbuilt components for connecting various data sources.
- Drag and drop interface.
- Improves the productivity and time required for deployment are using GUI and inbuilt components.
- Easily deployable in a cloud environment.
- Data can be merged and transforms traditional and Big Data into Talend Open Studio.
- The online user community is available for any technical support.
- Pricing: Talend is a free open source ETL tool.
4. Fivetran | Product documentation
- Helps you to build robust, automated pipelines with standardized schemas.
- Adding new data sources as fast as you need.
- No training or custom coding required.
- Support for BigQuery, Snowflake, Azure, Redshift, etc.
- Access to all your data in SQL.
- Complete replication by default.
- Automated connectors for Database sources
- API Access for programmatic management and access of Fivetran
- Pricing: $1.50 / Credit, only pay or data that is new or changed in any month.
joekendal
Investigate the features, pricing and dependencies. Preferred solutions are serverless. Read some documentation as well to see how they work in deployment.