search

apache / beam

Apache Beam is a unified programming model for Batch and Streaming data processing.
https://beam.apache.org/
Apache License 2.0
7.78k stars 4.22k forks source link

[Task]: Shutdown Jenkins PreCommits #28426

Closed Abacn closed 6 months ago

Abacn commented 1 year ago

What needs to happen?

All precommits are now migrated to GitHub Action, and we agreed start the process of remove the Jenkins counterpart.

Issue Priority

Priority: 2 (default / most normal work should be filed as P2)

Issue Components

Abacn commented 1 year ago

This comment tracks the test suites that are shutdown in Jenkins

Name Phrase Jenkins status GitHub Action status
PreCommit_PythonDocs Run PythonDocs PreCommit Build Status .github/workflows/beam_PreCommit_PythonDocs.yml
PreCommit_PythonLint Run PythonLint PreCommit Build Status .github/workflows/beam_PreCommit_PythonLint.yml
PreCommit_RAT Run RAT PreCommit Build Status .github/workflows/beam_PreCommit_RAT.yml
PreCommit_Spotless Run Spotless PreCommit Build Status .github/workflows/beam_PreCommit_Spotless.yml
PreCommit_Website Run Website PreCommit Build Status .github/workflows/beam_PreCommit_Website.yml
PreCommit_Website_Stage_GCS Run Website_Stage_GCS PreCommit Build Status PreCommit Website Stage GCS
PreCommit_Whitespace Run Whitespace PreCommit Build Status .github/workflows/beam_PreCommit_Whitespace.yml
PreCommit_PythonFormatter Run PythonFormatter PreCommit Build Status .github/workflows/beam_PreCommit_PythonFormatter.yml
PreCommit_Kotlin_Examples Run Kotlin_Examples PreCommit Build Status .github/workflows/beam_PreCommit_Kotlin_Examples.yml
PreCommit_Go Run Go PreCommit Build Status .github/workflows/beam_PreCommit_Go.yml
PreCommit_GoPortable Run GoPortable PreCommit Build Status .github/workflows/beam_PreCommit_GoPortable.yml
PreCommit_TypeScript Run Typescript PreCommit Build Status .github/workflows/beam_PreCommit_Typescript.yml
PreCommit_CommunityMetrics Run CommunityMetrics PreCommit Build Status .github/workflows/beam_PreCommit_CommunityMetrics.yml
PreCommit_Java_Examples_Dataflow Run Java_Examples_Dataflow PreCommit Build Status .github/workflows/beam_PreCommit_Java_Examples_Dataflow.yml
PreCommit_Java_Examples_Dataflow_Java11 Run Java_Examples_Dataflow_Java11 PreCommit Build Status .github/workflows/beam_PreCommit_Java_Examples_Dataflow_Java11.yml
PreCommit_Java_Examples_Dataflow_Java17 Run Java_Examples_Dataflow_Java17 PreCommit Build Status .github/workflows/beam_PreCommit_Java_Examples_Dataflow_Java17.yml
PreCommit_Java_PVR_Flink_Docker Run Java_PVR_Flink_Docker PreCommit Build Status .github/workflows/beam_PreCommit_Java_PVR_Flink_Docker.yml
PreCommit_Java_Spark3_Versions Run Java_Spark3_Versions PreCommit Build Status .github/workflows/beam_PreCommit_Spark3_Versions.yml
PreCommit_Java_Amqp_IO Run Java_Amqp_IO_Direct PreCommit Build Status .github/workflows/beam_PreCommit_Amqp_IO_Direct.yml
PreCommit_Java_Cassandra_IO Run Java_Cassandra_IO_Direct PreCommit Build Status .github/workflows/beam_PreCommit_Cassandra_IO_Direct.yml
PreCommit_Java_Cdap_IO Run Java_Cdap_IO_Direct PreCommit Build Status .github/workflows/beam_PreCommit_Cdap_IO_Direct.yml
PreCommit_Java_Clickhouse_IO Run Java_Clickhouse_IO_Direct PreCommit Build Status .github/workflows/beam_PreCommit_Clickhouse_IO_Direct.yml
PreCommit_Java_Csv_IO Run Java_Csv_IO_Direct PreCommit Build Status .github/workflows/beam_PreCommit_Csv_IO_Direct.yml
PreCommit_Java_Debezium_IO Run Java_Debezium_IO_Direct PreCommit Build Status .github/workflows/beam_PreCommit_Debezium_IO_Direct.yml
PreCommit_Java_ElasticSearch_IO Run Java_ElasticSearch_IO_Direct PreCommit Build Status .github/workflows/beam_PreCommit_ElasticSearch_IO_Direct.yml
PreCommit_Java_File-schema-transform_IO Run Java_File-schema-transform_IO_Direct PreCommit Build Status .github/workflows/beam_PreCommit_File-schema-transform_IO_Direct.yml
PreCommit_Java_HBase_IO Run Java_HBase_IO_Direct PreCommit Build Status .github/workflows/beam_PreCommit_HBase_IO_Direct.yml
PreCommit_Java_HCatalog_IO Run Java_HCatalog_IO_Direct PreCommit Build Status .github/workflows/beam_PreCommit_HCatalog_IO_Direct.yml
PreCommit_Java_InfluxDb_IO Run Java_InfluxDb_IO_Direct PreCommit Build Status .github/workflows/beam_PreCommit_InfluxDb_IO_Direct.yml
PreCommit_Java_JDBC_IO Run Java_JDBC_IO_Direct PreCommit Build Status .github/workflows/beam_PreCommit_JDBC_IO_Direct.yml
PreCommit_Java_Jms_IO Run Java_Jms_IO_Direct PreCommit Build Status .github/workflows/beam_PreCommit_Jms_IO_Direct.yml
PreCommit_Java_Kafka_IO Run Java_Kafka_IO_Direct PreCommit Build Status .github/workflows/beam_PreCommit_Kafka_IO_Direct.yml
PreCommit_Java_Kudu_IO Run Java_Kudu_IO_Direct PreCommit Build Status .github/workflows/beam_PreCommit_Kudu_IO_Direct.yml
PreCommit_Java_MongoDb_IO Run Java_MongoDb_IO_Direct PreCommit Build Status .github/workflows/beam_PreCommit_MongoDb_IO_Direct.yml
PreCommit_Java_Mqtt_IO Run Java_Mqtt_IO_Direct PreCommit Build Status .github/workflows/beam_PreCommit_Mqtt_IO_Direct.yml
PreCommit_Java_Neo4j_IO Run Java_Neo4j_IO_Direct PreCommit Build Status .github/workflows/beam_PreCommit_Neo4j_IO_Direct.yml
PreCommit_Java_Parquet_IO Run Java_Parquet_IO_Direct PreCommit Build Status .github/workflows/beam_PreCommit_Parquet_IO_Direct.yml
PreCommit_Java_RabbitMq_IO Run Java_RabbitMq_IO_Direct PreCommit Build Status .github/workflows/beam_PreCommit_RabbitMq_IO_Direct.yml
PreCommit_Java_Redis_IO Run Java_Redis_IO_Direct PreCommit Build Status .github/workflows/beam_PreCommit_Redis_IO_Direct.yml
PreCommit_Java_SingleStore_IO Run Java_SingleStore_IO_Direct PreCommit Build Status .github/workflows/beam_PreCommit_SingleStore_IO_Direct.yml
PreCommit_Java_Snowflake_IO Run Java_Snowflake_IO_Direct PreCommit Build Status .github/workflows/beam_PreCommit_Snowflake_IO_Direct.yml
PreCommit_Java_Solr_IO Run Java_Solr_IO_Direct PreCommit Build Status .github/workflows/beam_PreCommit_Solr_IO_Direct.yml
PreCommit_Java_Splunk_IO Run Java_Splunk_IO_Direct PreCommit Build Status .github/workflows/beam_PreCommit_Splunk_IO_Direct.yml
PreCommit_Java_Thrift_IO Run Java_Thrift_IO_Direct PreCommit Build Status .github/workflows/beam_PreCommit_Thrift_IO_Direct.yml
PreCommit_Java_Tika_IO Run Java_Tika_IO_Direct PreCommit Build Status .github/workflows/beam_PreCommit_Tika_IO_Direct.yml
PreCommit_Portable_Python Run Portable_Python PreCommit Build Status .github/workflows/beam_PreCommit_Portable_Python.yml
PreCommit_SQL Run SQL PreCommit Build Status .github/workflows/beam_PreCommit_SQL.yml
PreCommit_SQL_Java11 Run SQL_Java11 PreCommit Build Status .github/workflows/beam_PreCommit_SQL_Java11.yml
PreCommit_SQL_Java17 Run SQL_Java17 PreCommit Build Status .github/workflows/beam_PreCommit_SQL_Java17.yml
Abacn commented 1 year ago

This comments tracks the test suites only available on GitHub Action

Name Phrase GitHub status
PostCommit_Go_Dataflow_ARM Run Go PostCommit Dataflow ARM .github/workflows/beam_PostCommit_Go_Dataflow_ARM.yml
PostCommit_Java_Examples_Dataflow Run Java examples on Dataflow .github/workflows/beam_PostCommit_Java_Examples_Dataflow.yml
PostCommit_Java_Examples_Dataflow_ARM Run Java_Examples_Dataflow_ARM PostCommit 8
Run Java_Examples_Dataflow_ARM PostCommit 11
Run Java_Examples_Dataflow_ARM PostCommit 17		 .github/workflows/beam_PostCommit_Java_Examples_Dataflow_ARM.yml
PostCommit_Java_IO_Performance_Tests Run Java PostCommit IO Performance Tests .github/workflows/beam_PostCommit_Java_IO_Performance_Tests.yml
PostCommit_Python_Arm Run Python PostCommit Arm 3.8
Run Python PostCommit Arm 3.9
Run Python PostCommit Arm 3.10
Run Python PostCommit Arm 3.11		 .github/workflows/beam_PostCommit_Python_Arm.yml
PreCommit_GHA Run GHA PreCommit .github/workflows/beam_PreCommit_GHA.yml
PreCommit_ItFramework Run It_Framework PreCommit .github/workflows/beam_PreCommit_ItFramework.yml
PreCommit_Java_Flink_Versions Run Java_Flink_Versions PreCommit .github/workflows/beam_PreCommit_Java_Flink_Versions.yml
PreCommit_Java_RequestResponse_IO_Direct Run Java_RequestResponse_IO_Direct PreCommit .github/workflows/beam_PreCommit_Java_RequestResponse_IO_Direct.yml
Python_ValidatesContainer_Dataflow_ARM Run Python ValidatesContainer Dataflow ARM 3.8
Run Python ValidatesContainer Dataflow ARM 3.9
Run Python ValidatesContainer Dataflow ARM 3.10
Run Python ValidatesContainer Dataflow ARM 3.11		 .github/workflows/beam_Python_ValidatesContainer_Dataflow_ARM.yml
Abacn commented 11 months ago

Current Performance tests:

Name Phrase Jenkins status GitHub Action status
PerformanceTests_AvroIOIT - Build Status .github/workflows/beam_PerformanceTests_AvroIOIT.yml
PerformanceTests_BigQueryIO_Batch_Java_Avro - Build Status .github/workflows/beam_PerformanceTests_BigQueryIO_Batch_Java_Avro.yml
PerformanceTests_BigQueryIO_Batch_Java_Json - Build Status .github/workflows/beam_PerformanceTests_BigQueryIO_Batch_Java_Json.yml
PerformanceTests_BigQueryIO_Streaming_Java - Build Status .github/workflows/beam_PerformanceTests_BigQueryIO_Streaming_Java.yml
PerformanceTests_BiqQueryIO_Read_Python - Build Status .github/workflows/beam_PerformanceTests_BiqQueryIO_Read_Python.yml
PerformanceTests_BiqQueryIO_Write_Python_Batch - Build Status .github/workflows/beam_PerformanceTests_BiqQueryIO_Write_Python_Batch.yml
PerformanceTests_Cdap - Build Status .github/workflows/beam_PerformanceTests_Cdap.yml
PerformanceTests_Compressed_TextIOIT - Build Status .github/workflows/beam_PerformanceTests_Compressed_TextIOIT.yml
PerformanceTests_HadoopFormat - Build Status .github/workflows/beam_PerformanceTests_HadoopFormat.yml
PerformanceTests_JDBC - Build Status .github/workflows/beam_PerformanceTests_JDBC.yml
PerformanceTests_Kafka_IO - Build Status .github/workflows/beam_PerformanceTests_Kafka_IO.yml
PerformanceTests_ManyFiles_TextIOIT - Build Status .github/workflows/beam_PerformanceTests_ManyFiles_TextIOIT.yml
PerformanceTests_ManyFiles_TextIOIT_HDFS - Build Status .github/workflows/beam_PerformanceTests_ManyFiles_TextIOIT_HDFS.yml
PerformanceTests_MongoDBIO_IT - Build Status .github/workflows/beam_PerformanceTests_MongoDBIO_IT.yml
PerformanceTests_ParquetIOIT - Build Status .github/workflows/beam_PerformanceTests_ParquetIOIT.yml
PerformanceTests_ParquetIOIT_HDFS - Build Status .github/workflows/beam_PerformanceTests_ParquetIOIT_HDFS.yml
PerformanceTests_PubsubIOIT_Python_Streaming - Build Status .github/workflows/beam_PerformanceTests_PubsubIOIT_Python_Streaming.yml
PerformanceTests_SQLBigQueryIO_Batch_Java - Build Status .github/workflows/beam_PerformanceTests_SQLBigQueryIO_Batch_Java.yml
PerformanceTests_SpannerIO_Read_2GB_Python - Build Status .github/workflows/beam_PerformanceTests_SpannerIO_Read_2GB_Python.yml
PerformanceTests_SpannerIO_Write_2GB_Python_Batch - Build Status .github/workflows/beam_PerformanceTests_SpannerIO_Write_2GB_Python_Batch.yml
PerformanceTests_SparkReceiver_IO - Build Status .github/workflows/beam_PerformanceTests_SparkReceiver_IO.yml
PerformanceTests_TFRecordIOIT - Build Status .github/workflows/beam_PerformanceTests_TFRecordIOIT.yml
PerformanceTests_TFRecordIOIT_HDFS - N/A .github/workflows/beam_PerformanceTests_TFRecordIOIT_HDFS.yml
PerformanceTests_TextIOIT - Build Status .github/workflows/beam_PerformanceTests_TextIOIT.yml
PerformanceTests_TextIOIT_HDFS - Build Status .github/workflows/beam_PerformanceTests_TextIOIT_HDFS.yml
PerformanceTests_TextIOIT_Python - Build Status .github/workflows/beam_PerformanceTests_TextIOIT_Python.yml
PerformanceTests_WordCountIT_PythonVersions - N/A .github/workflows/beam_PerformanceTests_WordCountIT_PythonVersions.yml
PerformanceTests_XmlIOIT - Build Status .github/workflows/beam_PerformanceTests_XmlIOIT.yml
PerformanceTests_XmlIOIT_HDFS - Build Status .github/workflows/beam_PerformanceTests_XmlIOIT_HDFS.yml
PerformanceTests_xlang_KafkaIO_Python - Build Status .github/workflows/beam_PerformanceTests_HadoopFormat.yml

Flaky:

Name Phrase Jenkins status GitHub Action status
PerformanceTests_AvroIOIT_HDFS - Build Status .github/workflows/beam_PerformanceTests_AvroIOIT_HDFS.yml
PerformanceTests_Compressed_TextIOIT_HDFS - Build Status .github/workflows/beam_PerformanceTests_Compressed_TextIOIT_HDFS.yml