Closed morningman closed 7 months ago
For regression test and performance test, we could follow clickhouse's test method. If it is allowed, I could do this.
Clang compile is already on process, see https://github.com/apache/incubator-doris/pull/7451
Could you please open an email to discuss Roadmap 2022 of Doris ?
支持parquet 文件存储格式也应该加进去吧
希望考虑跨版本升级功能。
What about supporting AVRO format in LOAD function?
Looking forward to push based pipeline engine @morningman-cmy @yiguolei
Doris Manager: 1.Follow-up Doris Manager upgrade 2.User UI interaction improvement 3.Doris Manager supports Doris automated upgrade
我们公司已经有一个回归测试框架。大体是用groovy的dsl去完成测试sql、stream load、安装tpch等功能,大概使用方式如下图。 后续可以提给社区
既然后续有这么多内容,关于社区部分建一个 RFC 目录挺有必要的,大型的 PR 的 design doc 放进去,一方面是为了社区新人的快速融入,另外也减小PR review的压力
既然后续有这么多内容,关于社区部分建一个 RFC 目录挺有必要的,大型的 PR 的 design doc 放进去,一方面是为了社区新人的快速融入,另外也减小PR review的压力
好主意,你是否有一些RFC 模板可供参考?
What about supporting AVRO format in LOAD function?
What about supporting AVRO format in LOAD function?
7650
Thx for opening an issue.
What about supporting AVRO format in LOAD function?
7650
Thx for opening an issue.
Add to the roadmap
support decimal data type for create table as select statement. (detailMessage = Unsupported type 'DECIMAL(9,0)' in create table as select statement)
7680 Data export function supports exporting to db, kafka, etc.
7678 max_by, min_by aggregate function support
Added to the Roadmap
support decimal data type for create table as select statement. (detailMessage = Unsupported type 'DECIMAL(9,0)' in create table as select statement)
Added to the roadmap
Could use vectorized method to optimize load process??
Why Doris need push based query execution engine?
@i7xh Two example:
Provides Schemaless semantics for fast analysis of semi-structured data
Json Parsing Optimization
There are a lot of user cases schema-less or semi-structured that lead to support json optimization especially,mainly with decouple change of the schema?
look forward imperatively
want to know when will vectorized query engine could be released.
When will support data type Decimal(38,18) ?
When will support data type Decimal(38,18) ?
This feature will be released as an experimental feature in version 1.2.0,which will be released at the end of this month.
When will support data type Decimal(38,18) ?
This feature will be released as an experimental feature in version 1.2.0,which will be released at the end of this month.
cool, thanks.
请教一下“Pipeline execution engine”是否可以更方便地实现多线程执行算法,然后大幅提升多核利用率? 拿doris测试过tpc-ds,目前对多核心的利用率貌似比较低
The following is the Roadmap for the Doris community in 2022. The plan includes all aspects of code features, documentation, community building, etc. that are to be developed, have already been developed, and have been completed but require ongoing optimization.
Features
[ ] #7571
[ ] #7572
[ ] #7573
[x] #7574
Provides Schemaless semantics for fast analysis of semi-structured data.
[x] #7575 (Q1)
Supports cold data storage to object storage at partition granularity with remote access capabilities and local Cache acceleration.
[ ] #7503
Doris' current "materialized view" is more of a "materialized index" concept. Doris will later implement a true Materialized View to support full and incremental construction of single and multi-table views.
[x] #7576
Provide Kudu-like data update support.
[x] #7577
[ ] #7578
Support for the new UDF framework has solved the problems of high writing difficulty, poor isolation, and poor compatibility with existing C++ frameworks.
[ ] #7579 (Good First Issue)
[ ] Add more resource limits
[ ] More builtin function support
Performance Optimization
[ ] #7580 (Q1)
[ ] Json Parsing Optimization (Good First Issue)
[x] #7743
Optimize the performance of compaction task. And try to refactor the compaction logic. For example, only one replica do the compaction and sync to other replicas.
Stability and Observability
[x] #7553 (Q1)
Solve the problems of inaccurate memory prediction and OOM, and improve memory observability by global + thread + task level memory management.
[x] #7581
Provides fine-grained IO speed limit, priority scheduling, etc. through global IO management.
[x] #7582
Introduces OpenTelemetry to enhance system internal state observability and unify monitoring data format.
Testing
[ ] #7583
[ ] FE
Refine the FE single test framework to support multi-node simulation testing of features.
[ ] BE
Provide testing framework to simplify the difficulty of writing complex unit tests (e.g. data builds) for BE.
[x] #7584
Provide Case collection or submission framework for refining and accumulating regression test sets.
[x] #7585
Provide a Benchmark testing framework to ensure that adding new code does not impact performance.
[ ] #7586
Implement ChaosMesh chaos testing to improve the correctness and stability of the system in case of anomalies.
Functional Optimization
[ ] #7149
[ ] Lateral View
[ ] Other features
Deployment and Maintenance
[ ] #7591 (Q1)
Provides a visual interface for Doris deployment, monitoring, and operations maintenance. Simplifies Doris deployment, scaling, upgrades, task management, status checking, and other operations.
[ ] #7587 (Good First Issue)
Provides a Helm Charts-based K8S deployment solution.
Peripheral Ecology
[x] #7588
[ ] Parquet File Format Support
Generate and read Parquet files directly to facilitate Doris data to be read directly by external systems.
[ ] Data Integration
[ ] Compilation Tools
Community
[x] #7589 (Q1)
Refactored Doris official website to provide best practices, community progress, blog posts, FAQ, and more.
[ ] Doris Documentation (Good First Issue)
Non-code contributions are as important as code contributions, and the community is very open to developers improving and proofreading the project documentation.
[ ] Github Action (Good First Issue)
Introduced more Github Actions to help improve the management of the code base. This includes but is not limited to PR autoresponders, tagging, etc. If you have a good Action to recommend, please leave a comment.