Tracking: potential concurrent futures

TennyZhuang commented 6 months ago

Found by #14013

Hummock

[ ] compaction_event_loop
- [ ] src/meta/src/hummock/manager/mod.rs:2686:41
- [ ] https://github.com/risingwavelabs/risingwave/blob/18b5a7bc34529be34e4851bdf16af52f75e7399a/src/meta/src/hummock/manager/mod.rs#L2686
[ ] on_handle_check_split_multi_group
[ ] hummock_timer_task
[ ] init_metadata_for_version_replay
[ ] https://github.com/risingwavelabs/risingwave/blob/18b5a7bc34529be34e4851bdf16af52f75e7399a/src/meta/src/hummock/manager/mod.rs#L1806
[ ] for context_id in context_ids {
[ ] commit_epoch_sanity_check
[ ] release_invalid_contexts
[ ] move_state_table_to_compaction_group
[ ] list_compaction_group
[ ] register_table_ids
[ ] validate_ssts
[ ] https://github.com/risingwavelabs/risingwave/blob/18b5a7bc34529be34e4851bdf16af52f75e7399a/src/storage/src/hummock/store/version.rs#L1089
[ ] https://github.com/risingwavelabs/risingwave/blob/18b5a7bc34529be34e4851bdf16af52f75e7399a/src/storage/src/hummock/store/version.rs#L1070
[ ] https://github.com/risingwavelabs/risingwave/blob/18b5a7bc34529be34e4851bdf16af52f75e7399a/src/storage/src/hummock/store/version.rs#L1043
[ ] iter_inner
[ ] HummockVersionReader::get
[ ] flush_imms
[ ] put_sst
[ ] DeleteRangeIterator::seek
[ ] compaction_utils::generate_splits
[ ] UdfExpression::eval

Frontend

[ ] complete_stage
[ ] cancel_all_scheducancled_tasks
[ ] clean_all_stages
[ ] all_children_scheduled
[ ] QueryRunner::run

Stream

[ ] build_actors
[ ] create_nodes_inner
[ ] build_agg_state_storages_from_proto
[ ] many apply_chunk
[ ] StatelessSimpleAggExecutor::apply_chunk
[ ] SourceStateTableHandler::trim_state
[ ] SourceStateTableHandler::take_snapshot
[ ] rebuild_stream_reader_from_error
[ ] SourceExecutor::update_state_if_changed
[ ] get_diff
[ ] replace_with_new_batch_reader
[ ] SimpleAggExecutor::apply_chunk
[ ] map_filter_chunk
[ ] dispatch_barrier
[x] persist_state_per_vnode
[ ] recover_backfill_state
[x] parse_debezium_chunk
[ ] ColumnDeduplicater::dedup
[ ] AggGroup::apply_chunk
[ ] StateTableInner::commit
[ ] LogStoreRowOpStream::init

Batch

[ ] HashShuffleSender::send_done
[ ] HashShuffleSender::send_chunk
[ ] ConsistentHashShuffleSender
[ ] BroadcastSender
[ ] ExecutorBuilder::try_build
[ ] SortAggExecutor::output_agg_states
[ ] update_agg_states
[ ] InsertExecutor::do_execute
[ ] ProjectionOrderBy::get_result

Connector

[ ] KafkaSplitEnumerator::fetch_offset_for_time
[ ] list_splits_batch
[ ] get_watermarks
[ ] await_one_delivery
[ ] PartitionDeltaWriter::flush
[ ] PartitionDeltaWriter::delete
[ ] PartitionDeltaWriter::write

JNI

[ ] HummockJavaBindingIterator::new

Storage

[ ] memtable::flush

Rpc Client

[ ] GrpcMetaClient::try_build_rpc_channel
[ ] refresh_members
[ ] start_heartbeat_loop

hzxa21 commented 6 months ago

@Li0k @Little-Wallace Can you help review the ones in Hummock section? Most of them are related to compaction logic in the meta side. I believe we may not be able to make it concurrent because the existence of compaction/version lock.

github-actions[bot] commented 2 days ago

This issue has been open for 60 days with no activity.

If you think it is still relevant today, and needs to be done in the near future, you can comment to update the status, or just manually remove the no-issue-activity label.

You can also confidently close this issue as not planned to keep our backlog clean. Don't worry if you think the issue is still valuable to continue in the future. It's searchable and can be reopened when it's time. 😄

risingwavelabs / risingwave