happyfish100 / fastdfs

FastDFS is an open source high performance distributed file system (DFS). It's major functions include: file storing, file syncing and file accessing, and design for high capacity and load balance. Wechat/Weixin public account (Chinese Language): fastdfs
GNU General Public License v3.0
8.91k stars 1.97k forks source link

store_path_count大于42时,store_path路径一次自动创建最多创建到42,并且storage启动不成功,这个时候双tracker下第二台的storage启动失败 #697

Open freewind100 opened 4 months ago

freewind100 commented 4 months ago

环境centos7.4, ipv6 双tracker 双storage fastdfs v6.12.1 libfastcommon v1.73 当store_path_count大于42时,当前测试环境为43。 storage启动时仅创建store_path42个,并且启动失败,后续每次执行storage启动命令,仅继续创建一个store_path,并且需要多次执行storage启动第一个storage节点才能正常监听。第二个storage节点启动不成功。具体如下:

第一台: [root@localhost storage]# /etc/init.d/fdfs_trackerd start Starting FastDFS tracker server: [root@localhost storage]# /etc/init.d/fdfs_storaged start Starting FastDFS storage server: [root@localhost storage]# ps -ef|grep fdfs root 6847 1 0 01:57 ? 00:00:00 /usr/local/cemscore/bin/fdfs_trackerd /usr/local/cemscore/etc/fdfs/tracker.conf start root 6882 31047 0 01:58 pts/1 00:00:00 grep --color=auto fdfs [root@localhost logs]# tail 20f storaged.log tail: cannot open ‘20f’ for reading: No such file or directory ==> storaged.log <== mkdir data path: 17 ... mkdir data path: 18 ... mkdir data path: 19 ... mkdir data path: 1A ... mkdir data path: 1B ... mkdir data path: 1C ... mkdir data path: 1D ... mkdir data path: 1E ... mkdir data path: 1F ... data path: /usr/local/FastDFS/data/storage/data42/data, mkdir sub dir done. [root@localhost logs]# /etc/init.d/fdfs_storaged start Starting FastDFS storage server: [root@localhost logs]# tail 20f storaged.log tail: cannot open ‘20f’ for reading: No such file or directory ==> storaged.log <== mkdir data path: 17 ... mkdir data path: 18 ... mkdir data path: 19 ... mkdir data path: 1A ... mkdir data path: 1B ... mkdir data path: 1C ... mkdir data path: 1D ... mkdir data path: 1E ... mkdir data path: 1F ... data path: /usr/local/FastDFS/data/storage/data43/data, mkdir sub dir done. [root@localhost logs]# ps -ef|grep fdfs root 6847 1 0 01:57 ? 00:00:00 /usr/local/cemscore/bin/fdfs_trackerd /usr/local/cemscore/etc/fdfs/tracker.conf start root 6934 31047 0 01:58 pts/1 00:00:00 grep --color=auto fdfs [root@localhost logs]# /etc/init.d/fdfs_storaged start Starting FastDFS storage server: [root@localhost logs]# ps -ef|grep fdfs root 6847 1 0 01:57 ? 00:00:00 /usr/local/cemscore/bin/fdfs_trackerd /usr/local/cemscore/etc/fdfs/tracker.conf start root 7044 31047 0 01:58 pts/1 00:00:00 grep --color=auto fdfs [root@localhost logs]# tail 20f storaged.log tail: cannot open ‘20f’ for reading: No such file or directory ==> storaged.log <== mkdir data path: 1E ... mkdir data path: 1F ... data path: /usr/local/FastDFS/data/storage/data43/data, mkdir sub dir done. [2024-03-01 01:58:52] INFO - FastDFS v6.12.1, base_path=/usr/local/FastDFS/data/storage, max_connections=1024, connect_timeout=30, network_timeout=3600, thread_stack_size=512 KB, buff_size=256 KB, tcp_quick_ack=1, log_level=INFO, run_by_group=root, run_by_user=root, error-log: {sync_log_buff_interval=1, rotate_everyday=1, rotate_time=00:00, rotate_on_size=0, compress_old=0, compress_days_before=1, keep_days=0, delete_old_time=01:30}, port=33000, bind_addr=2001::1121, address_family=auto, accept_threads=4, work_threads=4, store_path_count=43, subdir_count_per_path=32, group_name=g1, client_bind=1, disk_rw_separated=1, disk_reader_threads=1, disk_writer_threads=1, disk_recovery_threads=3, heart_beat_interval=30s, stat_report_interval=60s, tracker_server_count=2, sync_wait_msec=50ms, sync_interval=0ms, sync_start_time=00:00, sync_end_time=23:59, write_mark_file_freq=500, allow_ip_count=-1, file_distribute_path_mode=1, file_distribute_rotate_count=100, fsync_after_written_bytes=0, sync_binlog_buff_interval=1s, sync_stat_file_interval=300s, upload_priority=10, if_alias_prefix=, check_file_duplicate=0, file_signature_method=hash, FDHT group count=0, FDHT server count=0, FDHT key_namespace=, FDHT keep_alive=0, HTTP server port=8888, domain name=, use_access_log=0, rotate_access_log=1, access_log_rotate_time=00:00, compress_old_access_log=1, compress_access_log_days_before=7, rotate_access_log_size=10485760, file_sync_skip_invalid_record=0, use_connection_pool=1, g_connection_pool_max_idle_time=3600s, compress_binlog=1, compress_binlog_time=01:30, check_store_path_mark=1 [2024-03-01 01:58:52] INFO - file: storage_param_getter.c, line: 221, use_storage_id=1, id_type_in_filename=id, trust_storage_server_id=1, storage_ip_changed_auto_adjust=1, store_path=0, reserved_storage_space=20.00%, use_trunk_file=0, slot_min_size=256, slot_max_size=1024 KB, trunk_alloc_alignment_size=256, trunk_file_size=64 MB, trunk_create_file_advance=0, trunk_create_file_time_base=02:00, trunk_create_file_interval=86400, trunk_create_file_space_threshold=20 GB, trunk_init_check_occupying=0, trunk_init_reload_from_binlog=0, trunk_free_space_merge=1, delete_unused_trunk_files=0, trunk_compress_binlog_min_interval=86400, trunk_compress_binlog_interval=86400, trunk_compress_binlog_time_base=03:00, trunk_binlog_max_backups=7, store_slave_file_use_link=0 [2024-03-01 01:58:52] INFO - file: storage_func.c, line: 338, tracker_client_ip: 2001::1121, my_server_id_str: 1, g_server_id_in_filename: 1 [2024-03-01 01:58:52] INFO - file: tracker_client_thread.c, line: 305, successfully connect to tracker server 2001::1122:32122, as a tracker client, my ip is 2001::1121 [2024-03-01 01:58:52] INFO - file: tracker_client_thread.c, line: 2051, tracker server: #0. 2001::1121:32122, my_report_result: -1 [2024-03-01 01:58:52] INFO - file: tracker_client_thread.c, line: 305, successfully connect to tracker server 2001::1121:32122, as a tracker client, my ip is 2001::1121 [2024-03-01 01:58:52] INFO - file: tracker_client_thread.c, line: 2051, tracker server: #0. 2001::1121:32122, my_report_result: -1

第二台: [root@localhost logs]# tail -10f storaged.log mkdir data path: 17 ... mkdir data path: 18 ... mkdir data path: 19 ... mkdir data path: 1A ... mkdir data path: 1B ... mkdir data path: 1C ... mkdir data path: 1D ... mkdir data path: 1E ... mkdir data path: 1F ... data path: /usr/local/FastDFS/data/storage/data42/data, mkdir sub dir done. [root@localhost logs]# ps -ef|grep fdfs root 22475 1 0 01:53 ? 00:00:00 /usr/local/cemscore/bin/fdfs_trackerd /usr/local/cemscore/etc/fdfs/tracker.conf start root 22519 5637 0 01:53 pts/4 00:00:00 grep --color=auto fdfs [root@localhost logs]# /etc/init.d/fdfs_storaged start Starting FastDFS storage server: [root@localhost logs]# ps -ef|grep fdfs root 22475 1 0 01:53 ? 00:00:00 /usr/local/cemscore/bin/fdfs_trackerd /usr/local/cemscore/etc/fdfs/tracker.conf start root 22529 5637 0 01:53 pts/4 00:00:00 grep --color=auto fdfs [root@localhost logs]# tail -10f storaged.log mkdir data path: 17 ... mkdir data path: 18 ... mkdir data path: 19 ... mkdir data path: 1A ... mkdir data path: 1B ... mkdir data path: 1C ... mkdir data path: 1D ... mkdir data path: 1E ... mkdir data path: 1F ... data path: /usr/local/FastDFS/data/storage/data43/data, mkdir sub dir done. [root@localhost logs]# /etc/init.d/fdfs_storaged start Starting FastDFS storage server: [root@localhost logs]# ps -ef|grep fdfs root 22475 1 0 01:53 ? 00:00:00 /usr/local/cemscore/bin/fdfs_trackerd /usr/local/cemscore/etc/fdfs/tracker.conf start root 22556 5637 0 01:54 pts/4 00:00:00 grep --color=auto fdfs [root@localhost logs]# tail -10f storaged.log mkdir data path: 1D ... mkdir data path: 1E ... mkdir data path: 1F ... data path: /usr/local/FastDFS/data/storage/data43/data, mkdir sub dir done. [2024-03-01 01:54:04] INFO - FastDFS v6.12.1, base_path=/usr/local/FastDFS/data/storage, max_connections=1024, connect_timeout=30, network_timeout=3600, thread_stack_size=512 KB, buff_size=256 KB, tcp_quick_ack=1, log_level=INFO, run_by_group=root, run_by_user=root, error-log: {sync_log_buff_interval=1, rotate_everyday=1, rotate_time=00:00, rotate_on_size=0, compress_old=0, compress_days_before=1, keep_days=0, delete_old_time=01:30}, port=33000, bind_addr=2001::1122, address_family=auto, accept_threads=1, work_threads=4, store_path_count=43, subdir_count_per_path=32, group_name=g1, client_bind=1, disk_rw_separated=1, disk_reader_threads=1, disk_writer_threads=1, disk_recovery_threads=3, heart_beat_interval=30s, stat_report_interval=60s, tracker_server_count=2, sync_wait_msec=50ms, sync_interval=0ms, sync_start_time=00:00, sync_end_time=23:59, write_mark_file_freq=500, allow_ip_count=-1, file_distribute_path_mode=1, file_distribute_rotate_count=100, fsync_after_written_bytes=0, sync_binlog_buff_interval=1s, sync_stat_file_interval=300s, upload_priority=10, if_alias_prefix=, check_file_duplicate=0, file_signature_method=hash, FDHT group count=0, FDHT server count=0, FDHT key_namespace=, FDHT keep_alive=0, HTTP server port=8888, domain name=, use_access_log=0, rotate_access_log=1, access_log_rotate_time=00:00, compress_old_access_log=1, compress_access_log_days_before=7, rotate_access_log_size=10485760, file_sync_skip_invalid_record=0, use_connection_pool=1, g_connection_pool_max_idle_time=3600s, compress_binlog=1, compress_binlog_time=01:30, check_store_path_mark=1 [2024-03-01 01:54:04] INFO - file: storage_param_getter.c, line: 221, use_storage_id=1, id_type_in_filename=id, trust_storage_server_id=1, storage_ip_changed_auto_adjust=1, store_path=0, reserved_storage_space=20.00%, use_trunk_file=0, slot_min_size=256, slot_max_size=1024 KB, trunk_alloc_alignment_size=256, trunk_file_size=64 MB, trunk_create_file_advance=0, trunk_create_file_time_base=02:00, trunk_create_file_interval=86400, trunk_create_file_space_threshold=20 GB, trunk_init_check_occupying=0, trunk_init_reload_from_binlog=0, trunk_free_space_merge=1, delete_unused_trunk_files=0, trunk_compress_binlog_min_interval=86400, trunk_compress_binlog_interval=86400, trunk_compress_binlog_time_base=03:00, trunk_binlog_max_backups=7, store_slave_file_use_link=0 [2024-03-01 01:54:04] ERROR - file: storage_func.c, line: 289, config file: /usr/local/cemscore/etc/fdfs/storage.conf, server_id: 0▒▒ is invalid [2024-03-01 01:54:04] CRIT - file: storage_func.c, line: 2161, get my server id from tracker server fail, errno: 22, error info: Invalid argument [2024-03-01 01:54:04] CRIT - exit abnormally!

happyfish100 commented 4 months ago

配置的 server_id 不能有前置0。

freewind100 commented 4 months ago

ipv6下双tracker配置后,server_id问题处理后,2个storage监听正常。但还有下列问题: 服务器1 2001::1188 服务器2 2001::1189 storage_ids.conf配置: image storage.conf配置 image

1.第二台的storage节点不同步第一个节点数据 2.当store_path_count大于42时,storage一次启动不成功。 具体报错如下: [第一个服务器tracker.log image 第一个服务器storage.log image 第二个服务器tracker.log image

第二个服务器storage.log image

happyfish100 commented 4 months ago

今天 review 代码,发现适配 IPv6有一个地方漏掉了,预计明天可以修复。