search

iovisor / bcc

BCC - Tools for BPF-based Linux IO analysis, networking, monitoring, and more
Apache License 2.0
20.31k stars 3.85k forks source link

why not trace mysql function #1640

Open kevinbin opened 6 years ago

kevinbin commented 6 years ago

[root@hadoop03 tools]# ./trace -U -p 26720 '/usr/local/mysql/bin/mysqld:log_buffer_flush_to_disk' could not determine address of symbol log_buffer_flush_to_disk

[root@hadoop03 ~]# objdump -t /usr/local/mysql/bin/mysqld|grep -i log_buffer_flush_to_disk 00000000019523a5 g F .text 0000000000000057 _Z24log_buffer_flush_to_diskb

[root@hadoop03 ~]# gdb -p 26720 ... (gdb) b log_buffer_flush_to_disk Breakpoint 1 at 0x19523b3: file /root/mysql-5.7.21/storage/innobase/log/log0log.cc, line 1417.

[root@hadoop03 tools]# uname -sr Linux 4.15.9-1.el7.elrepo.x86_64

Is there a problem with my usage?

goldshtn commented 6 years ago

Can you please try using the mangled name, _Z24log_buffer_flush_to_diskb?

On Wed, Mar 21, 2018 at 08:27 hongbin notifications@github.com wrote:

[root@hadoop03 tools]# ./trace -U -p 26720 '/usr/local/mysql/bin/mysqld:log_buffer_flush_to_disk' could not determine address of symbol log_buffer_flush_to_disk

[root@hadoop03 ~]# objdump -t /usr/local/mysql/bin/mysqld|grep -i log_buffer_flush_to_disk 00000000019523a5 g F .text 0000000000000057 _Z24log_buffer_flush_to_diskb

[root@hadoop03 ~]# gdb -p 26720 ... (gdb) b log_buffer_flush_to_disk Breakpoint 1 at 0x19523b3: file /root/mysql-5.7.21/storage/innobase/log/log0log.cc, line 1417.

[root@hadoop03 tools]# uname -sr Linux 4.15.9-1.el7.elrepo.x86_64

Is there a problem with my usage?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/iovisor/bcc/issues/1640, or mute the thread https://github.com/notifications/unsubscribe-auth/ABm3vb7MYdWV6LSAb0wqiYkHCqpyOUDpks5tgfLtgaJpZM4SzEIv .

-- Sasha Goldshtein | CTO, Sela Group | Microsoft C# MVP | blog.sashag.net | @goldshtn http://twitter.com/goldshtn

kevinbin commented 6 years ago

It's OK, but not friendly

[root@hadoop03 tools]./trace -U -p 26720 '/usr/local/mysql/bin/mysqld:_Z24log_buffer_flush_to_diskb' PID TID COMM FUNC 26720 230362 mysqld _Z24log_buffer_flush_to_diskb log_buffer_flush_to_disk(bool)+0x0 [mysqld] handler::ha_create(char const, TABLE, st_ha_create_information)+0x74 [mysqld] ha_create_table(THD, char const, char const, char const, st_ha_create_information, bool, bool)+0x241 [mysqld] rea_create_table(THD, char const, char const, char const, st_ha_create_information, List&, unsigned int, st_key, handler, bool)+0x1dd [mysqld] create_table_impl(THD, char const, char const, char const, char const, st_ha_create_information, Alter_info, bool, unsigned int, bool, bool*, st_key*, unsigned int)+0x1498 [mysqld] mysql_create_table_no_lock(THD, char const, char const, st_ha_create_information, Alter_info, unsigned int, bool)+0x17c [mysqld] mysql_create_table(THD, TABLE_LIST, st_ha_create_information, Alter_info)+0xf4 [mysqld] mysql_execute_command(THD, bool)+0x1de4 [mysqld] mysql_parse(THD, Parser_state)+0x5fc [mysqld] dispatch_command(THD, COM_DATA const, enum_server_command)+0xca9 [mysqld] do_command(THD)+0x4b2 [mysqld] handle_connection+0x1e0 [mysqld] pfs_spawn_thread+0x170 [mysqld] start_thread+0xc5 [libpthread-2.17.so]

[root@hadoop03 tools]# ./trace -U -p 26720 '_Z24log_buffer_flush_to_diskb' Failed to attach BPF to kprobe [root@hadoop03 tools]# ./trace -U -p 26720 '_Z24log_buffer_flush_to_diskb' check dmesg output for possible cause Failed to attach BPF to kprobe

goldshtn commented 6 years ago

Without specifying the module, we would have to magically guess that you mean a user function and not a kernel function. Even if there was a switch to specify it, if we don't know the module name we'd have to look at all the modules loaded into the process and search all the symbols -- I think it's probably not the best idea.

Re figuring out the mangled name automatically, it's also a bit hard, because the mangled name is produced from the full signature. I.e. "f(int, int)" would have a different mangled name than "f(double)" etc. -- so we'd have to get the full signature from the user in order to figure out the mangled name. I think the wildcard-based approach is probably friendlier -- we don't have support for it in the 'trace' tool yet, but we have it implemented in funccount, stackcount and others, and it probably makes sense to add here as well.

On Wed, Mar 21, 2018 at 9:01 AM, hongbin notifications@github.com wrote:

It's OK, but not friendly

[root@hadoop03 tools]./trace -U -p 26720 '/usr/local/mysql/bin/mysqld:_ Z24log_buffer_flush_to_diskb' PID TID COMM FUNC 26720 230362 mysqld _Z24log_buffer_flush_to_diskb log_buffer_flush_to_disk(bool)+0x0 [mysqld] handler::ha_create(char const, TABLE, st_ha_create_information)+0x74 [mysqld] ha_create_table(THD, char const, char const, char const, st_ha_create_information, bool, bool)+0x241 [mysqld] rea_create_table(THD, char const, char const, char const, st_ha_create_information, List&, unsigned int, st_key, handler, bool)+0x1dd [mysqld] create_table_impl(THD, char const, char const, char const, char const, st_ha_create_information, Alter_info, bool, unsigned int, bool, bool*, st_key*, unsigned int)+0x1498 [mysqld] mysql_create_table_no_lock(THD, char const, char const, st_ha_create_information, Alter_info, unsigned int, bool)+0x17c [mysqld] mysql_create_table(THD, TABLE_LIST, st_ha_create_information, Alter_info)+0xf4 [mysqld] mysql_execute_command(THD, bool)+0x1de4 [mysqld] mysql_parse(THD, Parser_state)+0x5fc [mysqld] dispatch_command(THD, COM_DATA const, enum_server_command)+0xca9 [mysqld] do_command(THD)+0x4b2 [mysqld] handle_connection+0x1e0 [mysqld] pfs_spawn_thread+0x170 [mysqld] start_thread+0xc5 [libpthread-2.17.so]

[root@hadoop03 tools]# ./trace -U -p 26720 '_Z24log_buffer_flush_to_diskb' Failed to attach BPF to kprobe [root@hadoop03 tools]# ./trace -U -p 26720 ' _Z24log_buffer_flush_to_diskb' check dmesg output for possible cause Failed to attach BPF to kprobe

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/iovisor/bcc/issues/1640#issuecomment-374848319, or mute the thread https://github.com/notifications/unsubscribe-auth/ABm3vaqrhQmUPYX54EdCXc4QCqFbu59Hks5tgfrTgaJpZM4SzEIv .

-- Sasha Goldshtein | CTO, Sela Group | Microsoft C# MVP | blog.sashag.net | @goldshtn http://twitter.com/goldshtn

yonghong-song commented 6 years ago

I think the wildcard-based approach is probably friendlier -- we don't have support for it in the 'trace' tool yet, but we have it implemented in funccount, stackcount and others, and it probably makes sense to add here as well.

I second this. This will be useful especially for return probes, in which case, you can sample a large portion of function with return probe and will now which one is actually called with anticipated value. Saving a lot of times to try one after another.

On Wed, Mar 21, 2018 at 5:30 AM, Sasha Goldshtein notifications@github.com wrote:

Without specifying the module, we would have to magically guess that you mean a user function and not a kernel function. Even if there was a switch to specify it, if we don't know the module name we'd have to look at all the modules loaded into the process and search all the symbols -- I think it's probably not the best idea.

Re figuring out the mangled name automatically, it's also a bit hard, because the mangled name is produced from the full signature. I.e. "f(int, int)" would have a different mangled name than "f(double)" etc. -- so we'd have to get the full signature from the user in order to figure out the mangled name. I think the wildcard-based approach is probably friendlier -- we don't have support for it in the 'trace' tool yet, but we have it implemented in funccount, stackcount and others, and it probably makes sense to add here as well.

On Wed, Mar 21, 2018 at 9:01 AM, hongbin notifications@github.com wrote:

It's OK, but not friendly

[root@hadoop03 tools]./trace -U -p 26720 '/usr/local/mysql/bin/mysqld:_ Z24log_buffer_flush_to_diskb' PID TID COMM FUNC 26720 230362 mysqld _Z24log_buffer_flush_to_diskb log_buffer_flush_to_disk(bool)+0x0 [mysqld] handler::ha_create(char const, TABLE, st_ha_create_information)+0x74 [mysqld] ha_create_table(THD, char const, char const, char const, st_ha_create_information, bool, bool)+0x241 [mysqld] rea_create_table(THD, char const, char const, char const, st_ha_create_information, List&, unsigned int, st_key, handler, bool)+0x1dd [mysqld] create_table_impl(THD, char const, char const, char const, char const, st_ha_create_information, Alter_info, bool, unsigned int, bool, bool*, st_key*, unsigned int)+0x1498 [mysqld] mysql_create_table_no_lock(THD, char const, char const, st_ha_create_information, Alter_info, unsigned int, bool)+0x17c [mysqld] mysql_create_table(THD, TABLE_LIST, st_ha_create_information, Alter_info)+0xf4 [mysqld] mysql_execute_command(THD, bool)+0x1de4 [mysqld] mysql_parse(THD, Parser_state)+0x5fc [mysqld] dispatch_command(THD, COM_DATA const, enum_server_command)+0xca9 [mysqld] do_command(THD)+0x4b2 [mysqld] handle_connection+0x1e0 [mysqld] pfs_spawn_thread+0x170 [mysqld] start_thread+0xc5 [libpthread-2.17.so]

[root@hadoop03 tools]# ./trace -U -p 26720 '_Z24log_buffer_flushto diskb' Failed to attach BPF to kprobe [root@hadoop03 tools]# ./trace -U -p 26720 ' _Z24log_buffer_flush_to_diskb' check dmesg output for possible cause Failed to attach BPF to kprobe

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/iovisor/bcc/issues/1640#issuecomment-374848319, or mute the thread https://github.com/notifications/unsubscribe-auth/ ABm3vaqrhQmUPYX54EdCXc4QCqFbu59Hks5tgfrTgaJpZM4SzEIv .

-- Sasha Goldshtein CTO, Sela Group Microsoft C# MVP blog.sashag.net

@goldshtn http://twitter.com/goldshtn

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/iovisor/bcc/issues/1640#issuecomment-374920775, or mute the thread https://github.com/notifications/unsubscribe-auth/ALq6ovCLvnWYD3FjaChZ8QgHHw6FTLVbks5tgkgBgaJpZM4SzEIv .