search

erlang / otp

Erlang/OTP
http://erlang.org
Apache License 2.0
11.3k stars 2.94k forks source link

Dialyzer incremental analysis crashed! #8684

Open williamweicode opened 1 month ago

williamweicode commented 1 month ago

Describe the bug Dialyzer incremental analysis crashed!

  1. first, run analysis with incremental mode, it is ok image
  2. then, change a file, run incremental analysis again, it crashed! image

To Reproduce crash dump content，omit the binary value. image

Expected behavior No Crash

Affected versions Operating System: x86_64-pc-linux-gnu ERTS: Erlang/OTP 26 [erts-14.2] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:1] [jit:ns] Dialyzer version v5.1.2

Additional context I tried to parse plt file. ifile_plt‘s "types" field parsed failed! Other ifile_plt fileds parsed ok. image

image

jhogberg commented 1 month ago

Thanks for your report! Unfortunately, we cannot do much with this and your other report, #8681, because they lack steps to reproduce the issue. For issues relating to the compiler and dialyzer we need the source code you provide as input (in this case your entire project, basically). We are not going to make much progress with any of these issues until you provide that (or a minimized version thereof).

williamweicode commented 1 month ago

I reproduced the problem locally by adding more and more analysis modules. I added some output logs in dialyzer_iplt.erl. image I ran two analyses and got the logs respectively. The second analysis adds 20 new modules to the code from the first analysis.

Normal Analysis Output: to file ets types info [{id,#Ref<0.1790361082.685637633.231597>}, {decentralized_counters,false}, {read_concurrency,false}, {write_concurrency,false}, {compressed,true}, {memory,290670482}, {owner,<0.9.0>}, {heir,none}, {name,plt_types}, {size,1561}, {node,nonode@nohost}, {named_table,false}, {type,set}, {keypos,1}, {protection,public}]

Record types 121947007

Abnormal Analysis output Proceeding with incremental analysis... compile (+0.00s): 4.65s ( 174 modules) clean (+0.00s): 0.47s remote (+4.29s):3280.48s order (+0.16s): 0.09s typesig (+0.00s): 82.31s ( 3858 SCCs) order (+0.00s): 0.10s refine (+0.00s): 10.63s ( 174 modules) order (+0.00s): 0.03s typesig (+0.00s): 30.61s ( 943 SCCs) order (+0.00s): 0.01s refine (+0.00s): 9.10s ( 130 modules) warning (+0.13s): 9.95s ( 174 modules) (+40.65s) to file ets types info [{id,#Ref<0.1140431130.2039611393.209923>}, {decentralized_counters,false}, {read_concurrency,false}, {write_concurrency,false}, {compressed,true}, {memory,305916734}, {owner,<0.9.0>}, {heir,none}, {name,plt_types}, {size,1581}, {node,nonode@nohost}, {named_table,false}, {type,set}, {keypos,1}, {protection,public}]

Record types 6152010

the ETS seems normal, but after term_to_binary(dialyzer_utils:ets_tab2list(ETSTypes), [{compressed,9}]). The binary looks truncated. Is this information useful for you? If not, I have to close this issule. It really took me a lot of time, thank you! @jhogberg

jhogberg commented 1 month ago

Thank you, can you reproduce the issue with {compressed,9} removed?

williamweicode commented 1 month ago

{compressed,9} removed, the binary is not truncated, unfortunately, it crashed when term_to_binary(Record = #ifile_plt{}). It seems, the record is too large.I'll run it again on a machine with more memory(128G).

{compressed,9} removed Analysis output: Proceeding with incremental analysis... compile (+0.05s): 45.18s (1712 modules) clean (+0.01s): 0.87s remote (+0.75s):1557.55s order (+0.29s): 7.56s typesig (+0.00s): 725.98s ( 90189 SCCs) order (+0.00s): 8.05s refine (+0.00s): 31.12s (1712 modules) order (+0.00s): 3.43s typesig (+0.00s): 169.71s ( 36958 SCCs) order (+0.00s): 1.46s refine (+0.00s): 30.33s (1659 modules) order (+0.00s): 0.09s typesig (+0.00s): 18.41s ( 2451 SCCs) order (+0.00s): 0.02s refine (+0.00s): 7.35s ( 209 modules) warning (+0.81s): 35.68s (1712 modules) (+53.80s) to file ets types info [{id,#Ref<0.1247172929.1811546113.92810>}, {decentralized_counters,false}, {read_concurrency,false}, {write_concurrency,false}, {compressed,true}, {memory,310638775}, {owner,<0.9.0>}, {heir,none}, {name,plt_types}, {size,1581}, {node,nonode@nohost}, {named_table,false}, {type,set}, {keypos,1}, {protection,public}] Record types 4515823319

Runtime terminating during boot ({{case_clause,{system_limit,[{erlang,term_to_binary,[{ifile_plt,"v5.1.3",[{'OTP-PUB-KEY',<<251,161,106,43,154,44,61,162,75,78,224,224,134,92,239,96>>},{'PKCS-FRAME',<<22,48,139,3,155,234,96,145,210,208,2,93,254,128,223,182>>},{account_cli,<<156,33,29,148,107,242,102,88,209,21,212,0,54,220,169,22>>},{account_priviledge,<<95,103,178,56,101,139,169,171,219,107,232,88,222,19,149,246>>},{account_settings,<<26,175,240,200,29,121,33,244,76,239,121,222,18,245,119,233>>},{action,<<117,11,53,142,169,14,216,206,2,35,17,2,130,34,239,113>>},{activity,<<91,181,237,170,186,121,174,137,238,248,15,170,185,94,85,64>>},{activity_amend,<<1,5,9,113,154,103,61,117,30,108,175,195,208,91,41,91>>},{activity_anniversary_creation,<<175,76,100,126,241,224,170,193,247,147,254,153,117,45,251,168>>},{activity_anniversary_legend_benefit,<<42,4,199,70,107,30,219,204,96,85,97,171,148,26,140,121>>},{activity_anniversary_mail,<<212,79,186,173,126,243,197,250,89,231,177,190,196,24,1,235>>},{activityanniversary

Crash dump is being written to: erl_crash.dump...done

image the erl_crash.dump size is 5.7G @jhogberg

williamweicode commented 1 month ago

[root@k8s-wgn-test Server]# dialyzer --incremental --statistics --verbose

{compressed,9} removed Analysis output: PLT does not yet exist at /root/.cache/erlang/.dialyzer_iplt, so an analysis must be run for 2052 modules to populate it Proceeding with incremental analysis... compile (+0.07s): 100.03s (2052 modules) clean (+0.02s): 2.65s remote (+3.31s):4735.31s order (+0.37s): 13.84s typesig (+0.00s): 797.04s ( 98547 SCCs) order (+0.00s): 14.51s refine (+0.00s): 93.58s (2052 modules) order (+0.00s): 5.90s typesig (+0.00s): 270.50s ( 42790 SCCs) order (+0.00s): 4.12s refine (+0.00s): 90.07s (1982 modules) order (+0.00s): 0.27s typesig (+0.00s): 118.46s ( 4680 SCCs) order (+0.00s): 0.20s refine (+0.00s): 33.46s ( 502 modules) warning (+2.50s): 81.81s (2052 modules) (+*****s) to file ets types info [{id,#Ref<0.1822072438.2821586945.135145>}, {decentralized_counters,false}, {read_concurrency,false}, {write_concurrency,false}, {compressed,true}, {memory,783014501}, {owner,<0.9.0>}, {heir,none}, {name,plt_types}, {size,1915}, {node,nonode@nohost}, {named_table,false}, {type,set}, {keypos,1}, {protection,public}] Record types 11388243954

Runtime terminating during boot ({{case_clause,{system_limit,[{erlang,term_to_binary,[{ifile_plt,"v5.1.3",[{'OTP-PUB-KEY',<<190,210,80,65,92,105,94,225,124,136,78,34,98,224,146,128>>},{'PKCS-FRAME',<<176,237,179,236,60,99,74,146,127,103,121,242,15,127,106,252>>},{account_cli,<<119,13,90,7,148,217,125,134,85,87,64,220,88,242,48,16>>},{account_priviledge,<<128,64,217,149,64,191,132,66,219,194,177,235,114,46,8,104>>},{account_settings,<<12,53,76,165,43,124,29,166,236,72,228,105,127,224,87,12>>},{action,<<117,11,53,142,169,14,216,206,2,35,17,2,130,34,239,113>>},{activity,<<218,225,207,9,135,95,141,80,203,246,245,100,68,179,39,59>>},{activity_amend,<<18,18,79,190,245,74,211,236,252,16,16,210,89,210,108,209>>},{activity_anniversary_creation,<<219,29,47,246,109,149,77,148,90,7,179,24,126,228,19,103>>},{activity_anniversary_legend_benefit,<<167,166,47,163,201,158,59,241,125,24,240,239,147,190,241,41>>},{activity_anniversary_mail,<<127,188,195,14,191,14,59,25,50,64,119,178,253,158,0,92>>},{activity_anniversary_mini_h

Crash dump is being written to: erl_crash.dump...done

@jhogberg After {compressed,9} removed , the analysis crashed again, I cant figure out what the system_limit is related to

erl_crash.dump first 100 lines: =erl_crash_dump:0.5 Fri Aug 2 11:38:35 2024 Slogan: Runtime terminating during boot ({{case_clause,{system_limit,[{erlang,term_to_binary,[{ifile_plt,"v5.1.3",[{'OTP-PUB-KEY',<<190,210,80,65,92,105,94,225,124,136,78,34,98,224,146,128>>},{'PKCS-FRAME',<<176,237,179,236,60,99,74,146,127,103,121,242,15,127,106,252>>},{account_cli,<<119,13,90,7,148,217,125,134,85,87,64,220,88,242,48,16>>},{account_priviledge,<<128,64,217,149,64,191,132,66,219,194,177,235,114,46,8,104>>},{account_settings,<<12,53,76,165,43,124,29,166,236,72,228,105,127,224,87,12>>},{action,<<117,11,53,142,169,14,216,206,2,35,17,2,130,34,239,113>>},{activity,<<218,225,207,9,135,95,141,80,203,246,245,100,68,179,39,59>>},{activity_amend,<<18,18,79,190,245,74,211,236,252,16,16,210,89,210,108,209>>},{activity_anniversary_creation,<<219,29,47,246,109,149,77,148,90,7,179,24,126,228,19,103>>},{activity_anniversary_legend_benefit,<<167,166,47,163,201,158,59,241,125,24,240,239,147,190,241,41>>},{activity_anniversary_mail,<<127,188,195,14,191,14,59,25,50,64,119,178,253,158,0,92>>},{activity_anniversary_mini_h System version: Erlang/OTP 26 [erts-14.2] [source] [64-bit] [smp:32:32] [ds:32:32:10] [async-threads:1] [jit:ns] Taints: Atoms: 165776 Calling Thread: scheduler:1 =scheduler:1 Scheduler Sleep Info Flags: Scheduler Sleep Info Aux Work: DD | DD_THR_PRGR Current Port: Run Queue Max Length: 0 Run Queue High Length: 0 Run Queue Normal Length: 1 Run Queue Low Length: 0 Run Queue Port Length: 0 Run Queue Flags: NONEMPTY_NORMAL | OUT_OF_WORK | HALFTIME_OUT_OF_WORK | NONEMPTY | EXEC Current Process: <0.0.0> Current Process State: Running Current Process Internal State: ACT_PRIO_NORMAL | USR_PRIO_NORMAL | PRQ_PRIO_NORMAL | ACTIVE | RUNNING Current Process Program counter: 0x00007f6742d4700c (init:halt_string/2 + 92) Current Process Limited Stack Trace: 0x00007f5dbbba0760:SReturn addr 0x42DB24A0 (erlang:halt/1 + 136) 0x00007f5dbbba0778:SReturn addr 0x42D3FF88 () =scheduler:2 Scheduler Sleep Info Flags: SLEEPING | TSE_SLEEPING | WAITING Scheduler Sleep Info Aux Work: Current Port: Run Queue Max Length: 0 Run Queue High Length: 0 Run Queue Normal Length: 0 Run Queue Low Length: 0 Run Queue Port Length: 0 Run Queue Flags: OUT_OF_WORK | INACTIVE Current Process: =scheduler:3 Scheduler Sleep Info Flags: SLEEPING | TSE_SLEEPING | WAITING Scheduler Sleep Info Aux Work: Current Port: Run Queue Max Length: 0 Run Queue High Length: 0 Run Queue Normal Length: 0 Run Queue Low Length: 0 Run Queue Port Length: 0 Run Queue Flags: OUT_OF_WORK | INACTIVE Current Process: =scheduler:4 Scheduler Sleep Info Flags: SLEEPING | TSE_SLEEPING | WAITING Scheduler Sleep Info Aux Work: DD_THR_PRGR Current Port: Run Queue Max Length: 0 Run Queue High Length: 0 Run Queue Normal Length: 0 Run Queue Low Length: 0 Run Queue Port Length: 0 Run Queue Flags: OUT_OF_WORK | INACTIVE Current Process: =scheduler:5 Scheduler Sleep Info Flags: SLEEPING | TSE_SLEEPING | WAITING Scheduler Sleep Info Aux Work: Current Port: Run Queue Max Length: 0 Run Queue High Length: 0 Run Queue Normal Length: 0 Run Queue Low Length: 0 Run Queue Port Length: 0 Run Queue Flags: OUT_OF_WORK | INACTIVE Current Process: =scheduler:6 Scheduler Sleep Info Flags: SLEEPING | TSE_SLEEPING | WAITING Scheduler Sleep Info Aux Work: Current Port: Run Queue Max Length: 0 Run Queue High Length: 0 Run Queue Normal Length: 0 Run Queue Low Length: 0 Run Queue Port Length: 0 Run Queue Flags: OUT_OF_WORK | INACTIVE Current Process: =scheduler:7 Scheduler Sleep Info Flags: SLEEPING | TSE_SLEEPING | WAITING Scheduler Sleep Info Aux Work: Current Port: Run Queue Max Length: 0 Run Queue High Length: 0 Run Queue Normal Length: 0 Run Queue Low Length: 0 Run Queue Port Length: 0 Run Queue Flags: OUT_OF_WORK | INACTIVE Current Process: =scheduler:8 Scheduler Sleep Info Flags: SLEEPING | TSE_SLEEPING | WAITING Scheduler Sleep Info Aux Work: Current Port: Run Queue Max Length: 0 Run Queue High Length: 0 Run Queue Normal Length: 0 Run Queue Low Length: 0 Run Queue Port Length: 0 Run Queue Flags: OUT_OF_WORK | INACTIVE

erl_crash.dump memory info: =memory total: 11503469144 processes: 28370800 processes_used: 28368768 system: 11475098344 atom: 5358433 atom_used: 5347822 binary: 11412391376 code: 8776534 ets: 485280 =hash_table:atom_tab size: 131072 used: 94232 objs: 165776 depth: 8 =index_table:atom_tab size: 165888 limit: 1048576 entries: 165776 =hash_table:module_code size: 128 used: 97 objs: 162 depth: 5 =index_table:module_code size: 1024 limit: 65536 entries: 162 =hash_table:export_list :

williamweicode commented 1 month ago

Hi, Will this issue move forward? @jhogberg

jhogberg commented 1 month ago

Not in the near term, I'm afraid. We'll add it to our backlog.

williamweicode commented 1 month ago

Not in the near term, I'm afraid. We'll add it to our backlog.

OK, Thanks!