search

fixstars / ion-kit

Modernized graph-based data processing framework
MIT License
7 stars 6 forks source link

Ordering of auto dispose #249

Closed iitaku closed 7 months ago

iitaku commented 7 months ago

Reported in #245. I confirmed it happens on our test (simple_graph_jit).

17f645c:

ION_LOG_LEVEL=debug LD_LIBRARY_PATH=./test ./test/simple_graph_jit                                                                                                                        ion-kit/git/main !
[2024-02-27 13:14:07.133] [ion] [debug] ion-kit version is v1.6.0-23-gfbf23bb
[2024-02-27 13:14:07.133] [ion] [debug] Determine free port _ion_iport_0 as input on Node 60a2b9b8-83a5-4451-9329-bedfceb598ed
[2024-02-27 13:14:07.133] [ion] [debug] Determine free port _ion_iport_1 as min0 on Node 60a2b9b8-83a5-4451-9329-bedfceb598ed
[2024-02-27 13:14:07.133] [ion] [debug] Determine free port _ion_iport_2 as extent0 on Node 60a2b9b8-83a5-4451-9329-bedfceb598ed
[2024-02-27 13:14:07.133] [ion] [debug] Determine free port _ion_iport_3 as min1 on Node 60a2b9b8-83a5-4451-9329-bedfceb598ed
[2024-02-27 13:14:07.133] [ion] [debug] Determine free port _ion_iport_4 as extent1 on Node 60a2b9b8-83a5-4451-9329-bedfceb598ed
[2024-02-27 13:14:07.134] [ion] [debug] Determine free port _ion_iport_5 as v on Node 60a2b9b8-83a5-4451-9329-bedfceb598ed
0'th loop
[2024-02-27 13:14:07.134] [ion] [info] Start building pipeline
[2024-02-27 13:14:07.134] [ion] [info] Builder::register_disposer
[2024-02-27 13:14:07.134] [ion] [info] consume_dispose is called with id=60a2b9b8-83a5-4451-9329-bedfceb598ed
[2024-02-27 13:14:07.617] [ion] [debug] Inserted "___ion_port_0" instance at #4
[2024-02-27 13:14:07.617] [ion] [debug] Inserted "___ion_port_$10" instance at #0
[2024-02-27 13:14:07.617] [ion] [debug] Inserted "___ion_port_$20" instance at #1
[2024-02-27 13:14:07.617] [ion] [debug] Inserted "___ion_port_$30" instance at #2
[2024-02-27 13:14:07.617] [ion] [debug] Inserted "___ion_port_$40" instance at #3
[2024-02-27 13:14:07.617] [ion] [debug] Inferred arguments stub
[2024-02-27 13:14:07.626] [ion] [debug]   #0 name(___ion_port_$10) kind(InputScalar) dimensions(0) type(int32_t)
[2024-02-27 13:14:07.627] [ion] [debug]   #1 name(___ion_port_$20) kind(InputScalar) dimensions(0) type(int32_t)
[2024-02-27 13:14:07.627] [ion] [debug]   #2 name(___ion_port_$30) kind(InputScalar) dimensions(0) type(int32_t)
[2024-02-27 13:14:07.627] [ion] [debug]   #3 name(___ion_port_$40) kind(InputScalar) dimensions(0) type(int32_t)
[2024-02-27 13:14:07.627] [ion] [debug]   #4 name(___ion_port_0) kind(InputScalar) dimensions(0) type(int32_t)
[2024-02-27 13:14:07.627] [ion] [debug] Generating arguments instance
[2024-02-27 13:14:07.627] [ion] [debug]   #0 0x7ffc5085a4c0
[2024-02-27 13:14:07.627] [ion] [debug]   #1 0x7ffc5085a4c4
[2024-02-27 13:14:07.627] [ion] [debug]   #2 0x7ffc5085a4c8
[2024-02-27 13:14:07.627] [ion] [debug]   #3 0x7ffc5085a4cc
[2024-02-27 13:14:07.627] [ion] [debug]   #4 0x7ffc5085a4bc
[2024-02-27 13:14:07.627] [ion] [debug]   #5 0x557e4e299f28
[2024-02-27 13:14:07.627] [ion] [info] consume is called with id=60a2b9b8-83a5-4451-9329-bedfceb598ed
42 42 
42 42 
1'th loop
[2024-02-27 13:14:07.627] [ion] [info] consume is called with id=60a2b9b8-83a5-4451-9329-bedfceb598ed
42 42 
42 42 
2'th loop
[2024-02-27 13:14:07.627] [ion] [info] consume is called with id=60a2b9b8-83a5-4451-9329-bedfceb598ed
42 42 
42 42 
3'th loop
[2024-02-27 13:14:07.627] [ion] [info] consume is called with id=60a2b9b8-83a5-4451-9329-bedfceb598ed
42 42 
42 42 
4'th loop
[2024-02-27 13:14:07.627] [ion] [info] consume is called with id=60a2b9b8-83a5-4451-9329-bedfceb598ed
42 42 
42 42 
[2024-02-27 13:14:07.627] [ion] [info] consume_dispose is called with id=60a2b9b8-83a5-4451-9329-bedfceb598ed
Passed

v1.6.0:

ION_LOG_LEVEL=debug LD_LIBRARY_PATH=./test ./test/simple_graph_jit                                                                                                                 ion-kit/git/tags/v1.6.0 !
[2024-02-27 13:17:14.663] [ion] [debug] ion-kit version is v1.6.0
[2024-02-27 13:17:14.664] [ion] [debug] Determine free port _ion_iport_0 as input on Node 2b7daac1-9c0b-4aae-beed-beccfe1b9f49
[2024-02-27 13:17:14.664] [ion] [debug] Determine free port _ion_iport_1 as min0 on Node 2b7daac1-9c0b-4aae-beed-beccfe1b9f49
[2024-02-27 13:17:14.664] [ion] [debug] Determine free port _ion_iport_2 as extent0 on Node 2b7daac1-9c0b-4aae-beed-beccfe1b9f49
[2024-02-27 13:17:14.664] [ion] [debug] Determine free port _ion_iport_3 as min1 on Node 2b7daac1-9c0b-4aae-beed-beccfe1b9f49
[2024-02-27 13:17:14.664] [ion] [debug] Determine free port _ion_iport_4 as extent1 on Node 2b7daac1-9c0b-4aae-beed-beccfe1b9f49
[2024-02-27 13:17:14.664] [ion] [debug] Determine free port _ion_iport_5 as v on Node 2b7daac1-9c0b-4aae-beed-beccfe1b9f49
0'th loop
[2024-02-27 13:17:14.664] [ion] [info] Start building pipeline
[2024-02-27 13:17:14.664] [ion] [info] Builder::register_disposer
[2024-02-27 13:17:15.156] [ion] [info] consume is called with id=2b7daac1-9c0b-4aae-beed-beccfe1b9f49
42 42 
42 42 
1'th loop
[2024-02-27 13:17:15.156] [ion] [info] consume is called with id=2b7daac1-9c0b-4aae-beed-beccfe1b9f49
42 42 
42 42 
2'th loop
[2024-02-27 13:17:15.156] [ion] [info] consume is called with id=2b7daac1-9c0b-4aae-beed-beccfe1b9f49
42 42 
42 42 
3'th loop
[2024-02-27 13:17:15.156] [ion] [info] consume is called with id=2b7daac1-9c0b-4aae-beed-beccfe1b9f49
42 42 
42 42 
4'th loop
[2024-02-27 13:17:15.156] [ion] [info] consume is called with id=2b7daac1-9c0b-4aae-beed-beccfe1b9f49
42 42 
42 42 
[2024-02-27 13:17:15.156] [ion] [info] consume_dispose is called with id=2b7daac1-9c0b-4aae-beed-beccfe1b9f49
Passed

@xinyuli1204 Can you try to figure it out the reason and fix it?

xinyuli1204 commented 7 months ago

https://github.com/fixstars/ion-kit/pull/250

xinyuli1204 commented 7 months ago 
xinyuli@Xinyus-MacBook-Air test % ./simple_graph_jit                  
[2024-02-27 14:34:17.119] [ion] [debug] ion-kit version is v1.6.0-40-g17f645c
[2024-02-27 14:34:17.125] [ion] [debug] Determine free port _ion_iport_0 as input on Node 82cfca5c-02ab-4884-9306-fc1ec2b8b2f2
[2024-02-27 14:34:17.125] [ion] [debug] Determine free port _ion_iport_1 as min0 on Node 82cfca5c-02ab-4884-9306-fc1ec2b8b2f2
[2024-02-27 14:34:17.125] [ion] [debug] Determine free port _ion_iport_2 as extent0 on Node 82cfca5c-02ab-4884-9306-fc1ec2b8b2f2
[2024-02-27 14:34:17.125] [ion] [debug] Determine free port _ion_iport_3 as min1 on Node 82cfca5c-02ab-4884-9306-fc1ec2b8b2f2
[2024-02-27 14:34:17.125] [ion] [debug] Determine free port _ion_iport_4 as extent1 on Node 82cfca5c-02ab-4884-9306-fc1ec2b8b2f2
[2024-02-27 14:34:17.125] [ion] [debug] Determine free port _ion_iport_5 as v on Node 82cfca5c-02ab-4884-9306-fc1ec2b8b2f2
0'th loop
[2024-02-27 14:34:17.125] [ion] [info] Start building pipeline
[2024-02-27 14:34:17.126] [ion] [info] Builder::register_disposer
[2024-02-27 14:34:17.369] [ion] [debug] Inserted "___ion_port_0" instance at #4
[2024-02-27 14:34:17.369] [ion] [debug] Inserted "___ion_port_$10" instance at #0
[2024-02-27 14:34:17.369] [ion] [debug] Inserted "___ion_port_$20" instance at #1
[2024-02-27 14:34:17.369] [ion] [debug] Inserted "___ion_port_$30" instance at #2
[2024-02-27 14:34:17.369] [ion] [debug] Inserted "___ion_port_$40" instance at #3
[2024-02-27 14:34:17.369] [ion] [debug] Inferred arguments stub
[2024-02-27 14:34:17.369] [ion] [debug]   #0 name(___ion_port_$10) kind(InputScalar) dimensions(0) type(int32_t)
[2024-02-27 14:34:17.369] [ion] [debug]   #1 name(___ion_port_$20) kind(InputScalar) dimensions(0) type(int32_t)
[2024-02-27 14:34:17.369] [ion] [debug]   #2 name(___ion_port_$30) kind(InputScalar) dimensions(0) type(int32_t)
[2024-02-27 14:34:17.369] [ion] [debug]   #3 name(___ion_port_$40) kind(InputScalar) dimensions(0) type(int32_t)
[2024-02-27 14:34:17.369] [ion] [debug]   #4 name(___ion_port_0) kind(InputScalar) dimensions(0) type(int32_t)
[2024-02-27 14:34:17.369] [ion] [debug] Generating arguments instance
[2024-02-27 14:34:17.369] [ion] [debug]   #0 0x16d0334e8
[2024-02-27 14:34:17.369] [ion] [debug]   #1 0x16d0334e4
[2024-02-27 14:34:17.369] [ion] [debug]   #2 0x16d0334e0
[2024-02-27 14:34:17.369] [ion] [debug]   #3 0x16d0334dc
[2024-02-27 14:34:17.369] [ion] [debug]   #4 0x16d0334ec
[2024-02-27 14:34:17.369] [ion] [debug]   #5 0x600000770020
[2024-02-27 14:34:17.369] [ion] [info] consume is called with id=82cfca5c-02ab-4884-9306-fc1ec2b8b2f2
42 42 
42 42 
1'th loop
[2024-02-27 14:34:17.369] [ion] [info] consume is called with id=82cfca5c-02ab-4884-9306-fc1ec2b8b2f2
42 42 
42 42 
2'th loop
[2024-02-27 14:34:17.369] [ion] [info] consume is called with id=82cfca5c-02ab-4884-9306-fc1ec2b8b2f2
42 42 
42 42 
3'th loop
[2024-02-27 14:34:17.369] [ion] [info] consume is called with id=82cfca5c-02ab-4884-9306-fc1ec2b8b2f2
42 42 
42 42 
4'th loop
[2024-02-27 14:34:17.369] [ion] [info] consume is called with id=82cfca5c-02ab-4884-9306-fc1ec2b8b2f2
42 42 
42 42 
[2024-02-27 14:34:17.369] [ion] [info] consume_dispose is called with id=82cfca5c-02ab-4884-9306-fc1ec2b8b2f2
Passed
iitaku commented 7 months ago

Fixed in #250