cornell-zhang / heterocl

HeteroCL: A Multi-Paradigm Programming Infrastructure for Software-Defined Heterogeneous Computing
https://cornell-zhang.github.io/heterocl/
Apache License 2.0
322 stars 92 forks source link

invalid pointer error when importing hlib #336

Closed hecmay closed 3 years ago

hecmay commented 3 years ago

When importing hlib, I got the following error.

$ python
Python 3.6.10 |Anaconda, Inc.| (default, May  8 2020, 02:54:21) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import hlib
Using TensorFlow backend.
*** Error in `python': munmap_chunk(): invalid pointer: 0x00007ffdee9f36e0 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x7f7c4)[0x7f650a20c7c4]
/work/shared/users/phd/sx233/heterocl/tvm/lib/libhcl.so(_ZNSt10_HashtableISsSsSaISsENSt8__detail9_IdentityESt8equal_toISsESt4hashISsENS1_18_Mod_range_hashingENS1_20_Default_ranged_hashENS1_20_Prime_rehash_policyENS1_17_Hashtable_traitsILb1ELb1ELb1EEEE21_M_insert_unique_nodeEmmPNS1_10_Hash_nodeISsLb1EEE+0x128)[0x7f648772a778]
/work/shared/users/phd/sx233/conda/envs/test/lib/python3.6/site-packages/tensorflow/python/../libtensorflow_framework.so.2(+0xd64567)[0x7f643baeb567]
/work/shared/users/phd/sx233/conda/envs/test/lib/python3.6/site-packages/tensorflow/python/../libtensorflow_framework.so.2(_ZNK10tensorflow10OpRegistry21RegisterAlreadyLockedERKSt8functionIFNS_6StatusEPNS_18OpRegistrationDataEEE+0xe6)[0x7f643bc27ce6]
hecmay commented 3 years ago

@seanlatias @paldebjit Have you ever seen this error before?

paldebjit commented 3 years ago

Nopes. I used hlib for the BNN experiments only. It did not pop up then.

seanlatias commented 3 years ago

Me neither. The master test also works well on CI.

seanlatias commented 3 years ago

Also if you see the error message, the error is from tensorflow not from hcl.

hecmay commented 3 years ago

Yep. the last trace is from libc. Not an issue of HCL.

seanlatias commented 3 years ago

@Hecmay, if this is not an HCL issue please close it.