search

Liyyy2122 / TiRGN

Code of TiRGN
33 stars 8 forks source link

CUDA kernel launch error: no kernel image is available for execution on the device #1

Open showi001 opened 1 year ago

showi001 commented 1 year ago

Thank you very much for sharing, Good job! I encountered a problem while running the code, Could you give me some advice? Thank you very much! I used the version as follows: torch =1.6.0 torchvision = 0.7.0 dgl-cu102 = 0.5.2

Traceback (most recent call last): File "main.py", line 564, in run_experiment(args) File "main.py", line 301, in run_experiment loss_e, loss_r, loss_static = model.get_loss(history_glist, output[0], static_graph, one_hot_tail_seq, one_hot_rel_seq, use_cuda) File "/home/shaowei.zhang/TiRGN/src/../src/rrgcn.py", line 240, in get_loss evolve_embs, static_emb, remb, , _ = self.forward(glist, static_graph, use_cuda) File "/home/shaowei.zhang/TiRGN/src/../src/rrgcn.py", line 171, in forward self.statci_rgcn_layer(static_graph, []) File "/home/anaconda3/envs/zhang_tkg/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl result = self.forward(*input, *kwargs) File "/home/shaowei.zhang/TiRGN/src/../rgcn/layers.py", line 59, in forward self.propagate(g) File "/home/shaowei.zhang/TiRGN/src/../rgcn/layers.py", line 171, in propagate g.update_all(self.msg_func, fn.sum(msg='msg', out='h'), self.apply_func) File "/home/anaconda3/envs/zhang_tkg/lib/python3.8/site-packages/dgl/heterograph.py", line 4499, in update_all ndata = core.message_passing(g, message_func, reduce_func, apply_node_func) File "/home/anaconda3/envs/zhang_tkg/lib/python3.8/site-packages/dgl/core.py", line 291, in message_passing msgdata = invoke_edge_udf(g, ALL, g.canonical_etypes[0], mfunc, orig_eid=orig_eid) File "/home/anaconda3/envs/zhang_tkg/lib/python3.8/site-packages/dgl/core.py", line 73, in invoke_edge_udf u, v, eid = graph.edges(form='all') File "/home/anaconda3/envs/zhang_tkg/lib/python3.8/site-packages/dgl/view.py", line 159, in call return self._graph.all_edges(args, **kwargs) File "/home/anaconda3/envs/zhang_tkg/lib/python3.8/site-packages/dgl/heterograph.py", line 3150, in all_edges src, dst, eid = self._graph.edges(self.get_etype_id(etype), order) File "/home/anaconda3/envs/zhang_tkg/lib/python3.8/site-packages/dgl/heterograph_index.py", line 553, in edges edge_array = _CAPI_DGLHeteroEdges(self, int(etype), order) File "dgl/_ffi/_cython/./function.pxi", line 287, in dgl._ffi._cy3.core.FunctionBase.call File "dgl/_ffi/_cython/./function.pxi", line 222, in dgl._ffi._cy3.core.FuncCall File "dgl/_ffi/_cython/./function.pxi", line 211, in dgl._ffi._cy3.core.FuncCall3 File "dgl/_ffi/_cython/./base.pxi", line 155, in dgl._ffi._cy3.core.CALL dgl._ffi.base.DGLError: [11:05:14] /opt/dgl/src/array/cuda/array_op_impl.cu:252: Check failed: e == cudaSuccess || e == cudaErrorCudartUnloading: CUDA kernel launch error: no kernel image is available for execution on the device Stack trace: [bt] (0) /home/anaconda3/envs/zhang_tkg/lib/python3.8/site-packages/dgl/libdgl.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x4f) [0x7f07b5ebea4f] [bt] (1) /home/anaconda3/envs/zhang_tkg/lib/python3.8/site-packages/dgl/libdgl.so(dgl::runtime::NDArray dgl::aten::impl::Range<(DLDeviceType)2, long>(long, long, DLContext)+0x252) [0x7f07b66d83bb] [bt] (2) /home/anaconda3/envs/zhang_tkg/lib/python3.8/site-packages/dgl/libdgl.so(dgl::aten::Range(long, long, unsigned char, DLContext)+0x1fd) [0x7f07b5e96c9d] [bt] (3) /home/anaconda3/envs/zhang_tkg/lib/python3.8/site-packages/dgl/libdgl.so(dgl::UnitGraph::COO::Edges(unsigned long, std::string const&) const+0x9b) [0x7f07b668dcfb] [bt] (4) /home/anaconda3/envs/zhang_tkg/lib/python3.8/site-packages/dgl/libdgl.so(dgl::UnitGraph::Edges(unsigned long, std::string const&) const+0xa1) [0x7f07b6688931] [bt] (5) /home/anaconda3/envs/zhang_tkg/lib/python3.8/site-packages/dgl/libdgl.so(dgl::HeteroGraph::Edges(unsigned long, std::string const&) const+0x2a) [0x7f07b65b984a] [bt] (6) /home/anaconda3/envs/zhang_tkg/lib/python3.8/site-packages/dgl/libdgl.so(+0xecdafc) [0x7f07b65c2afc] [bt] (7) /home/anaconda3/envs/zhang_tkg/lib/python3.8/site-packages/dgl/libdgl.so(DGLFuncCall+0x48) [0x7f07b6553f88] [bt] (8) /home/anaconda3/envs/zhang_tkg/lib/python3.8/site-packages/dgl/_ffi/_cy3/core.cpython-38-x86_64-linux-gnu.so(+0x15f79) [0x7f07b3deef79]

ShivaniNR commented 1 year ago

Hello @showi001, I am also facing the same issue. Could you tell me how you resolved it?