search

graphcore / poptorch

PyTorch interface for the IPU
https://docs.graphcore.ai/projects/poptorch-user-guide/en/latest/
MIT License
176 stars 14 forks source link

"NotImplementedError: Could not run 'aten::_local_scalar_dense' with arguments from the 'Meta' backend." when calling tensor.item() or int() #16

Closed Bedledl closed 1 year ago

Bedledl commented 1 year ago

Description NotImplementedError: Could not run 'aten::_local_scalar_dense' with arguments from the 'Meta' backend. when calling tensor.item() or int().

Environment Graphcore Docker Image graphcore/pytorch-geometric 3.2.0

same behaviour executing either on the CPU with POPTORCH_IPU_MODEL=1 or on Graphcore Hardware(IP‌U-M2‌00‌0‌ + IP‌‍U‌-P‍OD‌4).

Minimal Example

I modified the "simple_adder.py" example by adding a int() or tensor.item() call. The behaviour is the same with commenting out i = int(y[0]) or i = y.item()

#!/usr/bin/env python3

# Copyright (c) 2020 Graphcore Ltd. All rights reserved.
import torch
import torch.nn as nn
import poptorch

class SimpleAdder(nn.Module):
    def forward(self, x, y):
#        i = int(y[0]) # same error message with either one of these lines
        i = y.item()
        return x + y * i

model = SimpleAdder()
inference_model = poptorch.inferenceModel(model)

t1 = torch.tensor([1.])
t2 = torch.tensor([2.])

print(inference_model(t1, t2))
print("Success")
Full Log ``` [08:59:07.342] [poptorch:cpp] [] Initializing aten symbols [08:59:07.343] [poptorch:cpp] [] Initializing torch_scatter symbols [08:59:07.343] [poptorch:cpp] [] Initializing supported operationss [08:59:07.343] [poptorch:cpp] [] Initializing poptorch symbols [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::glu [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::rrelu [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::rrelu_with_noise [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::softplus [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::hardsigmoid [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::hardswish [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::prelu [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::gelu [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::add [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::trunc [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::frac [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::floor_divide [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::mul [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::true_divide [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::clamp [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::clamp_min [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::clamp_max [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::addcdiv [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::addcmul [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::cross [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::linalg_cross [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::var [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::var_mean [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::std [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::std_mean [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::abs [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::acos [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::acosh [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::addmm [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::asin [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::asinh [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::atan [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::atan2 [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::atanh [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::cat [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::ceil [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::celu [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::constant_pad_nd [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::cos [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::cosh [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::detach [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::div [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::elu [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::eq [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::erf [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::erfc [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::exp [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::expm1 [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::floor [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::fmod [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::ge [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::gt [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::hardshrink [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::hardtanh [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::hinge_embedding_loss [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::index_select [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::isnan [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::l1_loss [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::le [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::leaky_relu [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::log [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::log10 [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::log1p [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::log2 [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::log_sigmoid [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::log_sigmoid_forward [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::logical_and [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::logical_not [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::logical_or [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::lt [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::margin_ranking_loss [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::masked_fill [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::mse_loss [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::ne [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::neg [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::normal_ [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::pixel_shuffle [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::pow [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::rand [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::randn [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::reciprocal [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::reflection_pad1d [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::reflection_pad2d [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::relu [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::remainder [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::replication_pad1d [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::replication_pad2d [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::replication_pad3d [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::round [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::rsqrt [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::rsub [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::selu [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::sigmoid [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::sign [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::silu [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::sin [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::sinh [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::smooth_l1_loss [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::soft_margin_loss [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::softshrink [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::sqrt [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::square [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::sub [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::t [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::tan [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::tanh [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::threshold [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::topk [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::uniform_ [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::where [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::bilinear [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::bitwise_and [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::bitwise_not [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::bitwise_or [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::bitwise_xor [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::__and__ [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::__or__ [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::__xor__ [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::matmul [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::baddbmm [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::addmv [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::mv [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::dot [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::bmm [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::mm [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::arange [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::ones [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::ones_like [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::new_ones [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::new_zeros [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::zeros [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::zeros_like [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::randperm [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::convolution [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::convolution_overrideable [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::_convolution [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::mkldnn_convolution [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::conv2d [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::cumsum [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol poptorch::custom_operation [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::pairwise_distance [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::cosine_similarity [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::feature_dropout [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::dropout [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::embedding [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::embedding_bag [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::_embedding_bag [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::one_hot [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::index [08:59:07.343] [poptorch:cpp] [] Registering handler for symbol aten::index_put [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::index_fill [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::nll_loss2d [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::nll_loss2d_forward [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::nll_loss_nd [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::nll_loss_forward [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::binary_cross_entropy [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::kl_div [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::poisson_nll_loss [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::binary_cross_entropy_with_logits [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::multilabel_soft_margin_loss [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::cosine_embedding_loss [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::triplet_margin_loss [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::ctc_loss [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol poptorch::ctc_beam_search_decoder [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::cross_entropy_loss [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::batch_norm [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::native_batch_norm [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::layer_norm [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::native_layer_norm [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::group_norm [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::native_group_norm [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::instance_norm [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::einsum [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::meshgrid [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::cartesian_prod [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::tensordot [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::scatter_add [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::scatter_reduce [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::index_reduce [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::_weight_norm [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::randint [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::random_ [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol poptorch::set_available_memory [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::max_pool1d [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::avg_pool1d [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::max_pool2d [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::avg_pool2d [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::max_pool3d [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::avg_pool3d [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::max_pool1d_with_indices [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::max_pool2d_with_indices [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::max_pool3d_with_indices [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::adaptive_avg_pool1d [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::adaptive_avg_pool2d [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::adaptive_avg_pool3d [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol poptorch::begin_ipu_block [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol poptorch::begin_multi_conv [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol poptorch::call_cpu_op [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol poptorch::end_cpu_op [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol poptorch::end_for_loop [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol poptorch::end_ipu_block [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol poptorch::identity_loss [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol poptorch::internal_cast [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol poptorch::nop [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol poptorch::optimizer_group [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol poptorch::pop_name_scope [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol poptorch::recomputation_checkpoint [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol poptorch::set_matmul_serialization [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol poptorch::start_for_loop [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol poptorch::update_param_inplace [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol torch_scatter::scatter_max [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol torch_scatter::scatter_min [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::gru [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::lstm [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::rnn_tanh [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::rnn_relu [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::normal [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::bernoulli [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::exponential_ [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::amax [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::amin [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::argmax [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::argmin [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::argsort [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::prod [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::mean [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::median [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::sum [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::logsumexp [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::norm [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::linalg_vector_norm [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::frobenius_norm [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::min [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::minimum [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::max [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::maximum [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::any [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::all [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::count_nonzero [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::nansum [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::broadcast_to [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::expand [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::expand_as [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::view [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::_unsafe_view [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::unsqueeze [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::flatten [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::reshape [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::_reshape_alias [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::select [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::split [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::split_with_sizes [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::chunk [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::unsafe_chunk [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::contiguous [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::permute [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::transpose [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::transpose_ [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::col2im [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::im2col [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::numpy_T [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::to [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::type_as [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::upsample_nearest1d [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::upsample_nearest2d [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::upsample_nearest3d [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::upsample_linear1d [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::upsample_bilinear2d [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::upsample_trilinear3d [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::upsample_bicubic2d [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::squeeze [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::as_strided [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::stack [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::Int [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::slice [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol poptorch::dynamic_slice [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol poptorch::dynamic_update [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::unbind [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::narrow [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::unfold [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::softmax [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::_softmax [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::log_softmax [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::_log_softmax [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::size [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol prim::NumToTensor [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::flip [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::repeat [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::is_complex [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::roll [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::clone [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::copy_ [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::linear [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::gather [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::scatter [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::full [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::new_full [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::full_like [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol aten::triu [08:59:07.344] [poptorch:cpp] [] Registering handler for symbol poptorch::ipu_print_tensor [08:59:07.363] [poptorch:cpp] [trace] Setting op source to: /usr/local/lib/python3.8/dist-packages/poptorch/_poplar_executor.py:656 [08:59:07.363] [poptorch:cpp] [debug] [DISPATCHER] Intercepting aten::empty.memory_format, device ipu:0 [08:59:07.363] [poptorch:cpp] [trace] Created IPU tensor: id 1 impl_ 0x6185cd0 size [1] strides 1 dtype Float [08:59:07.363] [poptorch:cpp] [trace] Setting op source to: /usr/local/lib/python3.8/dist-packages/poptorch/_poplar_executor.py:793 [08:59:07.363] [poptorch:cpp] [trace] Adding 0x6185cd0 to value mapper 0x705fa38, JIT ir: 6 [08:59:07.363] [poptorch:cpp] [debug] [DISPATCHER] Intercepting aten::copy_ [08:59:07.363] [poptorch:cpp] [trace] [Input] self impl_ 0x6185cd0 type ipu ID 1 sizes [1] dtype float [08:59:07.363] [poptorch:cpp] [trace] [Input] src impl_ 0x7057ac0 type cpu sizes [1] dtype float [08:59:07.363] [poptorch:cpp] [trace] Setting op source to: /usr/local/lib/python3.8/dist-packages/poptorch/_poplar_executor.py:793 [08:59:07.363] [poptorch:cpp] [trace] [DISPATCHER] Copying from CPU tensor 0x7057ac0 with data_ptr 0x7044680 [08:59:07.363] [poptorch:cpp] [trace] [DISPATCHER] Adding input: Value 0x70654a0 with cpu ptr 0x7044680 [08:59:07.364] [poptorch:cpp] [trace] Tracking tensor %7 [08:59:07.364] [poptorch:cpp] [trace] Adding 0x6185cd0 to value mapper 0x705fa38, JIT ir: 7 [08:59:07.364] [poptorch:cpp] [debug] copy_ CPU -> IPU input, new self impl_ 0x6185cd0 type ipu ID 1 sizes [1] dtype float [08:59:07.364] [poptorch:cpp] [debug] [DISPATCHER] Intercepting aten::empty.memory_format, device ipu:0 [08:59:07.364] [poptorch:cpp] [trace] Created IPU tensor: id 2 impl_ 0x5fc3900 size [1] strides 1 dtype Float [08:59:07.364] [poptorch:cpp] [trace] Setting op source to: /usr/local/lib/python3.8/dist-packages/poptorch/_poplar_executor.py:793 [08:59:07.364] [poptorch:cpp] [trace] Adding 0x5fc3900 to value mapper 0x705fa38, JIT ir: 14 [08:59:07.364] [poptorch:cpp] [debug] [DISPATCHER] Intercepting aten::copy_ [08:59:07.364] [poptorch:cpp] [trace] [Input] self impl_ 0x5fc3900 type ipu ID 2 sizes [1] dtype float [08:59:07.364] [poptorch:cpp] [trace] [Input] src impl_ 0x7057cc0 type cpu sizes [1] dtype float [08:59:07.364] [poptorch:cpp] [trace] Setting op source to: /usr/local/lib/python3.8/dist-packages/poptorch/_poplar_executor.py:793 [08:59:07.364] [poptorch:cpp] [trace] [DISPATCHER] Copying from CPU tensor 0x7057cc0 with data_ptr 0x7029c40 [08:59:07.364] [poptorch:cpp] [trace] [DISPATCHER] Adding input: Value 0x7066ff0 with cpu ptr 0x7029c40 [08:59:07.364] [poptorch:cpp] [trace] Tracking tensor %15 [08:59:07.364] [poptorch:cpp] [trace] Adding 0x5fc3900 to value mapper 0x705fa38, JIT ir: 15 [08:59:07.364] [poptorch:cpp] [debug] copy_ CPU -> IPU input, new self impl_ 0x5fc3900 type ipu ID 2 sizes [1] dtype float [08:59:07.364] [poptorch:cpp] [trace] exchangeDevice: current ipu:0 new ipu:0 [08:59:07.364] [poptorch:cpp] [debug] [DISPATCHER] Intercepting aten::select.int(Tensor(a) self, int dim, int index) -> Tensor(a) [08:59:07.364] [poptorch:cpp] [trace] Setting op source to: sst.py:15 [08:59:07.364] [poptorch:cpp] [trace] [Input aten::select] impl_ 0x5fc3900 type ipu ID 2 sizes [1] dtype float [08:59:07.364] [poptorch:cpp] [trace] [Input aten::select] 0 [08:59:07.364] [poptorch:cpp] [trace] [Input aten::select] 0 [08:59:07.364] [poptorch:cpp] [trace] [DISPATCHER][JIT] Looking for inplace arguments in schema aten::select.int(Tensor(a) self, int dim, int index) -> Tensor(a) [08:59:07.364] [poptorch:cpp] [trace] [DISPATCHER] Tensor input: tensor ptr 0x5fc3900 (sizes=[1], type=Float), jit ir %15 (scalar type Float) [08:59:07.364] [poptorch:cpp] [trace] [DISPATCHER] Constant input: jit ir %16, ivalue tag kind: Int [08:59:07.364] [poptorch:cpp] [trace] [DISPATCHER] Constant input: jit ir %17, ivalue tag kind: Int [08:59:07.364] [poptorch:cpp] [trace] [DISPATCHER][JIT] Node from schema %18 : Tensor = aten::select(%15, %16, %17) # sst.py:15:0 [08:59:07.364] [poptorch:cpp] [trace] [DISPATCHER] Using meta type inference for aten::select.int [08:59:07.364] [poptorch:cpp] [trace] Created IPU tensor: id 3 impl_ 0x5ffd2a0 size [] strides dtype Float [08:59:07.364] [poptorch:cpp] [trace] Adding 0x5ffd2a0 to value mapper 0x705fa38, JIT ir: 18 [08:59:07.364] [poptorch:cpp] [trace] [DISPATCHER][JIT] Output: Tensor ptr 0x5ffd2a0, jit ir %18 (scalar type Float) [08:59:07.364] [poptorch:cpp] [trace] [DISPATCHER][JIT] Pre canonicalisation %18 : Float(requires_grad=0, device=ipu:0) = aten::select(%15, %16, %17) # sst.py:15:0 [08:59:07.364] [poptorch:cpp] [trace] [DISPATCHER][JIT] Node tensor output at index 0 size: =[] [08:59:07.364] [poptorch:cpp] [trace] [Output aten::select] impl_ 0x5ffd2a0 type ipu ID 3 sizes [] dtype float [08:59:07.364] [poptorch:cpp] [trace] uncheckedSetDevice: current ipu:0 new ipu:0 [08:59:07.364] [poptorch:cpp] [trace] exchangeDevice: current ipu:0 new ipu:0 [08:59:07.364] [poptorch:cpp] [debug] [DISPATCHER] Intercepting aten::item(Tensor self) -> Scalar [08:59:07.364] [poptorch:cpp] [trace] Setting op source to: sst.py:15 [08:59:07.364] [poptorch:cpp] [trace] [Input aten::item] impl_ 0x5ffd2a0 type ipu ID 3 sizes [] dtype float [08:59:07.364] [poptorch:cpp] [trace] [DISPATCHER][JIT] Looking for inplace arguments in schema aten::item(Tensor self) -> Scalar [08:59:07.364] [poptorch:cpp] [trace] [DISPATCHER] Tensor input: tensor ptr 0x5ffd2a0 (sizes=[], type=Float), jit ir %18 (scalar type Float) [08:59:07.364] [poptorch:cpp] [trace] [DISPATCHER][JIT] Node from schema %19 : Tensor = aten::item(%18) # sst.py:15:0 [08:59:07.364] [poptorch:cpp] [trace] [DISPATCHER] Using meta type inference for aten::item [08:59:07.373] [poptorch:cpp] [trace] uncheckedSetDevice: current ipu:0 new ipu:0 [08:59:07.374] [poptorch::python] [critical] NotImplementedError: Could not run 'aten::_local_scalar_dense' with arguments from the 'Meta' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for possible resolutions. 'aten::_local_scalar_dense' is only available for these backends: [CPU, IPU, BackendSelect, Python, FuncTorchDynamicLayerBackMode, Functionalize, Named, Conjugate, Negative, ZeroTensor, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradHIP, AutogradXLA, AutogradMPS, AutogradIPU, AutogradXPU, AutogradHPU, AutogradVE, AutogradLazy, AutogradMeta, AutogradPrivateUse1, AutogradPrivateUse2, AutogradPrivateUse3, AutogradNestedTensor, Tracer, AutocastCPU, AutocastCUDA, FuncTorchBatched, FuncTorchVmapMode, Batched, VmapMode, FuncTorchGradWrapper, PythonTLSSnapshot, FuncTorchDynamicLayerFrontMode, PythonDispatcher]. CPU: registered at aten/src/ATen/RegisterCPU.cpp:30798 [kernel] IPU: registered at /opt/jenkins/workspace/poptorch/poptorch_ci_ubuntu_20_04/unprivileged/poptorch/poptorch/source/dispatch_tracer/RegisterOptionalAtenOps.cpp.inc:3 [kernel] BackendSelect: fallthrough registered at ../aten/src/ATen/core/BackendSelectFallbackKernel.cpp:3 [backend fallback] Python: registered at ../aten/src/ATen/core/PythonFallbackKernel.cpp:140 [backend fallback] FuncTorchDynamicLayerBackMode: registered at ../aten/src/ATen/functorch/DynamicLayer.cpp:488 [backend fallback] Functionalize: registered at ../aten/src/ATen/FunctionalizeFallbackKernel.cpp:291 [backend fallback] Named: fallthrough registered at ../aten/src/ATen/core/NamedRegistrations.cpp:11 [kernel] Conjugate: registered at ../aten/src/ATen/ConjugateFallback.cpp:18 [backend fallback] Negative: registered at ../aten/src/ATen/native/NegateFallback.cpp:18 [backend fallback] ZeroTensor: registered at ../aten/src/ATen/ZeroTensorFallback.cpp:86 [backend fallback] ADInplaceOrView: fallthrough registered at ../aten/src/ATen/core/VariableFallbackKernel.cpp:64 [backend fallback] AutogradOther: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:16899 [autograd kernel] AutogradCPU: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:16899 [autograd kernel] AutogradCUDA: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:16899 [autograd kernel] AutogradHIP: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:16899 [autograd kernel] AutogradXLA: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:16899 [autograd kernel] AutogradMPS: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:16899 [autograd kernel] AutogradIPU: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:16899 [autograd kernel] AutogradXPU: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:16899 [autograd kernel] AutogradHPU: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:16899 [autograd kernel] AutogradVE: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:16899 [autograd kernel] AutogradLazy: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:16899 [autograd kernel] AutogradMeta: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:16899 [autograd kernel] AutogradPrivateUse1: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:16899 [autograd kernel] AutogradPrivateUse2: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:16899 [autograd kernel] AutogradPrivateUse3: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:16899 [autograd kernel] AutogradNestedTensor: registered at ../torch/csrc/autograd/generated/VariableType_2.cpp:16899 [autograd kernel] Tracer: registered at ../torch/csrc/autograd/generated/TraceType_2.cpp:16890 [kernel] AutocastCPU: fallthrough registered at ../aten/src/ATen/autocast_mode.cpp:482 [backend fallback] AutocastCUDA: fallthrough registered at ../aten/src/ATen/autocast_mode.cpp:324 [backend fallback] FuncTorchBatched: registered at ../aten/src/ATen/functorch/BatchRulesDynamic.cpp:64 [kernel] FuncTorchVmapMode: fallthrough registered at ../aten/src/ATen/functorch/VmapModeRegistrations.cpp:28 [backend fallback] Batched: registered at ../aten/src/ATen/BatchingRegistrations.cpp:1064 [backend fallback] VmapMode: fallthrough registered at ../aten/src/ATen/VmapModeRegistrations.cpp:33 [backend fallback] FuncTorchGradWrapper: registered at ../aten/src/ATen/functorch/TensorWrapper.cpp:189 [backend fallback] PythonTLSSnapshot: registered at ../aten/src/ATen/core/PythonFallbackKernel.cpp:148 [backend fallback] FuncTorchDynamicLayerFrontMode: registered at ../aten/src/ATen/functorch/DynamicLayer.cpp:484 [backend fallback] PythonDispatcher: registered at ../aten/src/ATen/core/PythonFallbackKernel.cpp:144 [backend fallback] ```

Expected Behaviour i = int(y[0]) or i = y.item() successfully results ini == 2

AnthonyBarbier commented 1 year ago

item() always returns a CPU scalar which is why you can't use it in your non-CPU (i.e Meta or IPU) graph.

The only concept of device side scalar Pytorch has is using a zero size tensor or torch.scalar_tensor which essentially is the same thing (But for some reason doesn't seem to be documented).

Hope this helps

Bedledl commented 1 year ago

Hi:) thank you for the quick answer.

I discovered that i can use the length of a tensor as argument for an integer paramter. For example as argument for the ones() function:

class SimpleAdder(nn.Module):
    def forward(self, x, y):
        z = torch.ones(len(y) + len(x))
        z = z + x[0] # i must make the model non-trivial
        return z

model = SimpleAdder()
inference_model = poptorch.inferenceModel(model)

t1 = torch.tensor([1., 6., 3.])
t2 = torch.tensor([5, 8, 4, 6])

Is this, because the shape of a tensor is determined at compile time, while the contents of the tensor are abritrary?