search

parrt / tensor-sensor

The goal of this library is to generate more helpful exception messages for matrix algebra expressions for numpy, pytorch, jax, tensorflow, keras, fastai.
https://github.com/parrt/tensor-sensor
MIT License
794 stars 40 forks source link

various enhancements #1

Open koke2c95 opened 4 years ago

koke2c95 commented 4 years ago

great works!

debug these math expression code is a very big problem

not only scalar vector matrix , high rank tensor support

Visualizing 3D tensors and beyond:

TensorDim is contain : left to right [N,C,H,W] N->C->H->W TensorDim like flatten Onions,cabbage box in box represent dim TensorDim has real represention information (N,C,H,W etc.) image

namedtensor support

plaidml dsl Types Tensor TensorDim TensorIndex

represent actual data: audio, 1D plot, text : vector image ,2D plot : matrix etc.

expression graph:

different color input variable ,leaf parameters/variable with temporal variable / activation : "Road width" elementwise leaf / elementwise or slice or other connection(edge) print AST forward & backward mode

animation:

expression step computation mode (debug) slice ,reshape, .T() other manipulate N-d array TensorDim/TensorIndex operators example : matmul = m@v m@v = (v.squeeze(0).expand_as(w) * w).sum(1,keepdim=True).unsqueeze(1) image more ... conv2d etc

interactive:

interactive build block(visual programing) reverse interactive(debug) : selected tensor elements and follow it in expression graph selected element : muti view , like convolution-visualizer Input(Inputgrad) Weight(Weightgrad) Output(Outputgrad) views

NN support:

NN module visualization (conv2d) bigger computation graph : pytorchrec multi computation graph visualization and live debug

other useful link

SimpNet memory map image Visualize the virtual address space of a Windows process on a Hilbert curve.

nvidia ppt image

tensor network image

einops

parrt commented 4 years ago

hi! thanks for all the ideas and links. have you ever seen a good basic library for animating things like matrices?

koke2c95 commented 4 years ago

hi! thanks for all the ideas and links. have you ever seen a good basic library for animating things like matrices?

Sorry for the late reply to you. I search a lot , but I haven't seen visualization library like this idea.

now

back to basics.

The Ndarray/Tensor is the thing that stores data, values visualization by Think of each value as a pixel And those tensor operations are essentially is pixel coordinates in the operating space it can shuffle, drop, add any pixel

previous

you can make a visualizing by nest different type (dim Information & tensor data binding) example like. muti same image nest in a matrix eq. grid image and you can infinity nesting

(0,0) = R scalar
(0,1) = R vector
(0,2) = R matrix
(0,~) ~>=3 : R tensor
(1,2) = gray image
(3,2) = rgb image
(0,1,2) = 2 dim R field R^2
(0,2,2) = 2 dim C field C^2