adding real time script

[noumanriazkhan]

Just change input, output and checkpoint path.

The script reads all the images in the folder and saves combined segmentation and pose results in output folder.

[cemoody]

I couldn't get this script to work due to what appears to be shape mismatches. I tried putting in 5 images (it seems to expect the first dim to be 5?):

images_path = glob.glob('*fZ4pkV8.png')
out_path = '.'
input_feed_shape = (5, config.TAR_W, config.TAR_H, 3)

also putting in a:

img = np.tile(img, (5, 1, 1, 1))
preds = sess.run(res, feed_dict={img_tf: img} )

and then rewrote the load checkpoint (I couldn't find slim.d):

ifunc, idict = slim.assign_from_checkpoint(checkpoint_path, restore_map)
sess = tf.Session()
sess.run(ifunc, idict)

but still am having shape issues before giving up:

InvalidArgumentError (see above for traceback): flat indices[1976751, :] = [4, 51, 51, 9] does not index into param (shape: [5,51,51,17]).
     [[Node: GatherNd_1 = GatherNd[Tindices=DT_INT32, Tparams=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](Conv_3/BiasAdd, stack_8)]]

[noumanriazkhan]

@cemoody I should have mentioned that you need to set BATCH_SIZE=1 in config.py. Then try to re-run my code unchanged.

[noumanriazkhan]

slim.d() was a typo, I have fixed this. Thanks for mentioning.

[byebaibai]

After I changed BATCH_SIZE=1, I still couldn't run this code.

Here is the Traceback:

Thanks.

[noumanriazkhan]

Not sure why you are getting this, but one little hack is you can comment the lines dealing with 'lo_x' and 'lo_y' in model_base.py if you are not interested in segmentation but in keypoints only.

[Axxeption]

Hi! Thanks for this interesting code!

Unfortunately, I have exact the same error as @BaldwinHe. Is there a update on this? Indeed I can comment those with 'lo_x' and 'lo_y'. Wich is almost everything in the model_base.py. Plus, this feature is important for me...

Thanks in front!

[noumanriazkhan]

The code is working fine for me, not sure why the error is. However, I am pasting my config.py below, you can compare with this.

`import numpy as np

TAR_H = 256 TAR_W = 256

WORKER_SIZE = 8 IN_QUEUE_SIZE = 20 OUT_QUEUE_SIZE = 20

BATCH_SIZE = 1 PREFETCH_SIZE = 20 SHUFFLE_BUFFER_SIZE = 100 NUM_EPOCH = 100

NUM_RECURRENT = 2 NUM_RECURRENT_2 = 2 NUM_RECURRENT_3 = 2

RADIUS = 64 STRIDE = 8

KP_PEAK_LB = 5 KP_LINK_DIST = 16 KP_DUPLICATE_DIST = 8

KP_NAMES = [ 'nose', 'left_eye', 'right_eye', 'left_ear', 'right_ear', 'left_shoulder', 'right_shoulder', 'left_elbow', 'right_elbow', 'left_wrist', 'right_wrist', 'left_hip', 'right_hip', 'left_knee', 'right_knee', 'left_ankle', 'right_ankle' ] NUM_KP = len(KP_NAMES)

EDGES = np.array([ [15, 13], [13, 11], [16, 14], [14, 12], [11, 12], [ 5, 11], [ 6, 12], [ 5, 6], [ 5, 7], [ 6, 8], [ 7, 9], [ 8, 10], [ 1, 2], [ 0, 1], [ 0, 2], [ 1, 3], [ 2, 4], [ 3, 5], [ 4, 6] ]) EDGES = np.concatenate([EDGES, EDGES[:, ::-1]], axis=0) # bidirectional NUM_EDGE = len(EDGES)`

[Axxeption]

Thanks for your swift reply!

This doesn't change my problem, though.

Can you maybe tell which version of scipy you use? Because I had to install this as well.

[noumanriazkhan]

'1.1.0'

[Axxeption]

Hmmm, this seems to be the same as well. No idea where it goes wrong then.

[noumanriazkhan]

Actually the problem is somewhat of shape mismatch, needs debugging in base_model.py

[Axxeption]

Okey, just to be sure can you maybe print out all your installed packages with PIP? And what is you image you test this with?

Is it possible you run this on a GPU? Because this issue says that the checking does noet happen on GPU, but it does on CPU. (https://github.com/tensorflow/tensorflow/issues/15091)

Thanks!

[Axxeption]

Update @BaldwinHe and @noumanriazkhan:

I tried to run the code with Tensorflow GPU and this worked, this is because Tensorflow GPU does less assertions. So actually there is a shape mismatch but wich is ignored now.

[noumanriazkhan]

Well, Well....never knew CPU/GPU support can make such difference. Anyhow, happy estimating pose!

[Axxeption]

Thanks! This works really great.

One last question: in my solution, the people are highlighted (see pic) but there is no difference between people. Is it possible to get this information out of this model?

[noumanriazkhan]

So, what you are asking is instance segmentation while this model outputs semantic levels of segmentation. The code (or paper as well, may be) does not perform instance segmentation but I think one can do this by medium offsets (mo_x, mo_y) hough voting with segmentation heatmap. But what I will do first is playing with mask image. Like the binary image operations to do something with connected pixels.

[sydsim]

Just change input, output and checkpoint path.

The script reads all the images in the folder and saves combined segmentation and pose results in output folder.

thanks for your code, but it seems too unstandardized. it seems to be work only on your specific environment. actually, I don't understand why all you needs a executable script. you can understand structure by reading jupyter notebook, and build your own script, right? I will consider adding a easy, excutable script when I have some spare time, but not with your code. sorry.

[sydsim]

So, what you are asking is instance segmentation while this model outputs semantic levels of segmentation. The code (or paper as well, may be) does not perform instance segmentation but I think one can do this by medium offsets (mo_x, mo_y) hough voting with segmentation heatmap. But what I will do first is playing with mask image. Like the binary image operations to do something with connected pixels.

I didn't implement it in this repo but the paper includes human segmentation. please stop sharing wrong information.

[sydsim]

Thanks! This works really great.

One last question: in my solution, the people are highlighted (see pic) but there is no difference between people. Is it possible to get this information out of this model?

you can do human segmentation with long offset estimation. it's not implemented in this repository. please check the paper.