How to visual test results?

[ivy94419]

Hello @yangxuntu , Thank you for your repo of visual relationship detection, it's easy for new one to get started!!

I have followed your steps to train an model in VRD dataset, and run _test_vrd_vggpred.py and _eva_vrd_vgg_pred.py_ on my vrd_vgg0010.ckpt.

I want to know how to visual the test results such as the samples that paper gives :

Thank you!

[yangxuntu]

You need to write it by yourself. I do not provide that implementation, you can do this like the code provided here: https://visualgenome.org/api/v0/api_beginners_tutorial.html

---

发件人: ivy94419 notifications@github.com 发送时间: 2018年7月30日 17:17:29 收件人: yangxuntu/vrd 抄送: #YANG XU#; Mention 主题: [yangxuntu/vrd] How to visual test results? (#5)

Hello @yangxuntuhttps://github.com/yangxuntu , Thank you for your repo of visual relationship detection, it's easy for new one to get started!!

I have followed your steps to train an model in VRD dataset, and run test_vrd_vgg_pred.py and eva_vrd_vgg_pred.py on my vrd_vgg0010.ckpt.

I want to know how to visual the test results such as the samples that paper gives : [image]https://user-images.githubusercontent.com/4054390/43388635-3f09329a-941c-11e8-949b-ee6137205b6d.png

Thank you!

― You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/yangxuntu/vrd/issues/5, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AkiaOpFQcjUGjKpzwSqvj7V-TmsHbp8uks5uLs8pgaJpZM4VmDqU.

[xixd]

@ivy94419 hello,can you visual the result now

[ivy94419]

@xixd Haven't yet since these days I am busy... I will try it later

[yangxuntu]

You can use this file to visual the ground truth of vrd dataset:https://github.com/yangxuntu/vrd/blob/master/test3.py

[xixd]

@ivy94419 @yangxuntu thank you!!

[ivy94419]

@yangxuntu @xixd Hey, this code can visual predict result with this effect, and save this pic to 'vtranse/vis/xx.jpg'.

But you need to

1. In _test_vrd_vggpred.py, add 'image': roidb_use['image'] in _pred_roidbtemp
2. Find a proper fonts and change the path to fonts font = ImageFont.truetype("C:/Windows/Fonts/arial.ttf", 20)
3. Create dir 'vtranse/vis'

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

from model.ass_fun import *
import cv2
import json
from PIL import Image, ImageDraw, ImageFont

object_json = 'vtranse/input/objects.json'
predicate_json = 'vtranse/input/predicates.json'

with open(object_json, 'r') as f:
    object_dict = json.load(f)
with open(predicate_json, 'r') as f:
    predicate_dict = json.load(f)

subject_color = (0, 0, 255)
object_color = (255, 0, 0)
rela_color = (0, 0, 0)
font = ImageFont.truetype("C:/Windows/Fonts/arial.ttf", 20)

pred_path = cfg.DIR + 'vtranse/pred_res/vrd_pred_roidb.npz'
res_path = cfg.DIR + 'vtranse/pretrained_para/vgg_pretrained.ckpt'

pred_read = read_roidb(pred_path)
pred_roidb = pred_read['pred_roidb']

def drawbbox(raw_img, bbox, color):
    img = raw_img.copy()
    cv2.rectangle(img, (bbox[0], bbox[1]), (bbox[2], bbox[3]), color, 3)
    return img

def draw_text_img(subject, rela, object):
    rela = ' - ' + rela + ' - '
    s_img = cv2.cvtColor(np.asarray(drawtext(subject, subject_color[::-1])), cv2.COLOR_RGB2BGR)
    r_img = cv2.cvtColor(np.asarray(drawtext(rela, rela_color[::-1])), cv2.COLOR_RGB2BGR)
    o_img = cv2.cvtColor(np.asarray(drawtext(object, object_color[::-1])), cv2.COLOR_RGB2BGR)
    text_img = np.concatenate((s_img, r_img, o_img), 1)
    return text_img

def drawtext(text, color):
    w, h = font.getsize(text)
    white = Image.new('RGB', (w, 25), (255, 255, 255))
    draw = ImageDraw.Draw(white)
    draw.text((0, 0), text, color, font)
    return white

def draw_visual_relationship(roidb_use):
    img_name = roidb_use['image']
    raw_img = cv2.imread(img_name)
    sbox = roidb_use['sub_box_dete']
    obox = roidb_use['obj_box_dete']
    sb = roidb_use['sub_dete']
    ob = roidb_use['obj_dete']
    rela = roidb_use['pred_rela']

    N_box = len(sbox)  # totally N pairs of relationships  [subject - relation - object]

    for i in range(N_box):
        s, r, o = int(sb[i]), int(rela[i]), int(ob[i])
        s_l, s_t, s_r, s_d = list(map(int, sbox[i]))  # subject bbox [left, top, right, down]
        o_l, o_t, o_r, o_d = list(map(int, obox[i]))  # object bbox
        # calculate the bounding rectangle of subject and object bbox
        left = min(s_l, o_l)
        top = min(s_t, o_t)
        right = max(s_r, o_r)
        down = max(s_d, o_d)

        img = drawbbox(raw_img, [s_l, s_t, s_r, s_d], subject_color)
        img = drawbbox(img, [o_l, o_t, o_r, o_d], object_color)

        rect_img = img[top: down, left: right, :]
        rect_h, rect_w = rect_img.shape[:2]

        text_img = draw_text_img(object_dict[s], predicate_dict[r], object_dict[o])
        text_h, text_w = text_img.shape[:2]
        scale = rect_w / text_w

        text_img = cv2.resize(text_img, (rect_w, int(text_h*scale)))
        rect_img = np.concatenate((rect_img, text_img), 0)

        name = img_name.split('/')[-1].split('.')[0]  # extract image name
        cv2.imwrite('vtranse/vis/%s_%d.jpg' % (name, i), rect_img)

def main():
    N_pred = len(pred_roidb)
    for i in range(N_pred):
        print('====', i, '====')
        roidb_use = pred_roidb[i]
        draw_visual_relationship(roidb_use)

if __name__ == '__main__':
    main()

[yangxuntu]

Thanks to your assistance!!

I am very busy recently and sorry for lately response.

---

发件人: ivy94419 notifications@github.com 发送时间: 2018年8月6日 19:10:11 收件人: yangxuntu/vrd 抄送: #YANG XU#; Mention 主题: Re: [yangxuntu/vrd] How to visual test results? (#5)

@yangxuntuhttps://github.com/yangxuntu @xixdhttps://github.com/xixd Hey, this code can visual predict result with this effect, and save this pic to 'vtranse/vis/xx.jpg'.

But you need to

1. In test_vrd_vgg_pred.py, add 'image': roidb_use['image'] in pred_roidb_temp
2. Find a proper fonts and change the path to fonts font = ImageFont.truetype("C:/Windows/Fonts/arial.ttf", 20)
3. Create dir 'vtranse/vis'

[image]https://user-images.githubusercontent.com/4054390/43713291-a9b13d24-99ab-11e8-9622-b96d2bf2a130.png

from future import absolute_import from future import division from future import print_function

from model.ass_fun import * import cv2 import json from PIL import Image, ImageDraw, ImageFont

object_json = 'vtranse/input/objects.json' predicate_json = 'vtranse/input/predicates.json'

with open(object_json, 'r') as f: object_dict = json.load(f) with open(predicate_json, 'r') as f: predicate_dict = json.load(f)

subject_color = (0, 0, 255) object_color = (255, 0, 0) rela_color = (0, 0, 0) font = ImageFont.truetype("C:/Windows/Fonts/arial.ttf", 20)

pred_path = cfg.DIR + 'vtranse/pred_res/vrd_pred_roidb.npz' res_path = cfg.DIR + 'vtranse/pretrained_para/vgg_pretrained.ckpt'

pred_read = read_roidb(pred_path) pred_roidb = pred_read['pred_roidb']

def drawbbox(raw_img, bbox, color): img = raw_img.copy() cv2.rectangle(img, (bbox[0], bbox[1]), (bbox[2], bbox[3]), color, 3) return img

def draw_text_img(subject, rela, object): rela = ' - ' + rela + ' - ' s_img = cv2.cvtColor(np.asarray(drawtext(subject, subject_color[::-1])), cv2.COLOR_RGB2BGR) r_img = cv2.cvtColor(np.asarray(drawtext(rela, rela_color[::-1])), cv2.COLOR_RGB2BGR) o_img = cv2.cvtColor(np.asarray(drawtext(object, object_color[::-1])), cv2.COLOR_RGB2BGR) text_img = np.concatenate((s_img, r_img, o_img), 1) return text_img

def drawtext(text, color): w, h = font.getsize(text) white = Image.new('RGB', (w, 25), (255, 255, 255)) draw = ImageDraw.Draw(white) draw.text((0, 0), text, color, font) return white

def draw_visual_relationship(roidb_use): img_name = roidb_use['image'] raw_img = cv2.imread(img_name) sbox = roidb_use['sub_box_dete'] obox = roidb_use['obj_box_dete'] sb = roidb_use['sub_dete'] ob = roidb_use['obj_dete'] rela = roidb_use['pred_rela']

N_box = len(sbox)  # totally N pairs of relationships  [subject - relation - object]

for i in range(N_box):
    s, r, o = int(sb[i]), int(rela[i]), int(ob[i])
    s_l, s_t, s_r, s_d = list(map(int, sbox[i]))  # subject bbox [left, top, right, down]
    o_l, o_t, o_r, o_d = list(map(int, obox[i]))  # object bbox
    # calculate the bounding rectangle of subject and object bbox
    left = min(s_l, o_l)
    top = min(s_t, o_t)
    right = max(s_r, o_r)
    down = max(s_d, o_d)

    img = drawbbox(raw_img, [s_l, s_t, s_r, s_d], subject_color)
    img = drawbbox(img, [o_l, o_t, o_r, o_d], object_color)

    rect_img = img[top: down, left: right, :]
    rect_h, rect_w = rect_img.shape[:2]

    text_img = draw_text_img(object_dict[s], predicate_dict[r], object_dict[o])
    text_h, text_w = text_img.shape[:2]
    scale = rect_w / text_w

    text_img = cv2.resize(text_img, (rect_w, int(text_h*scale)))
    rect_img = np.concatenate((rect_img, text_img), 0)

    name = img_name.split('/')[-1].split('.')[0]  # extract image name
    cv2.imwrite('vtranse/vis/%s_%d.jpg' % (name, i), rect_img)

def main(): N_pred = len(pred_roidb) for i in range(N_pred): print('====', i, '====') roidb_use = pred_roidb[i] draw_visual_relationship(roidb_use)

if name == 'main': main()

― You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/yangxuntu/vrd/issues/5#issuecomment-410673467, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AkiaOvzAiZuPm6Tg2yaUvrqay6ggtQpdks5uOCQTgaJpZM4VmDqU.

[17000432]

@ivy94419 After training and testing with my own data set, I reported an error while executing the visual script. Error : list out of range .How to solve it?Thanks very much.

[CherishineNi]

I want to know how to visual the test results such as :

how to do ? @yangxuntu @yangxuntu @xixd @17000432 @ivy94419

[yangxuntu]

I am not the author of this paper, you need to ask the author of this paper.

---

发件人: CherishineNi notifications@github.com 发送时间: 2019年4月12日 11:00 收件人: yangxuntu/vrd 抄送: #YANG XU#; Mention 主题: Re: [yangxuntu/vrd] How to visual test results? (#5)

I want to know how to visual the test results such as : [微信图片_20190412105754]https://user-images.githubusercontent.com/44666138/56009347-18e58400-5d12-11e9-9a4b-500a0a7cf2b1.png

how to do ? @yangxuntuhttps://github.com/yangxuntu @yangxuntuhttps://github.com/yangxuntu @xixdhttps://github.com/xixd @17000432https://github.com/17000432

― You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/yangxuntu/vrd/issues/5#issuecomment-482418412, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AkiaOl08jzLnEsLP3-k2SxDy4MVztincks5vf_bQgaJpZM4VmDqU.