scanner-research / scanner

Efficient video analysis at scale
https://scanner-research.github.io/
Apache License 2.0
620 stars 108 forks source link

defining computational graph #222

Closed PushyamiKaveti closed 6 years ago

PushyamiKaveti commented 6 years ago

Hi, I have video data which is a synced video from 4 different views ( A quad splitscreen ). I want to run pose detection on one view and at the same time save the frames of individual views separately. So, I defined a graph in scanner which will crop the four quadrants in the frame and apply pose detection to one of the quadrants. Below is the excerpt from the code.

The problem here is that if I use for example output_table.column("crop3").save_mp4() , a get a video of the cropped quadrant3. however, if I use output_table.column("crop3").load() or output_table.load([ "crop3"]) like in the code below to iterate over the frames and save them individually I always get the frame that has the pose drawn on it. i.e as per the code below sampled_frames stream.

Am I doing something wrong with the way I am defining the graph? or is it a bug?

# custom operator to crop
@scannerpy.register_python_op()
def crop_fn(config, frame: FrameType) -> FrameType:

    n = frame[config.args['y_start'] : config.args['y_end'], config.args['x_start']:config.args['x_end'], :]
    return n

#computational graph
frame = db.sources.FrameColumn()
rng = db.streams.Gather(frame)

# crop1
crop_frame_fn = db.ops.crop_fn(frame=rng, x_start = 0 ,x_end = 640, y_start=360, y_end=720)

#crop2
crop2_frame_fn = db.ops.crop_fn(frame=rng, x_start = 0 ,x_end = 640, y_start=0, y_end=360)

#crop3
crop3_frame_fn = db.ops.crop_fn(frame=rng, x_start = 640 ,x_end = 1280, y_start=360, y_end=720)
#crop4
crop4_frame_fn = db.ops.crop_fn(frame=rng, x_start = 640 ,x_end = 1280, y_start=0, y_end=360)

poses_out = db.ops.OpenPose(frame=crop_frame_fn, device=device, args=pose_args, batch=10)

drawn_frame = db.ops.PoseDraw(frame=crop_frame_fn, frame_poses=poses_out)

sampled_frames = sampler(drawn_frame)
output = db.sinks.Column(columns={'frame': sampled_frames , "pose": poses_out ,
                                  'crop1' : crop_frame_fn , "crop2" : crop2_frame_fn ,
                                  "crop3" : crop3_frame_fn , "crop4" : crop4_frame_fn })

 for pos in output_table.load(["frame" , "pose", "crop1" , "crop2" , "crop3", "crop4"]):
        frame , p, f,b,s,r = pos
        <code>
         #some other code
       <code>
      # saving the individual frames to disk
       cv2.imwrite("crop1"+"str(C) +".jpg"), f)
       cv2.imwrite("crop2"+"str(C) +".jpg"), r)
       cv2.imwrite("crop3"+"str(C) +".jpg"), b) 
       cv2.imwrite("crop4"+"str(C) +".jpg"), r)   
willcrichton commented 6 years ago

Hi, thanks for the bug report! This was a caching error on our side, I fixed it in 520e23bde430ca59cc4926d12e773a831964efa7.