search

pmh47 / dirt

DIRT: a fast differentiable renderer for TensorFlow
MIT License
312 stars 63 forks source link

can I use placeholder(None,None,3)to feed images or render image with unknown size #72

Closed ffiioonnaa closed 4 years ago

ffiioonnaa commented 4 years ago

Hi, I want to render images with different size by making multiple calls to rasterise , and use placeholder (None,None,3)to feed image or read from file, but it seems that dirt needs to konw the size of image before feed image.I want to keep the raw size of images, rather than resize them to a same size,Did I do something wrong? or is there any way to do that?

ffiioonnaa commented 4 years ago

I find you write this in rasterise_ops.py:

height: a python int specifying the frame height; may be None if background has static shape width: a python int specifying the frame width; may be None if background has static shape channels: a python int specifying the number of color channels; may only be 1or 3. Again this may be None if background has static shape

But in my case ,I read the image and its shape from a tfrecord file, so the shape of image is tf.int type.Do you have any suggestions?

pmh47 commented 4 years ago

Currently it is not possible for the output shape to be dynamic -- as you read in the docstring, it must be given as a python integer, or defined by the static shape of background.

The usual solution would just be to resize the input images to a standard size (usually one wants this anyway so they can be put in batches). Alternatively, you can render at a (fixed) higher resolution, then downsample different renderings to different sizes in tf.

ffiioonnaa commented 4 years ago

Thank you~ And can tensorflow eager mode be used with dirt to solve this problem?

pmh47 commented 4 years ago

Yes, in theory, but it will be very resource-intensive, as every time a call to rasterise is made with a new image size, it'll allocate a new OpenGL context (and won't deallocate it afterwards). You can try and see what happens though.

An alternative solution, if you know/fix the maximum possible image size (or a size beyond which you decide just to resize down), is to pass that large size to rasterise, but modify the projection matrix to render the object in the top-left area of the image. Then, crop afterwards in tf to match the input image. The modified projection matrix would have to have different x/y diagonal elements (scaled according to the image size), and non-zero x/y translation (to shift the camera centre to somewhere nearer the top left).

ffiioonnaa commented 4 years ago

Thanks,I'll try them. I think we can close the issue