Open zzhuolun opened 1 year ago
Hi, can you share the code you used to create these images? The depth mask files have to be loaded with our custom loading functions.
from omegaconf import DictConfig
from pytorch3d.implicitron.dataset.json_index_dataset_map_provider_v2 import (
JsonIndexDatasetMapProviderV2,
)
from pytorch3d.implicitron.tools.config import expand_args_fields
import matplotlib.pyplot as plt
expand_args_fields(JsonIndexDatasetMapProviderV2)
dataset_map = JsonIndexDatasetMapProviderV2(
category="car",
subset_name="fewview_test",
test_on_train=False,
only_test_set=False,
load_eval_batches=True,
dataset_root=DATASET_ROOT,
dataset_JsonIndexDataset_args=DictConfig(
{
"remove_empty_masks": False,
"load_point_clouds": True,
"dataset_root": DATASET_ROOT,
}
),
).get_dataset_map()
train_dataset = dataset_map["train"]
N = 5
fig, ax = plt.subplots(nrows=4, ncols=N)
for i in range(N):
data = train_dataset[i * 200]
ax[0][i].imshow(data.image_rgb.permute(1, 2, 0).numpy())
ax[1][i].imshow(data.fg_probability.permute(1, 2, 0).numpy())
ax[2][i].imshow(data.depth_map.permute(1, 2, 0).numpy())
ax[3][i].imshow(data.depth_mask.permute(1, 2, 0).numpy())
for j in range(4):
ax[j][i].axes.xaxis.set_visible(False)
ax[j][i].axes.yaxis.set_visible(False)
plt.show()
Hi,
thanks for the reply! I used the JsonIndexDataset
to load the co3d dataset and the above code produces an images like this:
Hi, thanks a lot for spotting this. Indeed something is wrong with the depth masks. Please bear with us while we issue a fix. Should be done by end of this week.
Hello, I want to know that progress of this issue. Is it resolved or working in progress?
Hi apologies for the delay here, the a new version with correct masks is currently uploaded to the mirror. We are running last checks and should be able to release by beginning of the next week.
I also found that the depth masks in CO3D v1 are better.
Hi! I was wondering how I could find few sequences with correct/better depth masks (in the meanwhile)? I am currently facing this issue of incorrect depth masks within laptop
and apple
categories.
Also, is there a way to patch the holes in the depth masks (0 values)? I made sure to use the correct 16-bit depth masks loading functions. My current output looks like below.
Appreciate your help!
Hi, unfortunately, the depth masks are not compatible across datasets. Apologies again for the delays, re-uploading the whole dataset is taking longer than expected ...
Hi, sorry for a late fix. We have now fixed the depth masks. Please re-download the dataset (you might notice that links.json and the sha256 json have changed).
Hi, Many, if not all, depth masks are wrong, as shown in the image. I only checked the car and orange category, but I believe in other categories depth masks are also wrong. I also found that the depth masks in CO3D v1 are better.