Closed LennardBo closed 1 year ago
Hi Lennard, thanks for your interest in my work! You are indeed right in your comprehension of the voxel cubes, it is unfortunately the code that is wrong... However, the results of the paper were indeed achieved with binary voxel cubes.
For a little context, initially I used the MinkowskiEngine library to handle the sparse tensors and generate the datasets, and the coalesce function of MinkowskiEngine does not sum the features. The paper results were obtained with this approach. Before making this repository available, to make the code less dependent on an external library I replaced the MinkowskiEngine functions with torch.sparse functions. I remember that some time after I encountered the same issue as you where the data is non-binary, and I must have forgot to update this repo.
I am sorry to have mislead you and probably others, you can see in my most recent work that the intended way of constructing binary samples is to:
sparse_tensor = torch.sparse_coo_tensor(
coords.t().to(torch.int32),
feats,
(self.T, quantized_h, quantized_w, self.C),
)
sparse_tensor = sparse_tensor.coalesce().to(torch.bool)
- getitem() now simply returns `self.samples[index][0].to_dense().permute(0,3,1,2), self.samples[index][1]` (the dense sample and the target)
I will quickly push a PR to correct this, thank you again.
Ah, thanks for the explanation! Honestly, it's a rarity that these kind of revolutionary papers have any good code attached to it so I really appreciate your effort. I guess I make it a boolean tensor then, as well.
sparse_tensor = sparse_tensor.coalesce().to(torch.bool) throws the following error:
RuntimeError: "coalesce" not implemented for 'Bool'
How do I solve it?
It should be fixed in commit 6517e74, sorry for the inconvenience.
return sparse_tensor.to_dense().permute(0,3,1,2), target
throws the following error:
RuntimeError: "coalesce" not implemented for 'Bool'
How do I fix this?
return sparse_tensor.to_dense().permute(0,3,1,2), target
throws the following error:RuntimeError: "coalesce" not implemented for 'Bool'
How do I fix this?
Where you ever able to fix this issue? I am at this point now.
For my team's internal training pipeline I've derived a dataset based on the GEN1DetectionDataset code. When I load the data with the same hyperparameters (tbin, T, sample_size etc.) I've noticed that once I coalesce the sparse data as in L42, I loose the binary structure of the data and receive values bigger than 1. My assumption was that due to the quantizing nature of the tbins there simply were multiple events falling into one pixel bin (voxel cubes as in the paper) and as per Pytorch's documentation regarding coalescing duplicate values are simply summed up when coalescing sparse COO tensors.
However, the paper explicitly talks of binary voxel cubes which makes sense since we want to treat them as spikes for the SNNs. I fail to see where in the code this non-bianry coalesced data is actually made into binary data again. Am I missing something? Does the data originally not contain any events falling into the same time bin or do you use here actually non-binary data?