OBJ converting problem

[SilentPrayeCG]

Hi. I recently tried to convert OBJ with pretty dense mesh, to vox, and it converted successfully. I used --gs 1995, and it covered all world, as i wanted. I don't remember which version i used.

Now i made another OBJ with less dense mesh, and for some reason it just throw bunch of exceptions, and begin converting with gs about 1000. I don't think it something with latest version, i tried few previous, and also previous large OBJ still start converting ok. I don't get, whats exactly wrong.

At first it gave me just bunch of info about index of array out of bound and stuff. r:\FileToVox>filetovox --i "r:\landscape2.obj" --o "r:\landscape_voxels" --gs 1995 --e [INFO] Specified input file: r:\landscape2.obj [INFO] Specified output file: r:\landscape_voxels [INFO] Specified increase size: 1 [INFO] Specified grid size: 1995 [INFO] Enabled option: excavate [INFO] Specified output path: r:\landscape_voxels System.AggregateException: One or more errors occurred. (Index was outside the bounds of the array.) (Index was outside the bounds of the array.) (Index was outside the bounds of the array.) ---> System.IndexOutOfRangeException: Index was outside the bounds of the array. at g3.MeshSignedDistanceGrid.<>c__DisplayClass41_0.<make_level_set3_parallel>b__0(Int32 tid) at System.Threading.Tasks.Parallel.<>c__DisplayClass44_02.b1(IEnumerator& partitionState, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion) --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Exception source) at System.Threading.Tasks.Parallel.<>cDisplayClass44_02.<PartitionerForEachWorker>b__1(IEnumerator& partitionState, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion) at System.Threading.Tasks.TaskReplicator.Replica.Execute() --- End of inner exception stack trace --- at System.Threading.Tasks.TaskReplicator.Run[TState](ReplicatableUserAction1 action, ParallelOptions options, Boolean stopOnFirstFailure) at System.Threading.Tasks.Parallel.PartitionerForEachWorker[TSource,TLocal](Partitioner1 source, ParallelOptions parallelOptions, Action1 simpleBody, Action2 bodyWithState, Action3 bodyWithStateAndIndex, Func4 bodyWithStateAndLocal, Func5 bodyWithEverything, Func1 localInit, Action1 localFinally) --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Exception source) at System.Threading.Tasks.Parallel.ThrowSingleCancellationExceptionOrOtherException(ICollection exceptions, CancellationToken cancelToken, Exception otherException) at System.Threading.Tasks.Parallel.PartitionerForEachWorker[TSource,TLocal](Partitioner1 source, ParallelOptions parallelOptions, Action1 simpleBody, Action2 bodyWithState, Action3 bodyWithStateAndIndex, Func4 bodyWithStateAndLocal, Func5 bodyWithEverything, Func1 localInit, Action1 localFinally) at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IEnumerable1 source, ParallelOptions parallelOptions, Action1 body, Action2 bodyWithState, Action3 bodyWithStateAndIndex, Func4 bodyWithStateAndLocal, Func5 bodyWithEverything, Func1 localInit, Action1 localFinally) at System.Threading.Tasks.Parallel.ForEach[TSource](IEnumerable1 source, Action1 body) at g3.MeshSignedDistanceGrid.make_level_set3_parallel(Vector3f origin, Single dx, Int32 ni, Int32 nj, Int32 nk, DenseGrid3f distances, Int32 exact_band) at g3.MeshSignedDistanceGrid.Compute() at FileToVox.Converter.OBJToSchematic.WriteSchematic() in E:\Documents\FileToVox\SchematicToVoxCore\Converter\ObjToSchematic.cs:line 34 at FileToVox.Program.ProcessFile() in E:\Documents\FileToVox\SchematicToVoxCore\Program.cs:line 254 ---> (Inner Exception #1) System.IndexOutOfRangeException: Index was outside the bounds of the array. at g3.MeshSignedDistanceGrid.<>cDisplayClass41_0.b0(Int32 tid) at System.Threading.Tasks.Parallel.<>cDisplayClass44_02.<PartitionerForEachWorker>b__1(IEnumerator& partitionState, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion) --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Exception source) at System.Threading.Tasks.Parallel.<>c__DisplayClass44_02.b1(IEnumerator& partitionState, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion) at System.Threading.Tasks.TaskReplicator.Replica.Execute()<---

---> (Inner Exception #2) System.IndexOutOfRangeException: Index was outside the bounds of the array. at g3.MeshSignedDistanceGrid.<>cDisplayClass41_0.b0(Int32 tid) at System.Threading.Tasks.Parallel.<>cDisplayClass44_02.<PartitionerForEachWorker>b__1(IEnumerator& partitionState, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion) --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Exception source) at System.Threading.Tasks.Parallel.<>c__DisplayClass44_02.b1(IEnumerator& partitionState, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion) at System.Threading.Tasks.TaskReplicator.Replica.Execute()<---

[LOG] Done.`

And then, when i just scaled down obj to size of "working" obj, it begin giving me much less exceptions, but still won't convert.

`r:\FileToVox>filetovox --i "r:\landscape3.obj" --o "r:\landscape_voxels" --gs 1995 --e [INFO] Specified input path: r:\landscape3.obj [INFO] Specified output path: r:\landscape_voxels [INFO] Specified increase size: 1 [INFO] Specified grid size: 1995 [INFO] Enabled option: excavate [INFO] Specified output path: r:\landscape_voxels System.OverflowException: Arithmetic operation resulted in an overflow. at g3.DenseGrid3f.resize(Int32 ni, Int32 nj, Int32 nk) at g3.MeshSignedDistanceGrid.make_level_set3_parallel(Vector3f origin, Single dx, Int32 ni, Int32 nj, Int32 nk, DenseGrid3f distances, Int32 exact_band) at g3.MeshSignedDistanceGrid.Compute() at FileToVox.Converter.OBJToSchematic.WriteSchematic() in E:\Documents\FileToVox\SchematicToVoxCore\Converter\ObjToSchematic.cs:line 34 at FileToVox.Program.SchematicToVox(AbstractToSchematic converter, String outputPath) in E:\Documents\FileToVox\SchematicToVoxCore\Program.cs:line 292 at FileToVox.Program.ProcessFile() in E:\Documents\FileToVox\SchematicToVoxCore\Program.cs:line 245

[LOG] Done.`

I'm really don't understand what am i missing.

[Zarbuz]

Can you upload the model which fail ? It's hard to replicate the problem.

[SilentPrayeCG]

sure

landscape3.zip

[Zarbuz]

Well I don't think there is a real bug, it's just that the grid size you set is very very big, so you probably have not enough ram to do this conversion (I sucessfully converted your model with a grid size of 1000 but I flirt with the limit of my ram (almost 16Go of RAM used).

The OBJ conversion is deprecated and I will remove this feature in the future. Try to convert your model in PLY (a cloud point) and the convert it with FileToVox. Much better results (colors are also supported). I've done a video tutorial about this : https://www.youtube.com/watch?v=sg3z2GaMJzM

[SilentPrayeCG]

Well as i said, before this i made much denser mesh, which size 100mb instead of 20, and it converted ok. I have 64gb RAM, i don't think it's issue.

I rechecked, first obj i attached should be 252001 faces And this one ground.zip is 2011988 And it converted ok. As i understand 2000 is limit, it's not crash when i set gs 2000, just throw error at the end. But at 1995 it's converting ok ` r:\FileToVox>filetovox --i "r:\ground.obj" --o "r:\ground_test" --gs 2000 --e [INFO] Specified input path: r:\ground.obj [INFO] Specified output path: r:\ground_test [INFO] Specified increase size: 1 [INFO] Specified grid size: 2000 [INFO] Enabled option: excavate [INFO] Specified output path: r:\ground_test [INFO] Vox Width: 2005 [INFO] Vox Length: 2005 [INFO] Vox Height: 480 [ERROR] Voxel model is too big ! MagicaVoxel can't support model bigger than 2000^3 [LOG] Done.

r:\FileToVox>filetovox --i "r:\ground.obj" --o "r:\ground_test" --gs 1995 --e [INFO] Specified input path: r:\ground.obj [INFO] Specified output path: r:\ground_test [INFO] Specified increase size: 1 [INFO] Specified grid size: 1995 [INFO] Enabled option: excavate [INFO] Specified output path: r:\ground_test [INFO] Vox Width: 2000 [INFO] Vox Length: 2000 [INFO] Vox Height: 479 [LOG] Started to compute the first block for each region [LOG] Done. [INFO] Total blocks: 11180147 [LOG] Started to write chunks ... [####------] 45% -[LOG] Done. [LOG] Check total blocks after conversion: 11180147 [LOG] Done. ` And yes, in process it takes great amount of RAM, up to 50gb but it starts...

I guess in this case i can achieve same result with just depthmap, but i thought adding some details with other objs and then merge them.

the only difference (except face count) between those two objects, that new is thicker

---

If it possible to convert classic mesh to PLY, i will try. I thought PLY is standard for 3d scanners, i remember getting hard time to convert PLY model to 3dsmax as classic mesh, long time ago.

[SilentPrayeCG]

i was able to convert this landscape in to cloud of points, i also was able scale mesh to fit maximum size of world (almost of course, near maximum values) but i there is need tremendous amount of points to fill whole mesh surface. PLY file grown to about 2gb and it's still have holes i think "fill holes" option should help, but it just throw error, even if use default amount of points (vox scene then also looks like cloud of points)

[INFO] Specified input path: R:\landscape_2000.ply [INFO] Specified output path: R:\landscape_2000 [INFO] Specified increase size: 1 [INFO] Enabled option: fix-holes [INFO] Specified output path: R:\landscape_2000 [LOG] Start reading PLY data... [LOG] Done. [LOG] Started to voxelize data... [LOG] Done. System.ArgumentException: Value of '-26' is not valid for 'alpha'. 'alpha' should be greater than or equal to 0 and less than or equal to 255. at System.Drawing.Color.<CheckByte>g__ThrowOutOfByteRange|325_0(Int32 v, String n) at System.Drawing.Color.FromArgb(Int32 alpha, Int32 red, Int32 green, Int32 blue) at FileToVox.Quantizer.Quantizer.GetQuantizedPalette(Int32 colorCount, ColorData data, IEnumerable1 cubes, Int32 alphaThreshold) in E:\Documents\FileToVox\SchematicToVoxCore\Quantizer\Quantizer.cs:line 79 at FileToVox.Quantizer.QuantizerBase.QuantizeImage(Bitmap image, Int32 alphaThreshold, Int32 alphaFader, Int32 maxColorCount) in E:\Documents\FileToVox\SchematicToVoxCore\Quantizer\QuantizerBase.cs:line 33 at FileToVox.Extensions.Quantization.ApplyQuantization(List`1 blocks, Int32 colorLimit) in E:\Documents\FileToVox\SchematicToVoxCore\Extensions\Quantization.cs:line 15 [LOG] Started to fill holes... System.IndexOutOfRangeException: Index was outside the bounds of the array. at FileToVox.Converter.PointCloud.PointCloudToSchematic.Check1X1X1Hole(UInt32[,,] blocks, UInt16 x, UInt16 y, UInt16 z) in E:\Documents\FileToVox\SchematicToVoxCore\Converter\PointCloud\PointCloudToSchematic.cs:line 273 at FileToVox.Converter.PointCloud.PointCloudToSchematic.FillHoles(UInt32[,,] blocks, Schematic schematic) in E:\Documents\FileToVox\SchematicToVoxCore\Converter\PointCloud\PointCloudToSchematic.cs:line 101 at FileToVox.Converter.PointCloud.PointCloudToSchematic.WriteSchematic() in E:\Documents\FileToVox\SchematicToVoxCore\Converter\PointCloud\PointCloudToSchematic.cs:line 248 at FileToVox.Program.SchematicToVox(AbstractToSchematic converter, String outputPath) in E:\Documents\FileToVox\SchematicToVoxCore\Program.cs:line 292 at FileToVox.Program.ProcessFile() in E:\Documents\FileToVox\SchematicToVoxCore\Program.cs:line 245

[LOG] Done. ` also regardless of "fill holes" option, there is some error about alpha being -26, I'm not sure, at first test this error not appear and when it appear afterwards it's don't interrupt program and result opens in MagicaVoxel ok.

My goal was make landscape that fill whole available world space, and i was able to do this with that 100mb obj, and it's seems i was just lucky or something.

[SilentPrayeCG]

If fill holes suppose to fix holes in converted to voxels coud, it doesn't work for me at all. Even on small mesh it's everytime just throw error.

[LOG] Started to fill holes... System.IndexOutOfRangeException: Index was outside the bounds of the array. at FileToVox.Converter.PointCloud.PointCloudToSchematic.Check1X1X1Hole(UInt32[,,] blocks, UInt16 x, UInt16 y, UInt16 z) in E:\Documents\FileToVox\SchematicToVoxCore\Converter\PointCloud\PointCloudToSchematic.cs:line 278 at FileToVox.Converter.PointCloud.PointCloudToSchematic.FillHoles(UInt32[,,] blocks, Schematic schematic) in E:\Documents\FileToVox\SchematicToVoxCore\Converter\PointCloud\PointCloudToSchematic.cs:line 106 at FileToVox.Converter.PointCloud.PointCloudToSchematic.WriteSchematic() in E:\Documents\FileToVox\SchematicToVoxCore\Converter\PointCloud\PointCloudToSchematic.cs:line 253 at FileToVox.Program.ProcessFile() in E:\Documents\FileToVox\SchematicToVoxCore\Program.cs:line 254

maybe it need to be posted as separate issue?

[Zarbuz]

I recommend to convert in a binary PLY instead of a ASCII PLY to reduce the file size (more compressed). I will check the fill-holes option but keep in mind it will fill hole of 1x1 or 2x1 (very small holes). If in your .vox file you have bigger holes, you need to increase the density of your points. If you use CloudCompare to do this with your model, I would say you need something like 40M of points. I don’t know how much you have set with your precedent result. But there’s definitely some errors to check in my side

[SilentPrayeCG]

I'm used binary, it's first button, I'm actually didn't knew whats difference (askii it's text probably, as obj file, i dunno) So mesh with size about 2000x2000, and amount of points is 100 000 000 (it's what i used, it's wont let you set bigger number, i guess, except if you use density ratio instead) ply file is big, ~2gb, it's not problem, it's also converts fast, and ok, except this

[LOG] Started to voxelize data... [LOG] Done. System.ArgumentException: Value of '-11' is not valid for 'alpha'. 'alpha' should be greater than or equal to 0 and less than or equal to 255. at System.Drawing.Color.g__ThrowOutOfByteRange|325_0(Int32 v, String n) at System.Drawing.Color.FromArgb(Int32 alpha, Int32 red, Int32 green, Int32 blue) at FileToVox.Quantizer.Quantizer.GetQuantizedPalette(Int32 colorCount, ColorData data, IEnumerable1 cubes, Int32 alphaThreshold) in E:\Documents\FileToVox\SchematicToVoxCore\Quantizer\Quantizer.cs:line 79 at FileToVox.Quantizer.QuantizerBase.QuantizeImage(Bitmap image, Int32 alphaThreshold, Int32 alphaFader, Int32 maxColorCount) in E:\Documents\FileToVox\SchematicToVoxCore\Quantizer\QuantizerBase.cs:line 33 at FileToVox.Extensions.Quantization.ApplyQuantization(List1 blocks, Int32 colorLimit) in E:\Documents\FileToVox\SchematicToVoxCore\Extensions\Quantization.cs:line 15

as i wrote before, i dunno why, program not throw this error if mesh is smaller.

and there is still holes, but small ones, as you told, 1-2 voxels

i guess fix holes exactly why i need in this situation

[Zarbuz]

Hello, can you test this future version and tell me if it correct the problems you had ? FileToVox-v1.9.4.1.zip

[SilentPrayeCG]

Hi! I've checked fill holes flag, no errors. Then i decided to check if it actually fixes holes. Tried to convert that landscape from 2gb ply to voxels, it's converts with one error in log, and results is dummy file.

'[INFO] Specified input path: D:\Stuff\Landscape.ply [INFO] Specified output path: D:\Stuff\Landscape [INFO] Specified increase size: 1 [INFO] Specified output path: D:\Stuff\Landscape [LOG] Start reading PLY data... [LOG] Done. [LOG] Started to voxelize data... [LOG] Done. [INFO] Vox Width: 1998 [INFO] Vox Length: 1998 [INFO] Vox Height: 873 [LOG] Started to compute the first block for each region [LOG] Done. [INFO] Total blocks: 11357369 [LOG] Started to write chunks ... [LOG] Done. [LOG] Check total blocks after conversion: 0 [ERROR] There is a difference between total blocks before and after conversion. [LOG] Done.'

it says for some reason that total blocks after conversion is 0

i tried scale 0.95, result is same

[Zarbuz]

Do we still talk about this terrain ? I've manually converted your OBJ file into a PLY and made some test. I converted with the same argument as you and I didn't have any errors. (The scale isn't the same tho) I you can upload your 2Gb file I could probably better see the problem.

[SilentPrayeCG]

No, this one i was able to convert from obj, but then decided to make new one. We talking about new one, https://github.com/Zarbuz/FileToVox/files/4742619/landscape3.zip But this file not scaled to whole world.

i tested on this landscp.zip you can try yo convert to ply, i used 100 000 000 points, or there is my conversion to ply https://mega.nz/file/zXZkUCiT#4GJ82Y0z_NZWglaBqb4ow2rqS_qTkuYBLkIC6oUtQj4 if this file still will be needed, in case you don't have mega account (it probably won't let you download it because it big, if they didn't changed this stupid rule) i will upload on something else