Questions about "too many objects" to handle or to process

[enzyme69]

It's probably becoming a bit general questions, but can be for Sverchok.

I have 10000-12000 curves objects. It was from this iOS app, it produces and sample a photo and generate point stipples and arrange it on the fly. I can have realtime controls and sliders. Then I export it out as SVG.

I open it in Blender and Blender to my surprise slow down. Even when all I wanted to do is to select and scale all the 10000+ Circles Curves objects up. Because at the moment it is like under 1 unit in size.

All I need to do is: 1) Select all this list of object 2) Get the center matrix list of all those objects 3) Turn it into just a points of a single mesh.

Then Blender become very fast. Even with the curve thing, 10000-12000 objects are too many, but join them and suddenly Blender can handle it.

And I also try Python to do the same thing, and of course it is very fast.

With Sverchok, it is processing everything, everytime at the same time, right? Can we handle the Sverchok processing chunk by chunk?

[enzyme69]

It's very fast once it is a mesh.

But took me 10-15 minutes, selecting chunk, converting to mesh per chunk and merge all together as a single stippled Mesh Points.

[ly29]

Blender is very slow at object creation and at management of them. Even if the data in them is very simple.

http://blender.stackexchange.com/questions/7358/python-performance-with-blender-operators/7360#7360

[enzyme69]

No wonder, it's Blender operator :( that is slow.

Somewhat, I think what I am trying to do is faster using Python. Just script.

[enzyme69]

But hey, I use Animation Nodes, and to get every Poly center points and generate a mesh based on that point only takes less than 1 second.

So I think Center Polygons is heavy in Sverchok :8ball:

[enzyme69]

1) Create Points Stipples / Circle Packing algorithm > Maybe use Processing or App 2) Get Center points of every poly, faster when done chunk by chunk (of joined Circles) 3) Generate Delaunay > Sverchok FAST 4) Get center of every poly again > Animation Nodes FAST 5) Coloring every Face of Polygon as Vertex Color > Sverchok FAST

Anyhow, I learned some lessons from this project :) Cheers Linus for the example node setup and also how to use Cycles Texture Object position and also Image Sampling script.

Cheeers!

[ly29]

Centers Polygons is broken unless separate is checked, but also then it is not really what one wants #832, it could also do less calculation if one only wants the centers...

[enzyme69]

[zeffii]

hey @enzyme69 got a copy of that original svg?

[enzyme69]

Here you go, @zeffii https://www.dropbox.com/s/tsx3lq1wqkdu9q0/BlackDog_SnapDotStipple4.svg?dl=0

This is the result of Circle Packing algorithm. I don't know how they did it in app. It generates 10000-12000 circles, depends on the sliders.

Feel free to investigate.

Firstly, Blender does not like too many objects, over 10000 objects, and Blender is in a comma state. Cannot select all of them, I need to select by chunk, join them and so on.

To ask Sverchok to generate mesh based on the center of every polygon (after I convert the Curve into mesh), is almost impossible via nodes. Unless I missed something.

Needs Animation Nodes Looping over Poly and generate in 1 go only.

Ideally, I would just like to load Circles objects, then get the center point, and then generate Delaunay, and then from Delaunay, get the Face center of tesselated mesh poly, once again, and resample.

[ly29]

Center of polygons is broken, I waiting for @nortikin feedback before commit a fix (since it has been broken for a long time). 10000 faces with centers of polygon would result in an incredible amount of extra work right now. Like seriously broken, it generates an incredible amount of extra data...

[zeffii]

it's almost like the makers of that program that exports the .svg decided to use the least efficient way to store a collection of circles of different sizes..

this imports the file as dots in about 0.3 second

import bpy
import re
from math import cos, radians

track_path = r"C:\Users\zeffi\Downloads\rubber\BlackDog_SnapDotStipple4.svg"
pattern = "<path fill=\".*\" d=\"M(.+?,.+?)c\d"

points = []
with open(track_path) as tfile:
    string = ''.join(tfile.readlines())
    finds = re.findall(pattern, string)
    if finds:
        for coords in finds:
            x, y = [float(v)/400 for v in coords.split(',')]
            points.append([x, y, 0.0])

profile_mesh = bpy.data.meshes.new("Base_Profile_Data")
profile_mesh.from_pydata(points, [], [])
profile_mesh.update()

profile_object = bpy.data.objects.new("Base_Profile", profile_mesh)

scene = bpy.context.scene
scene.objects.link(profile_object)
profile_object.select = True

[zeffii]

looks like it may have skipped a few due to my lazy regex, oh well. (and it's flipped ) - i guess my point is it's worth doing as much processing as possible outside of the node tree.

of the 13009 paths in that svg the regex found all of them.