Use AssetBundles in procedural generation

[yamins81]

Is there some way to create a URL-accessible library of prefabs somewhere that allows them to be loaded? Let's say we create 10,000 prefabs on someone's desktop. Is that the only place we can create binaries from for real production? Is there some way to create a binary that will load prefabs from a storage location? Eg. so that we can add to the library of prefabs without having to rebuild a HUGE binary and then transfer that huge binary? It's really annoying.

@jazzguitarist What do you think? Is this even possible?

[yamins81]

This is basically the same issue as using AssetBundles.

[yamins81]

@jazzguitarist wrote to me in an email:

assetbundles stay loaded until unloaded. To avoid memory problems, its therefore necessary to unload assetbundles at some point. There are two easy solutions I thought of. One is to have a method in prefab database which unloads all assetbundles. Inserting a call to this in procedural generations init function will ensure that no more than the prefabs actively used in the scene are loaded. If this causes memory issues as well, prefabs can also be unloaded after instantiated. This may make the room generation very slow though, unsure. Next option is to determine a good number of assetbundles to be loaded at any given time. Then implement a cache structure which unloads the least recently used assetbundle whenever you load a new bundle and are out of space in the cache. Unsure how loaded assetbundles effect memory, but either of these solutions should work fine.

[pbashivan]

From what I have been reading so far, it seems like the only way to create new bundles is through the editor. If that's true then we can't really make a new bundle for each object.

[yamins81]

Can't you script it using this method or similar? https://docs.unity3d.com/460/Documentation/ScriptReference/BuildPipeline.BuildAssetBundle.html

[pbashivan]

I used this function to build the bundles, but you can't use this function to assign assets to bundles using this method. Assigning assets to bundles and creating new bundle names is the issue.

[yamins81]

Hm I don't understand the problem. Isn't that function's arguments like a list of assets and a pathname? Why can you start with a list of prefabs, loop through the list, and create new assetbundles for each one with a distinct path name?

[pbashivan]

I'll check on this but I think path is where to put the bundle files. Not sure if it's actually filename, or at least that's how I used this function before. I'll update when I get to test it.

[yamins81]

But I'm still having trouble understanding your problem ... What makes the suggestion I made above not possible to implement?

[pbashivan]

What I mean to say is, the path is a path to a directory (for putting the bundle files in it) not path to a specific filename for a bundle file. Maybe we can discuss it this evening.

[yamins81]

you're saying that BuildAssetBundle function is deprecated in unity 5.x.

what about this http://answers.unity3d.com/questions/1002779/creating-assetbundles-entirely-in-code-in-5x.html

[yamins81]

look at the second function signature here:

https://docs.unity3d.com/ScriptReference/BuildPipeline.BuildAssetBundles.html

[yamins81]

Notes from @jazzguitarist about using assed bundles:

1) Extend the Setup Prefabs process in PrefabDatabase.cs to load the prefabs into Assets/PrefabDatabase/ as AssetBundles.

--> This also would mean you need to move the actual prefabs to the PrefabDatabase folder as well, so they are not contained in the resources folder.

--> You will also need to refactor some of the Generate and Setup prefab code to operate in the PrefabDatabase directory instead of the Resources directory.

(As a note, I recommend making a directory structure like this to keep things clean and clear: /Assets/PrefabDatabase/GeneratedPrefabs/... and /Assets/PrefabDatabase/AssetBundles/..., this way you can just duplicate the tree of GeneratedPrefabs/ in AssetBundles but contain the bundles instead. If you prefer to just have the bundle and prefab in the same place, that's probably fine too, but might cause some confusion)

2) After all this is done, the next task would be to implement a Load method (i already inserted an unimplemented one in PrefabDatabase.cs) this will be used to replace the call of Resources.Load() in ProceduralGeneration.cs. I put a comment in there about switching once that's done.

As for storing which prefabs are being used (for loading and unloading AssetBundles), that data can be recorded whenever you call that load method to be implemented in PrefabDatabase.cs.

You should also probably implement an unload method in PrefabDatabase.cs and make a call to it in Init() in ProceduralGeneration.cs.

Later down the road you can try to store the ones which will be used again, but sadly, the things being used are not pre-determined (re: this, I wrote a comment early on in ProceduralGeneration.cs that said something regarding where prefabs are predetermined, but that was incorrect, so ignore that comment, I just forgot to delete it).

Once all this stuff is implemented, you can change the call from Resources.Load to prefabDatabase.Load in ProceduralGeneration, run the new Prefab Generation and Prefab setup code. And hopefully things will run.

[yamins81]

Still have to test whether it's better have all the assets in one big bundle or many small asset bundles

[chengxuz]

The right thing todo is to put all prefabs in mongo database (including all the information like scales, etc) and then use LoadFromCacheOrDownload to load all the things, rather than loading all the prefabs just from directory.

Currently the loading is finished by loading all the resources under the folder.(line 291 in PrefabDatabase.cs)

[pbashivan]

@chengxuz I'm changing the code to use the asset bundles instead of resources. I'm using "LoadFromFile" to load the bundles locally. I have looked at the "LoadFromCacheOrDownload" function, it takes a url and load the asset from there. Are you suggesting to put the bundle files on some server and put the link to it on MongoDB?

[chengxuz]

Yes, that would be the ideal state that we would like to get to. The comment is actually responding to your question about Prefabdatabase on Slack. So currently the prefabdatabase is built by loading all the local resources under the folder (see line 291 in PrefabDatabase.cs). I think PrefabDatabase.cs actually describes how this struct is built and what is in there.

[pbashivan]

Oh I see. PrefabDatabase keeps a list of all available prefabs as a property, right? I changed the code such that it goes over all bundle files in /PrefabDatabase/AssetBundles and adds those entries to the prefabs list property. I'm going to stick with local bundle files for now as we need to wrap it up very soon and begin the tests on the environment. Another relevant question, do you know why we have duplicate code in ProceduralGeneration.cs for setting up prefabs?

[qbilius]

Hey guys, why don't we move chatting of this sort to Slack?

[chengxuz]

I think there are currently two different Slack channels for 3Dworld, and one of them (in yaminslab) actually has been linked with the Github repo which means every issue and responses would be posted in that channel too. So @qbilius , do you think the channel at dicarlolab also needs this link? Or we should simply discuss those things in the dicarlolab (as you and Pouya are not in yaminslab)?

[chengxuz]

I would continue Pouya's work for this issue now. And there are 3 things to do:

1. TEST: Prefabs cross-platform generating and using.
2. TEST: Assetbundles creating and loading through binary files with 5000 objects.
3. IMPLEMENT: Replacing loading from files under some folder to Loadfromcacheordownload (https://docs.unity3d.com/ScriptReference/WWW.LoadFromCacheOrDownload.html), including modification to SetupBundles in PrefabDatabase and CompileAssetBundles in PrefabDatabase.

@pbashivan , please feel free to add other things you think need to be done. Thanks for talking about and explaining your implementations to me!

Best, Chengxu

[chengxuz]

Hi, @pbashivan , I met a bug about objects falling through floor. It seems that this should be related to the objects and models imported. I don't have this bug before merging. And your branch without merging already has this bug (you can test by setting permitted_items to be ["bed", "sofa"]). Some of the objects will fall, while some won't.

From google, I learned that I should change something about the object or model. But should I do it before Create Prefabs? Or during dynamically loading?

Any idea about how to fix this?

Thanks, Chengxu

[pbashivan]

@chengxuz I'm seeing this happening only for the objects in "Props" and not for the ones I try from shapenet (Can you also confirm?). Apparently it's not because of bundling cause when I manually insert prefabs of those objects into the scene the same thing happens. Probably something related to generating the prefabs.

[chengxuz]

HI, @pbashivan , thanks for responding. Then maybe we can try to solve this falling through floor later. Currently I met a problem during building the binary file. It turns out that AssetDatabase.LoadAssetAtPath in this sentence:

        PrefabDatabase database =  AssetDatabase.LoadAssetAtPath<PrefabDatabase> 
                ("Assets/ScenePrefabs/PrefabDatabase.prefab");

is only available in Editor mode. So I tried to replace it with some other functions such as AssetBundle.LoadFromFile, but I failed. I wonder whether you have done something similar and I would be grateful if you could provide some suggestions.

Best, Chengxu

[chengxuz]

OK, I think I have solved this using GameObject.FindObjectOfType<PrefabDatabase>().

[pbashivan]

That would be my guess also. Did it work?

On Mon, Oct 10, 2016 at 5:16 PM Chengxu Zhuang notifications@github.com wrote:

OK, I think I have solved this using GameObject.FindObjectOfType().

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dicarlolab/ThreeDWorld/issues/37#issuecomment-252748629, or mute the thread https://github.com/notifications/unsubscribe-auth/AL1xVxDN3j6TeqTSXO90du0LkeDPHNbPks5qyqtFgaJpZM4JLp4M .

[chengxuz]

Yes. I can now build the binary file and successfully run it on Mac for Mac, Linux for Linux. But not cross-platform. The binary file could be run, but no objects would be in the room. I guess it is mainly because of the absolute path for those assetbundles in prefabs. I would try to possibly implement Loadfromcacheordownload method to make it work or find someway tricky to avoid using absolute path for just testing the cross-platform assetbundle things.

[chengxuz]

Hi, Pouya, Jonas, @qbilius @pbashivan I did some test about importing Shapenet models (V2) into the current project. There are two main problems existing:

1. All the new imported objects do not have color. They are white, regardless of the objects.
2. They would fall through the floor.

Actually I think on both linux and mac, those two problems also happen for "sofa", "bed", and some other objects (but not all of them).

I am still trying to solve those two problems. But I would be grateful if you could provide any suggestions.

And Dan told me that it seemed to be issues that you have already solved before on Windows Machine. So I wonder whether it's the same problem and how you solved them before.

I would describe how I import and create prefabs for Shapenet models (V2) and possibly I did something wrong.

1. I start by downloading the shapenet models from sandbox (under Jonas' directory, I guess).
2. After unzipping the file, I copied part files under ShapeNetCore.v2/02691156 to ThreeDWorld/Assets/Models/sel_objs/test_mine (I make this dir my self).
3. I opened the Unity, and after some importing, the error below happened (for every folder I imported):

MaterialProcessor.ProcessMtlFile (System.String fileLocation) (at Assets/Scripts/Editor/MaterialProcessor.cs:66)
MaterialProcessor.OnPostprocessModel (UnityEngine.GameObject obj) (at Assets/Scripts/Editor/MaterialProcessor.cs:54)
System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Reflection/MonoMethod.cs:222)
Rethrow as TargetInvocationException: Exception has been thrown by the target of an invocation.
System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Reflection/MonoMethod.cs:232)
System.Reflection.MethodBase.Invoke (System.Object obj, System.Object[] parameters) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Reflection/MethodBase.cs:115)
UnityEditor.AttributeHelper.InvokeMemberIfAvailable (System.Object target, System.String methodName, System.Object[] args) (at /home/builduser/buildslave/unity/build/Editor/Mono/AttributeHelper.cs:249)
UnityEditor.AssetPostprocessingInternal.PostprocessMesh (UnityEngine.GameObject gameObject) (at /home/builduser/buildslave/unity/build/Editor/Mono/AssetPostprocessor.cs:222)

1. After ignoring those errors, I selected all the folders I imported, then create the prefabs, build the assetbundle seperately, during which, the error shown below occured:

InvalidOperationException: Operation is not valid due to the current state of the object
System.Collections.Stack.Pop () (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Collections/Stack.cs:329)
UnityEngine.GUILayoutUtility.EndLayoutGroup () (at /home/builduser/buildslave/unity/build/Runtime/IMGUI/Managed/GUILayoutUtility.cs:288)
UnityEditor.SplitterGUILayout.EndVerticalSplit () (at /home/builduser/buildslave/unity/build/Editor/Mono/GUI/Splitter.cs:491)
UnityEditor.ConsoleWindow.OnGUI () (at /home/builduser/buildslave/unity/build/Editor/Mono/ConsoleWindow.cs:496)
System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Reflection/MonoMethod.cs:222)

1. I setup the assetbundles. But it seems that those imported objects are not in the same folder structure as the originals. They are both under another subfolder called "Models" (e.g., 5f2aa0bc5a7fc2beacf77e718d93f3e1/models), which seems to make some problems for naming (all of them are under a large assetbundle naming "model_normalized").
2. I clicked run, and all the imported objects are white and falling through the floor.

Best, Chengxu

[pbashivan]

Couldn't this be an access permission issue? Probably @qbilius would have a better idea. BTW, I tried creating prefabs (about 200 of them) from shapenet files on my mac and the prefabs I generated didn't have texture or falling through issues when I imported them from prefabs or bundles.

[qbilius]

No idea. I didn't have any issues importing; though, admittedly, building prefabs crashed, as you know, and the build was also not working, but I never looked into how the objects looked like.

Maybe it would make sense for you to try running the same procedure on a Windows machine (say, kubelik) and see if that works.

[chengxuz]

@pbashivan @qbilius , thanks for responding! May I ask which version of shapenet you guys are using? I used v2, but it seems that Pouya used v1, is that right? What's the difference of this two version? Which one should we use?

[qbilius]

I'm using v2

On Tue, Oct 11, 2016, 13:19 Chengxu Zhuang notifications@github.com wrote:

@pbashivan https://github.com/pbashivan @qbilius https://github.com/qbilius , thanks for responding! May I ask which version of shapenet you guys are using? I used v2, but it seems that Pouya used v1, is that right? What's the difference of this two version? Which one should we use?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dicarlolab/ThreeDWorld/issues/37#issuecomment-252983721, or mute the thread https://github.com/notifications/unsubscribe-auth/ABMaMd9ZM2RzENdN_ZG880VS7Q72BbwUks5qy8UugaJpZM4JLp4M .

[pbashivan]

@chengxuz I also used v2. v1 had some problems with importing the textures if I remember correctly.

[chengxuz]

OK, I think I might have found the reason of my problems. I downloaded the ShapeNetCore.v2.zip directly. But it seems that the contents there is different from that in the folder "ShapeNetCore.v2". This might cause some problems.

Anyway, after copying files directly from the folder, I could now import, create, and setup the assetbundles successfully.

Thanks for helping!

[chengxuz]

Hi, @pbashivan @qbilius , is it possible to delete something in sandbox to free some space?

I would like to zip the shapenet.v2 folder (as the zip file already existing seems not having the same contents), but it seems that there is no enough space left. Could you for example delete the Shapenet.V2.zip file and create it again? Or where do you think I should download the data from?

A better way is of course expanding the disk space of sand-box.

[pbashivan]

Can't really comment on that cause I haven't really used it.

On Tue, Oct 11, 2016 at 4:19 PM Chengxu Zhuang notifications@github.com wrote:

Hi, @pbashivan https://github.com/pbashivan @qbilius https://github.com/qbilius , is it possible to delete something in sandbox to free some space?

I would like to zip the shapenet.v2 folder (as the zip file already existing seems not having the same contents), but it seems that there is no enough space left. Could you for example delete the Shapenet.V2.zip file and create it again? Or where do you think I should download the data from?

A better way is of course expanding the disk space of sand-box.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dicarlolab/ThreeDWorld/issues/37#issuecomment-253033228, or mute the thread https://github.com/notifications/unsubscribe-auth/AL1xV8qvp_cr-vuG9QZUwhMsHQwCmUtqks5qy-8-gaJpZM4JLp4M .

[qbilius]

Could you get the zip from kubelik instead?

On Tue, Oct 11, 2016, 20:15 Pouya notifications@github.com wrote:

Can't really comment on that cause I haven't really used it.

On Tue, Oct 11, 2016 at 4:19 PM Chengxu Zhuang notifications@github.com wrote:

Hi, @pbashivan https://github.com/pbashivan @qbilius https://github.com/qbilius , is it possible to delete something in sandbox to free some space?

I would like to zip the shapenet.v2 folder (as the zip file already existing seems not having the same contents), but it seems that there is no enough space left. Could you for example delete the Shapenet.V2.zip file and create it again? Or where do you think I should download the data from?

A better way is of course expanding the disk space of sand-box.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub < https://github.com/dicarlolab/ThreeDWorld/issues/37#issuecomment-253033228 , or mute the thread < https://github.com/notifications/unsubscribe-auth/AL1xV8qvp_cr-vuG9QZUwhMsHQwCmUtqks5qy-8-gaJpZM4JLp4M

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dicarlolab/ThreeDWorld/issues/37#issuecomment-253083950, or mute the thread https://github.com/notifications/unsubscribe-auth/ABMaMWssghPNje-0nlkdhlUUOHJQ4IkAks5qzCaSgaJpZM4JLp4M .

[chengxuz]

Thanks for responding!

@qbilius Is the kubelik the windows machine with user threed? I am actually doing that now, but the contents there (in C:\Users\threed\ThreeDWorld\Assets\Models\sel_objs) seem to be different from that in sandbox, for example, there are only 53 subfolders there, instead of 55, and the total space occupied is also different. I guess it should be OK, right?

Best, Chengxu

[qbilius]

Sorry, I can't check it now. I think you're looking at the imported objects and not all were imported, I think. There is another place where the original objects are.

On Tue, Oct 11, 2016, 20:22 Chengxu Zhuang notifications@github.com wrote:

Thanks for responding!

@qbilius https://github.com/qbilius Is the kubelik the windows machine with user threed? I am actually doing that now, but the contents there (in C:\Users\threed\ThreeDWorld\Assets\Models\sel_objs) seem to be different from that in sandbox, for example, there are only 53 subfolders there, instead of 55, and the total space occupied is also different. I guess it should be OK, right?

Best, Chengxu

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dicarlolab/ThreeDWorld/issues/37#issuecomment-253084979, or mute the thread https://github.com/notifications/unsubscribe-auth/ABMaMaIYudTPdnpE7BQpesF7q96GCtXnks5qzChggaJpZM4JLp4M .

[chengxuz]

Hi, @qbilius , thanks very much for responding at later night. I think I have found the original objects in the windows machine!

[chengxuz]

Hi, @pbashivan , how fast is the creating prefabs on Mac or Windows? It takes me more than 10 hours to only create prefabs for 545 objects on Linux, is it normal?

[qbilius]

Sounds normal to me -- see this plot I made: https://dicarlolab.slack.com/messages/3d_environment/search/prefab/

Also, let me reiterate again: can we please switch back to Slack for chatting? We have the 3d_environment channel there, we've been chatting there all the time, so I don't really see why we're using GitHub issues for this.

2016-10-12, tr, 12:48 Chengxu Zhuang notifications@github.com rašė:

Hi, @pbashivan https://github.com/pbashivan , how fast is the creating prefabs on Mac or Windows? It takes me more than 10 hours to only create prefabs for 545 objects on Linux, is it normal?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dicarlolab/ThreeDWorld/issues/37#issuecomment-253270257, or mute the thread https://github.com/notifications/unsubscribe-auth/ABMaMTZxLE514Uto2HxYKuGs4j3GJiN4ks5qzQ9mgaJpZM4JLp4M .

[pbashivan]

I tried generating ~200 prefabs on my mac, it took almost an hour or so. Actually, it was very memory demanding so I tried generating them in small batches to speed it up. On Wed, Oct 12, 2016 at 12:54 PM Jonas Kubilius notifications@github.com wrote:

Sounds normal to me -- see this plot I made: https://dicarlolab.slack.com/messages/3d_environment/search/prefab/

Also, let me reiterate again: can we please switch back to Slack for chatting? We have the 3d_environment channel there, we've been chatting there all the time, so I don't really see why we're using GitHub issues for this.

2016-10-12, tr, 12:48 Chengxu Zhuang notifications@github.com rašė:

Hi, @pbashivan https://github.com/pbashivan , how fast is the creating prefabs on Mac or Windows? It takes me more than 10 hours to only create prefabs for 545 objects on Linux, is it normal?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub < https://github.com/dicarlolab/ThreeDWorld/issues/37#issuecomment-253270257 , or mute the thread < https://github.com/notifications/unsubscribe-auth/ABMaMTZxLE514Uto2HxYKuGs4j3GJiN4ks5qzQ9mgaJpZM4JLp4M

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dicarlolab/ThreeDWorld/issues/37#issuecomment-253271980, or mute the thread https://github.com/notifications/unsubscribe-auth/AL1xVwJdTnunn9RK1DzCUvz3zDClLsg4ks5qzRDSgaJpZM4JLp4M .

[yamins81]

@chengxuz @pbashivan @qbilius

Are you making sure to hit "play" in the editor when creating prefabs?

See the little section on "Generating prefabs” on the readme in the ThreedWorld repo. That may clear up some of the memory problems being seen here.

[qbilius]

Well, I wrote that section so my benchmarking plot was done using those guidelines.

2016-10-12, tr, 15:08 Dan Yamins notifications@github.com rašė:

@chengxuz https://github.com/chengxuz @pbashivan https://github.com/pbashivan @qbilius https://github.com/qbilius

See the little section on "Generating prefabs” on the readme in the ThreedWorld repo That may clear up some of the memory problems being seen here.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dicarlolab/ThreeDWorld/issues/37#issuecomment-253308342, or mute the thread https://github.com/notifications/unsubscribe-auth/ABMaMeRROIi3rdVUqiJfUoEpk-uA1N30ks5qzTAegaJpZM4JLp4M .

[yamins81]

Ah, ok. I think @pbashivan might have not been doing that if he had problems with memory. @chengxuz might also might not have been, so just wanted to make sure they knew it.

[chengxuz]

@yamins81 @pbashivan @qbilius thank you all for commenting. I would try to generate the prefabs during "Play" mode.

BTW, about whether we should discuss things here or on slack, I was originally thinking that discussion here would be more organized and specific for only one issue, which means once this issue is solved, we could close it and don't need to read those discussions in future. But discussions on slack seem to be always there, which means that they would show up when you look for older discussions about other things or different issues would be discussed in the channel together. But now it seems that some discussions do have general use for the whole project. So I don't know where we should discuss it here or on slack. What about other people?

[pbashivan]

@yamins81 I did hit the play button, but it would still consume many GB of memory nonetheless.

[chengxuz]

The current status of this issue is:

1. TEST: Prefabs cross-platform generating and using.
2. TEST: Assetbundles creating and loading through binary files with 5000 objects. The speed is too slow to generate so many prefabs.
3. IMPLEMENT: Replacing loading from files under some folder to Loadfromcacheordownload, including modification to SetupBundles in PrefabDatabase and CompileAssetBundles in PrefabDatabase. There is still one thing we can do: Make it possible for using private assetbundles, this might require to use C# SDK for AWS in Unity.

[yamins81]

@chengxuz we should replace using LoadFromCacheOrDownload with our own downloading and caching and use of LoadFromFile, to get around the hard 4GB cache size limit. This is very time sensitive.

[pbashivan]

@chengxuz @yamins81 I think it would make more sense to use the object name in scale_relat_dictrather than the full bundle URL path. e.g.: 46e777a46aa76681f4fb4dee5181bee instead of "http://threedworld.s3.amazonaws.com/46e777a46aa76681f4fb4dee5181bee.bundle"

Let me know if you agree.