encryption - .blend file deployment

[GoogleCodeExporter]

GameKit should support a way to deploy encrypted .blend files. 

So for example, GameKit should read the file based on it's extension.

gamekit.exe example.blend   ---read the file normally.

gamekit.exe example.gamekit  --- read file to memory, decrypt, and read normal 
.blend file.

a utility exe file, or maybe even gamekit.exe with a special command line that 
takes a file, and a key value and encrypts a .blend file.

example: 

gamekit.exe -enc example.blend "kdjfdk8343943kidjfkdfjkdjfd"

output:

example.gamekit

Original issue reported on code.google.com by info.sin...@gmail.com on 13 Nov 2010 at 2:13

[GoogleCodeExporter]

actually, a key should not be given to gamekit.exe, the key should be hard 
coded somewhere in gamekit.exe code (not the best method but for it's intended 
purpose it's good enough).

so the process is:

gamekit.exe -enc example.blend

output:

example.gamekit

gamekit.exe contains the key, the key can be changed by person compiling 
gamekit.exe (i.e. changed inside source code). Gamekit still detects files 
based on extension.

so to encrypt above, gamekit.exe uses key embedded inside it.

When given a .gamekit file, it simply uses the same key inside to decrypt.

gamekit.exe example.gamekit        (detect file extension, use key inside to 
decrypt)

If you give gamekit.exe a normal .blend file, it plays it without using any 
encryption.

gamekit.exe example.blend          (detect file extension, play normally)

Original comment by info.sin...@gmail.com on 13 Nov 2010 at 11:15

[GoogleCodeExporter]

Thanks for your suggestion, and I think it is a necessary feature, too.
But I don't have enough time for this right now.

How about this one.

A user can register a callback handler about the input stream of a blend file 
before parsing it. If you want, you can decrypt the input file in the callback 
handler.

The callback handler will be called, here.
http://code.google.com/p/gamekit/source/browse/trunk/Engine/Loaders/Blender2/gkB
lendFile.cpp#110

Original comment by harkon...@gmail.com on 14 Nov 2010 at 9:35

[GoogleCodeExporter]

We could also add a simple function to the Blender loader API that take a 
stream instead of a file name and let the user encrypt/decrypt the way he wants.

However, I think info@sinsoft.com does not want it trought the API but wants a 
complete solution incorporated by default in the Runtime and some tools for non 
programmers to encrypt/decrypt easily.

I am not sure it fits inside our objective for gamekit and it does not makes 
much sense security wise to me. After all is the key is hardcoded in the 
executable, then anybody can retrieve it with a little debugging or 
reverse-engineering (in the case of a closed source app).

In this case with the source available and a unique key for all .blend of all 
user, one would just have to do a little search inside the source code and all 
blend files of every user would be compromised.

Original comment by xavier.thomas.1980@gmail.com on 15 Nov 2010 at 2:27

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

"In this case with the source available and a unique key for all .blend of all 
user, one would just have to do a little search inside the source code and all 
blend files of every user would be compromised."

right. So the idea is to let each user of gamekit hard code their own key 
inside the code (hardcoded) before shipping content. Meaning they have to be 
able to compile gamekit themselves. They also have to "hide the key somehow so 
that it's not retrievable". 

But I agree, it's not a good solution because if someone wanted to they can 
always reverse engineer the shipped app to find the key. 

I want the ability to encrypt files so "gamekit.exe -enc filename.blend" (spits 
out new encrypted file filename.gamekit), so that you can batch encrypt blend 
files for shipping. There should be no "gamekit.exe -dec filename.blend", 
because you always have the original blend files. The decryption happens in the 
app. At runtime, if gamekit.exe is fed a .gamekit (encrypted file), it detects 
the extension, and decrypts the file before playing normally.

Original comment by info.sin...@gmail.com on 15 Nov 2010 at 5:40

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

I think most shipping games don't encrypt their data, they just ship optimized 
data instead of the original resources (.blend, Maya .ma / mb, Max 3ds etc).

We could strip a lot of the data in a .blend file and save it as a new file 
format. Aside from stripping data, the data in the new format can be optimized 
for specific target hardware platform.
This new file format cannot be read by Blender.

Is that acceptable, or do you really require encryption?
Thanks,
Erwin

Original comment by erwin.coumans on 15 Nov 2010 at 7:49

[GoogleCodeExporter]

yeah that would be acceptable. But implementing something like this would 
require a lot of resources. Not exactly sure if it's within the gamekit project 
scope.

For me at least, I don't care if the end user has access to the mesh 
data/textures/etc. All I care about is hiding the gameplay code (i.e. lua code).

So the only points I care about:

-a regular non-technical user should not be able to open entire game level in 
Blender (so your solution would fix that)

-game script code (lua code) should be stored in bytecode, so that a more 
technical user should not be able to easily change script code (to cheat for 
example) in shipped files. However if the lua script code is stored in binary 
rather than text in this new file format, that would solve the problem as well.

Original comment by info.sin...@gmail.com on 15 Nov 2010 at 8:34

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Sorry, so to answer the question, no encryption is not necessary. As long as 
Blender is unable to read shipped files, and lua script is in some binary 
format (not text).

Original comment by info.sin...@gmail.com on 15 Nov 2010 at 8:41

[GoogleCodeExporter]

Lya bytecode is not portable, but if you only have a few platforms to deal with 
you could simply ship bytecode for each platform, next to their native 
executable. (for example iPhone, Windows, Linux, Mac OSX all 32bit)

Would that be acceptable?

Otherwise, encryption or obfuscation is an option as previously mentioned.

Original comment by erwin.coumans on 15 Nov 2010 at 10:19

[GoogleCodeExporter]

I just tried saving a lua file externally inside of Blender. When you save, 
close Blender, *delete* the lua file, open Blender again, it's still there in 
text format (embedded). But this could change if we have a new .gamekit file 
format.

So, I'm not sure how the compiled bytecode files will link to the original 
gamekit file, and if you embed them into gamekit files then they would be 
stored in text format?

((assuming here that we have a new format that GameKit can read which is 
.gamekit formats)).

If the lua files are stored inside .gamekit files, are they stored in text or 
are they in binary?

If it's in binary, then that would be acceptable. I really like the new file 
format idea. Encryption is not a good solution. Because once the key is found, 
the end user has access to the whole .blend file, including all lua scripts.

Original comment by info.sin...@gmail.com on 15 Nov 2010 at 10:33

[GoogleCodeExporter]

So I don't mind compiling the lua files for each platform. You can create a 
batch file to do all that. But will GameKit be able to tell and use the 
bytecode files instead. And will the new gamekit file format not have any of 
the scripts embedded inside?especially in text format?

Original comment by info.sin...@gmail.com on 15 Nov 2010 at 10:38

[GoogleCodeExporter]

I checked a sample blend file, it turns out blender stores text files inside in 
binary format. So this means lua byte code is not needed, we don't really need 
to worry about it. Embedding lua inside the files is better as it keeps it 
multiplatform and not need to compile to bytecode for a target platform.

So we could go ahead with stripping data in a .blend file idea and save it in a 
new file format (optimized?).

The only requirement now is that the new file format cannot be read by Blender.

Sinan

Original comment by info.sin...@gmail.com on 15 Nov 2010 at 11:44

[GoogleCodeExporter]

Did you unzip that sample.blend file first?

Original comment by erwin.coumans on 16 Nov 2010 at 2:35

[GoogleCodeExporter]

Yeah, it's not a zipped file. So when I "hex edit" a sample blend file with 
gvim, a blend file that contains text files inside (lua), you can't read the 
text files in either hex or plain file view. So I assume they are stored in 
binary.

I'm looking into encryption, but it's the wrong approach. Having a custom file 
format would be more preferable.

Original comment by info.sin...@gmail.com on 16 Nov 2010 at 4:43

[GoogleCodeExporter]

It's not a new file format, but provides blend file stripping.

It will read in a normal .blend file and reflect it back to disk without the 
structures contained in skipList. This will make the .blend unreadable by 
Blender because of the missing bScreen, ScrVert, ScrEdge structures. GameKit 
does not care whether or not these structures exist, therefore the file can 
still be read.

It is a more optimized .blend, in the sense that it will take less time to be 
read by GameKit. bScreen, ScrVert, and ScrEdge make up a big portion of data 
inside the .blend

The patch should be applied in trunk/Tools/FileTools

Original comment by snailr...@gmail.com on 16 Nov 2010 at 10:54

Attachments:

• BlendFileStrip.patch

[GoogleCodeExporter]

This feature is already done!?

In that case, this issue is now solved.

Original comment by info.sin...@gmail.com on 16 Nov 2010 at 11:41

[GoogleCodeExporter]

Thanks, Charlie.
I applied your patch to r786.

Original comment by harkon...@gmail.com on 19 Nov 2010 at 9:13

• Changed state: Fixed

[GoogleCodeExporter]

Use CMake FBT_BLEND_STRIP otpion for build.

Original comment by harkon...@gmail.com on 19 Nov 2010 at 10:17

[GoogleCodeExporter]

StripBlend remove the window layout data on the blend file.
So, Users can still open the stripped file on Blend file open in menu, but 
can't open by double click on Explorer.

Original comment by harkon...@gmail.com on 27 Nov 2010 at 2:21

• Changed state: Started

[GoogleCodeExporter]

Just UI data stripping can't prevent access from Blender.
r794 change the stripped file's headstring, but it isn't proper solution.
I think Gamekit custom file format is need for this.

By the way, UI data stripping reduce less than 1% of the momo_ogre.blend file 
size. It is smaller than expected.

Original comment by harkon...@gmail.com on 28 Nov 2010 at 11:53

[GoogleCodeExporter]

I just tried the new patched stripblend. It works very well now, with small and 
large files (MOMO).

So for now this is good enough. Yes advanced users can patch the file back to a 
blend file, but most regular users won't. The idea is stop regular users from 
accessing the files so easily. 

As for embedded script files, it seems that Lua Interpretor can detect a file 
and know whether its a text file lua code or byte code lua code. So it might be 
possible to embbed byte code files inside the blend so that they are unreadable 
by someone who does end up opening the file in Blender.

But, yes, I agree that a custom file format might be the answer here. But it 
may not be worth the effort, people can just write importers for that.

Original comment by info.sin...@gmail.com on 28 Nov 2010 at 5:55

[GoogleCodeExporter]

Cannot embed byte-code and have it work on all platforms. Lua byte-code is 
dependent on the OS & chipset being used to compile it. Unless someone patches 
GameKit Lua with a platform independent byte-code reading/writing interface - 
it won't work.

Original comment by BenT.Sol...@gmail.com on 28 Nov 2010 at 11:24

[GoogleCodeExporter]

Yes, you have to compile the lua code for the platform you're targeting and 
embed into their respective blend files. 

What I meant is that the lua interpretor will know if the file is text or byte 
code and act accordingly (that's according to the docs).

I am not against creating a custom file format (what ever works is fine with 
me).

Original comment by info.sin...@gmail.com on 29 Nov 2010 at 3:08

[GoogleCodeExporter]

Thanks for feedback.
Gamekit custom file format should be focused to reduce file size & loading time 
in the future.
This issue is closed.

Original comment by harkon...@gmail.com on 29 Nov 2010 at 4:36

• Changed state: Fixed