LZMA does not work (included under lime.utils)

[Jarrio]

Hello,

When attempting to decode using LZMA using the following code: var base = Base64.decode(content); var decodeLzma = LZMA.decode(base); (base results in haxe bytes)

results in the following compiler error:

error C3083: 'lime': the symbol to the left of a '::' must be a type error C3083: 'utils': the symbol to the left of a '::' must be a type error C3083: 'LZMA_obj': the symbol to the left of a '::' must be a type error C2039: 'decode' : is not a member of '`global namespace'' error C2064: term does not evaluate to a function taking 1 arguments

I am unaware of whether if this is a problem with my code, but searching for these errors suggests a compiler error, as these error codes are not unique to haxe, I am targetting Windows.

[Gama11]

What Haxe version are you using? Seems to compile for me using the latest dev build (4a6fdc37a95a87bccb3171be4a5354e692bc3d3b).

[Jarrio]

I am running 3.2.1 stable release, I do not know how to compile from source so I tend to avoid the dev builds and they usually require a ton of external things to install. But I may have to consider it.. Thanks for that.

• I found the dev build list, I'll install the new version now to see if it compiles for me.

[Jarrio]

I am unable to confirm whether this build will fix my issue as I get the following error(which halts the compile):

export/windows/cpp/haxe/ApplicationMain.hx:80: characters 47-65 : Warning : Use programPath instead Error: Could not find include file "P:/HaxeToolkit/hxcpp/3,2,205/src/hx/libs/std/Build.xml"

I think I'll assume the new build will fix my problem or my code is bad. I'll see if anyone has any alternative solutions in a little while. Thanks for your help

[nadako]

That error most probably means that you need latest git version of hxcpp: haxelib git hxcpp https://github.com/HaxeFoundation/hxcpp

[Jarrio]

Thanks, although that stopped showing the above error I now get the following:

Error: Source path "P:\HaxeToolkit\hxcpp/git/bin/Windows/std.dll" does not exist

[nadako]

Right, when using hxcpp from git you need to build something, I think it's described in the readme: https://github.com/HaxeFoundation/hxcpp

[Jarrio]

Yes, I just realised that. I was able to get it all running and I'm still getting the error, so I believe it's my code. I apologise for the thread, there was just limited information around and the error was unusual so I thought something was wrong with the compiler as it seems like no-one uses LZMA in haxe!

[Simn]

I'm assigning to Hugh because I'm not sure if this is an issue or not.

[Jarrio]

I am unsure whether it is an issue with my code or not at this point as I'm working with an external API which has limited documentation, I have contacted the author to find out.

Could someone explain what the error:

./src/PlayState.cpp(128) : error C3083: 'lime': the symbol to the left of a '::' must be a type ./src/PlayState.cpp(128) : error C3083: 'utils': the symbol to the left of a '::' must be a type ./src/PlayState.cpp(128) : error C3083: 'LZMA_obj': the symbol to the left of a '::' must be a type ./src/PlayState.cpp(128) : error C2039: 'decode' : is not a member of '`global namespace'' ./src/PlayState.cpp(128) : error C2064: term does not evaluate to a function taking 1 arguments

Means? My conclusion of it being with the compiler is due to the out of character error resonse, usually errors are lot more vague, compile and then crash. Whenever referencing LZMA.decode() it doesn't even compile at all. (I am now on the latest dev build)

My code is the following: var content = Json.parse(loadFile.readAll().toString()); var decode = Base64.decode(content.content); var lzma = LZMA.decode(decode);

So I am definitely feeding the method bytes, the only alternative is I'm not feeding it LZMA compressed data, but would it result in the error above?

[Gama11]

It mostly means that Haxe generates invalid cpp code which then causes the errors during native compilation. This is usually a Haxe / hxcpp bug.

[hughsando]

Yes, looks like a hxcpp issue with missing header, unless it is using some funky untyped code.

On Tue, Apr 12, 2016 at 2:08 AM, Gama11 notifications@github.com wrote:

It mostly means that Haxe generates invalid cpp code which then causes the errors during native compilation. This is usually a Haxe / hxcpp bug.

— You are receiving this because you were assigned. Reply to this email directly or view it on GitHub https://github.com/HaxeFoundation/haxe/issues/5092#issuecomment-208481151

[hughsando]

Actually, can you link to (or paste) the PlayState.hx that generates the PlayState.cpp ?

[Jarrio]

Yea sure: class PlayState extends FlxState { var beatmap = "658127"; var cache = "/assets/data/";

override public function create():Void { super.create(); var loadFile = File.read(cache + beatmap + ".txt"); var content = Json.parse(loadFile.readAll().toString()); var decode = Base64.decode(content.content); var lzma = LZMA.decode(decode); var getData = decode.getData(); ` trace("LZMA: " + lzma); trace("Get Data " + getData); } }`

and the contents of the text file is located: Here

On a side note, I've just received a response from the owner of the API I'm using and he mentioned that the API has outdated information on it and I am getting rawdata back after decoding it from base64 so LZMA would not work in this case, but if this helps catch an unexpected error, great!

Edit: After trying to use LZMA with some actual compressed data I am still receiving this error

I am able to get this same error (except with encode as well) completely irrelevant to the data I use: var test1 = LZMA.encode(data); var test2 = LZMA.decode(test1); trace("Test1:" + test1); trace("Test2:" + test2);

Does not compile at all and results in the above error

[hughsando]

This one is freaking my out. The LZMA class seems to be acquiring the "@:extern" meta tag when compiled though lime. I have absolutely no clue where this is coming from, maybe @jgranick can help. I'm not sure if the conventions have changed, or something else.

[Jarrio]

Are you able to reproduce the error or is it just me?

[hughsando]

If I use the LZMA.decode with dummy bytes from an empty "Main" class I get: haxelib run openfl ..... -> error haxe -main Main -lib lime -> no error

Even if I remove most of the functions from the LZMA class, I still get the error. I have not tried altering the name of this class yet.

Something in openfl is adding "extern" metadata to this class. I'm not sure if this is because the haxe has changed, or it always did this and hxcpp just ignored it.

On Thu, Apr 14, 2016 at 2:25 AM, Jazz914 notifications@github.com wrote:

Are you able to reproduce the error or is it just me?

— You are receiving this because you were assigned. Reply to this email directly or view it on GitHub https://github.com/HaxeFoundation/haxe/issues/5092#issuecomment-209580021

[Jarrio]

Oh that's unusual, I thought it was just a lime class. Do you happen to know an alternative way to decompress LZMA data? I couldn't seem to find anything obvious in the haxe/lime documentation.

[Jarrio]

Also if it's of any use, here's the full console output during compile: (I also threw in the encode just to get the error on paper).

Edit: I was just looking at the error in the above link and noticed it's complaining about lines 176 and 179, Thing is though the encode, decode statements are not on these lines, instead it's: 176: override public function update(elapsed:Float):Void { 179:

I'm guessing this has to do with how haxe compiles code, but usually line numbers are accurate. Just thought I'd mention it, in case something fishy is happening.

Edit #2: I forgot to mention I am using haxeflixel as well

[Jarrio]

I don't meant to be rude or if this is against any rules but I would just like to know if this is an accepted issue or not, as I would like to start looking for solutions(or alternative solutions, like externally decoding it) if this is just something wrong with my project.

Thanks again

[hughsando]

I have posted on the openfl page asking for help. Debugging someone else's macro is not very fun, since it could be run from pretty much any source file. I can possibly just remove the @:extern meta data feature - not sure it is needed anymore, but I would like to understand why openfl is doing this. I guess if they do not want to help I will have to progress without them.

On Thu, Apr 21, 2016 at 6:23 AM, Jazz914 notifications@github.com wrote:

I don't meant to be rude or if this is against any rules but I would just like to know if this is an accepted issue or not, as I would like to start looking for solutions(or alternative solutions, like externally decoding it) if this is just something wrong with my project.

Thanks again

— You are receiving this because you were assigned. Reply to this email directly or view it on GitHub https://github.com/HaxeFoundation/haxe/issues/5092#issuecomment-212636933

[Jarrio]

I appreciate your response, I think I may have more information. I was just continuing with alternative parts of my project when I needed to load some audio, for some reason I couldn't load audio through openfl or haxeflixel(it still compiled so not an issue for you guys). I was browsing the auto complete list and noticed lime had a way to load audio too, I wrote the following code: var test = new AudioBuffer(); test.src = path; var test = new AudioSource(test);

Which resulted in a very similar error message (and lack of a full compile): The full console output

[hughsando]

I wonder if lime is adding @:extern for flash asset loading, and then this shows up in hxcpp too. This makes sense for asset data - not so sure about lzma, but it might be related.

On Thu, Apr 21, 2016 at 9:31 AM, Jazz914 notifications@github.com wrote:

I appreciate your response, I think I may have more information. I was just continuing with alternative parts of my project when I needed to load some audio, for some reason I couldn't load audio through openfl or lime (it still compiled so not an issue for you guys). I was browsing the auto complete list and noticed lime had a way to load audio too, I wrote the following code: var test = new AudioBuffer(); test.src = path; var test = new AudioSource(test);

Which resulted in a very similar error message (and lack of a full compile): The full console output http://puu.sh/opZgt/fc1c5d5ebf.txt

— You are receiving this because you were assigned. Reply to this email directly or view it on GitHub https://github.com/HaxeFoundation/haxe/issues/5092#issuecomment-212683930

[hughsando]

So this is really hard to track down, bit I don't think it is a hxcpp issue. Findings so far: A simple openfl app that references LZMA.decode works. If I add <haxeflag name="-lib" value="flixel" /> it still works. If I add <haxelib name="flixel" />it breaks. The difference being that openfl imports the include.xml I think.

The symptom can be see with -D dump with the extern attribute before its even got to hxcpp extern class lime.utils.LZMA{

If I change the name to "LZMA1" and remove the build macro it still breaks. If I move it to the local directory, it works.

So, something in flixel that marks a class in lime.utils as "extern" that I know it does not even use (because I changed the name).

Ok - bit more info - you don't need flixel if you do this in the project:

    <haxelib name="openfl" />
    <set name="legacy" />
    <haxelib name="openfl" />

Not sure what happens here.

So one possible solution would be to remove an explicit reference to openfl in your project when using flixel.

My guess is a path conflict looking for lime.util files.

@Simn any ideas how cl_extern is getting set?

[hughsando]

OK, looking at the generated build.xml: --macro exclude('lime') aaaaaaaaaaagggggghhhhhhh! I want 2 hours of my life back. This is a lime bug.

[jgranick]

If a project uses legacy, it should not use lime.* classes

Do you think there's a way to break compilation rather than having it flagged as extern by the compiler?

[Jarrio]

@jgranick I'm not sure if that's directed at me, but I'm not really sure what you mean by that. I don't have much knowledge about behind the scenes stuff and I haven't typed "@:extern" any where manually. All I did was setup a haxeflixel project using cmd line, and wanted to use Lzma (or audiobuffer apparently).

[jgranick]

Flixel forces OpenFL "legacy" mode by default, you'll have to use <set name="next" /> before you include the Flixel haxelib in your project, I believe, to change that behavior.

I would prefer this error case to be a Haxe compile error, I wonder if anyone has ideas about how we could do that

[hughsando]

You could --reamap lime:openfl, which would work in this case and break where is it not possible. Or you could run a macro that throws an error, rather than "excludes" when you get to a class in the lime package. You could also "#if legacy #error" in all your lime files, but that would be a lot of changes.

[hughsando]

Also, the lime tool could simply detect where legacy is used in the same time it puts -classpath lime on the command line, and stop it before it even gets to haxe.

[Jarrio]

I would like to state for anyone who comes across this issue, that the solution @jgranick suggested fixes the issues mentioned.

Thanks again!