search

skadistats / clarity

Comically fast Dota 2, CSGO and CS2 replay parser written in Java.
BSD 3-Clause "New" or "Revised" License
629 stars 122 forks source link

Unknown Message Types #58

Open spheenik opened 8 years ago

spheenik commented 8 years ago

If you come here, you know why. Thanks for your help!

howardchung commented 7 years ago

unknown embedded message of kind SOURCE2/563. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58

Looks like the same message, should the latest version of clarity have the latest protobufs referenced?

spheenik commented 7 years ago

This is new. I will update the protobufs soon. After I've done this, it will work with all clarity versions.

spheenik commented 7 years ago

I have to retract my comment. It's not new, 563 is already mapped, and is a CDOTAUserMsg_SelectPenaltyGold. Make sure you have version 4.3 of clarity-protobuf.

xiaoyangmoa commented 7 years ago

Match ID: 3163437041 Running clarity-examples tick package. Happens on many ticks:

unknown embedded message of kind SOURCE2/568. Please report this in the correspon ding issue: #58

MichaelSnowden commented 7 years ago

Unknown embedded message of kind SOURCE2/568.

Match ID: 3191602004

WikiNeo commented 7 years ago

Unknown embedded message of kind SOURCE2/568

Match ID: 3175552353

Need to update clarity-protobuf?

eguven commented 7 years ago

unknown embedded message of kind SOURCE2/568

Match ID: 3209295791

spheenik commented 7 years ago

Pushed version 4.5 of the protobufs to maven central, should be available soon. Pushed new 2.2-SNAPSHOT of clarity.

Please test and report remaining issues. Thx.

STRATZ-Ken commented 7 years ago

3249880583 3249892973

spheenik commented 7 years ago

Thx @STRATZ-Ken, pushed new version 4.6 of protobufs which enable messages 564 and 565

WikiNeo commented 7 years ago

3307239021

unknown embedded message of kind SOURCE2/567

spheenik commented 7 years ago

clarity-protobuf-4.7: updated protobufs to include 567

jlfalcon3589 commented 6 years ago

Someone who could help me, I need help, I'm trying to parse a replay and this is the error that gives me:

Unknown embedded message of kind SOURCE2 / 127. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58

Sorry for my english...

spheenik commented 6 years ago

Thanks for the report. This is no error, but a warning, so you should still be able to parse the replay. Is it possible that you supply the replay to me, I would like to have a look at it. (Match ID would also suffice)

jlfalcon3589 commented 6 years ago

Thanks for replying, the game is Local so I can give you the MatchID, sorry but my internet connection does not allow me to upload such a large file. I'm using Dota 7.06e version 2.295 to save the replay

jlfalcon3589 commented 6 years ago

Regards, the problem I had when parsing the file is because I am using a replay saved manually in the console, as well explain @ykim at https://github.com/skadistats/clarity/issues/36, I have now tested with a replay unloaded from valve and the parser works perfectly.

I have a question, you do not know how I can parse the .dem that creates the game when I record it using the commands of the console.

Greetings and thanks in advance.

spheenik commented 6 years ago

It should work in both cases. Again: the "unknown embedded message"-message is just a warning! The parse should finish, with the parser skipping over the unknown message.

Noxville commented 6 years ago

unknown embedded message of kind SOURCE2/555. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58

unknown embedded message of kind SOURCE2/543. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58

Two newer ones ^

spheenik commented 6 years ago

added 543 and 555 just now.

STRATZ-Ken commented 6 years ago

Therrrrrrreeee baccckkkkkk.

[pool-3-thread-2] WARN clarity.runner - unknown embedded message of kind SOURCE2/145. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58 [pool-3-thread-2] WARN clarity.runner - unknown embedded message of kind SOURCE2/145. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58 [pool-3-thread-2] WARN clarity.runner - unknown embedded message of kind SOURCE2/145. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58 [

spheenik commented 6 years ago

need replay

STRATZ-Ken commented 6 years ago

Pretty sure literally everyone. I had to turn off logging because it made 55gb of a log in a matter of hours.

Sent from my iPhone

On Mar 13, 2018, at 3:51 AM, Martin Schrodt notifications@github.com<mailto:notifications@github.com> wrote:

need replay

- You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/skadistats/clarity/issues/58#issuecomment-372576417, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AX0vnYsvHwWh9RZwl7-VbZXqS27EtU3eks5td3qWgaJpZM4F8c5V.

spheenik commented 6 years ago

Pushing clarity-protobuf 4.9 just now. Should be available via maven central soon.

Hmm, I wonder whether I should only log this message once per replay?

spheenik commented 6 years ago

yea, did so in the newest 2.3 snapshot.

Ruijunzhang commented 6 years ago

unknown embedded message of kind SOURCE2/572. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58

spheenik commented 6 years ago

Thanks for the report. I enabled the missing message with this commit, and uploaded a new version of the protobufs, which should be publicly available soon.

p0d3x commented 6 years ago

unknown embedded message of kind SOURCE2/576. Please report this in the corresponding issue: #58

spheenik commented 6 years ago

@p0d3x: Damn, reminds me of a promise I made a while ago to fucking release! Sorry, swamped atm, will try to come to working over the protobufs and releasing on the next weekend!

Oddj0b commented 6 years ago 
Exception in thread "main" java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.simontuffs.onejar.Boot.run(Boot.java:340)
    at com.simontuffs.onejar.Boot.main(Boot.java:166)
Caused by: java.lang.IllegalArgumentException
    at java.nio.Buffer.position(Buffer.java:244)
    at skadistats.clarity.source.MappedFileSource.setPosition(MappedFileSource.java:42)
    at skadistats.clarity.model.engine.AbstractDotaEngineType.determineLastTick(AbstractDotaEngineType.java:35)
    at skadistats.clarity.source.Source.determineLastTick(Source.java:180)
    at skadistats.clarity.source.Source.getLastTick(Source.java:194)
    at skadistats.clarity.processor.runner.ControllableRunner.getLastTick(ControllableRunner.java:342)
    at skadistats.clarity.examples.matchend.Main.<init>(Main.java:35)
    at skadistats.clarity.examples.matchend.Main.main(Main.java:24)
    ... 6 more
unknown top level message of kind SOURCE2/1983. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58
unknown top level message of kind SOURCE2/47. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58
unknown top level message of kind SOURCE2/34. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58
unknown top level message of kind SOURCE2/-65. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58
unknown top level message of kind SOURCE2/4888. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58
spheenik commented 6 years ago

@Oddj0b: This looks to be a corrupt replay file.

Oddj0b commented 6 years ago

@spheenik The same assumption I had, but tried it out with five replays, and they all returned the same error. I am using examples from https://github.com/skadistats/clarity-examples and noticed when I ran java -jar target/matchend.one-jar.jar /Users/villarsgimm/Dropbox/Dota\ 2/3981130203.dem it returned the error from https://github.com/skadistats/clarity/issues/58#issuecomment-401563701 but running java -jar target/combatlog.one-jar.jar /Users/villarsgimm/Dropbox/Dota\ 2/3981130203.dem worked fine.

spheenik commented 6 years ago

added SOURCE2/576 in clarity-protobuf 4.12

spheenik commented 6 years ago

@Oddj0b: Yes, the matchend example reads entity data, while the combat log example doesn't, so if there's corruption in the entity data, you won't notice with the combat log example.

Oddj0b commented 6 years ago

@spheenik Makes sense, guessing private lobbies doesn't play well as all five replays have this issue.

armoucar commented 5 years ago

runner.getLastTick() is still broken on the matchend example. For some reason, it works on clarity-analyzer with the clarity@2.4-SNAPSHOT.

spheenik commented 5 years ago

@armoucar: Good find, my CSGO refactoring did introduce a bug that I did not find because of lazy testing.... Fixed and pushed a new 2.4-SNAPSHOT.

12yuens2 commented 5 years ago

unknown embedded message of kind SOURCE2/577. Please report this in the corresponding issue: #58 unknown embedded message of kind SOURCE2/575. Please report this in the corresponding issue: #58

spheenik commented 5 years ago

@12yuens2 Thanks for the report. I added the missing messages and pushed 4.13 to maven-central just now, so it should be available shortly.

Beaker12 commented 5 years ago 
$ java -jar target/combatlog.one-jar.jar ../data/csgo/match730_003258109643968217243_1770483764_900.dem 

unknown embedded message of kind CSGO/125. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58

unknown embedded message of kind CSGO/35. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58
total time taken: 2.019s
spheenik commented 5 years ago

@Beaker12: This is a consequence of a little bug with CSGO parsing that I just now fixed. Be aware though that there is no combat log in CSGO replays, so after fixing the bug it just will stop complaining about unknown messages, but there will be no other output. Look into GameEvents @OnGameEvent to get something like a combat log.

erGOD commented 5 years ago

unknown embedded message of kind CSGO/126. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58

That.s on 2.3, on 2.4 snapshot: Exception in thread "main" skadistats.clarity.ClarityException: oops. no provider found for required usage point interface skadistats.clarity.processor.packet.UsesPacketReader

I'm using the samples on the latest csgo demos from hltv.org

spheenik commented 5 years ago

@erGOD Need a link to the demo.

erGOD commented 5 years ago

https://www.hltv.org/download/demo/46195 was the one I tried. chaos-vs-pact-m1-overpass.dem specifically. Some of the examples (eg GameEvents) work, but others don't.

When I used 2.3 initially it said "Exception in thread "main" java.lang.UnsupportedOperationException: kind 69 not implemented" initially. Switching to 2.4-SNAPSHOT seems to have fixed that.

Let me know if you need more information. Tx!

spheenik commented 5 years ago

I ran chaos-vs-pact-m1-overpass.dem through 2.4-SNAPSHOT here with no problems whatsoever. There's no unknown embedded message of kind CSGO/126 found here. Can you provide more detailed steps for reproduction?

erGOD commented 5 years ago

I guess we should ignore the 126 error as it was only seen on 2.3. The no provider found issue was fixed by setting maven not to use workspace dependencies (even though clarity should have been on master.

The info package seems to fail with the latest from master however but this might just be a dota2/csgo thing. (log file included) clarityTestInDos.txt

Perhaps this is simply due to me not understanding the interface, but when I add a @OnPlayerInfo public void onPlayerInfo(PlayerInfoType info) { ... }

to the gameevent Main.java class example, I get this:

Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at com.simontuffs.onejar.Boot.run(Boot.java:340) at com.simontuffs.onejar.Boot.main(Boot.java:166) Caused by: java.lang.IllegalArgumentException: bad spread array length at java.lang.invoke.MethodHandleStatics.newIllegalArgumentException(Unknown Source) at java.lang.invoke.MethodHandle.asSpreaderChecks(Unknown Source) at java.lang.invoke.MethodHandle.asSpreader(Unknown Source) at skadistats.clarity.event.AbstractInvocationPoint.bind(AbstractInvocationPoint.java:57) at skadistats.clarity.processor.runner.ExecutionModel.bindInvocationPoints(ExecutionModel.java:193) at skadistats.clarity.processor.runner.ExecutionModel.initialize(ExecutionModel.java:289) at skadistats.clarity.processor.runner.AbstractRunner.initWithProcessors(AbstractRunner.java:45) at skadistats.clarity.processor.runner.AbstractFileRunner.initAndRunWith(AbstractFileRunner.java:37) at skadistats.clarity.processor.runner.SimpleRunner.runWith(SimpleRunner.java:36) at skadistats.clarity.examples.gameevent.Main.run(Main.java:29) at skadistats.clarity.examples.gameevent.Main.main(Main.java:35) ... 6 more

marksantos22 commented 4 years ago

hi, i got this message when parsing a replay in dota 2 unknown embedded message of kind SOURCE2/593. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58 Working around keyHistory underflow. Key '1' in table 'ActiveModifiers' is incomplete. Working around keyHistory underflow. Key '2' in table 'ActiveModifiers' is incomplete. Working around keyHistory underflow. Key '3' in table 'ActiveModifiers' is incomplete. Working around keyHistory underflow. Key '4' in table 'ActiveModifiers' is incomplete. Working around keyHistory underflow. Key '5' in table 'ActiveModifiers' is incomplete. Working around keyHistory underflow. Key '6' in table 'ActiveModifiers' is incomplete. Working around keyHistory underflow. Key '7' in table 'ActiveModifiers' is incomplete. Working around keyHistory underflow. Key '8' in table 'ActiveModifiers' is incomplete. Working around keyHistory underflow. Key '9' in table 'ActiveModifiers' is incomplete. Working around keyHistory underflow. Key '1' in table 'ActiveModifiers' is incomplete. Working around keyHistory underflow. Key '5' in table 'ActiveModifiers' is incomplete. Working around keyHistory underflow. Key '6' in table 'ActiveModifiers' is incomplete. match id: 5129833179

chrisrun commented 4 years ago

Hi, when parsing https://drive.google.com/open?id=1hk8nh9zxb2EzBQr2FMI6G0W-ueTG9z80 I got the following error: unknown embedded message of kind SOURCE2/571. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58

spheenik commented 4 years ago

I finally came around to bringing the protobufs up to date today. This should fix the recent unknown messages. There are several new ones that might be interesting, for example CDOTAUserMsg_FoundNeutralItem, CDOTAUserMsg_OutpostCaptured, CDOTAUserMsg_OutpostGrantedXP

spheenik commented 4 years ago

@marksantos22 Those messages you posted there, working around keyHistory underflow are from an issue that I never could solve. But I checked again today, and found that the parsing of sendtables was not correct for over 2 years now.

The bug basically made modifiers not update correctly in some cases. Since many people do not use modifiers, they probably do not care, but if you do, I suggest you update to the newest 2.5-SNAPSHOT, which contains the fix.

huertasdanny commented 4 years ago 
unknown embedded message of kind SOURCE2/523. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58
unknown embedded message of kind SOURCE2/540. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58
unknown embedded message of kind SOURCE2/529. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58
unknown embedded message of kind SOURCE2/574. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58
unknown embedded message of kind SOURCE2/556. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58
Runner thread crashed
java.lang.InterruptedException: null
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[na:1.8.0_242]
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[na:1.8.0_242]
    at skadistats.clarity.processor.runner.ControllableRunner$1.doLoopControl(ControllableRunner.java:64) ~[clarity-2.5-SNAPSHOT.jar:na]
    at skadistats.clarity.processor.runner.LoopController.doLoopControl(LoopController.java:35) ~[clarity-2.5-SNAPSHOT.jar:na]
    at skadistats.clarity.processor.runner.ControllableRunner$LockingLoopController.doLoopControl(ControllableRunner.java:160) ~[clarity-2.5-SNAPSHOT.jar:na]
    at skadistats.clarity.processor.reader.InputSourceProcessor.processSource(InputSourceProcessor.java:131) ~[clarity-2.5-SNAPSHOT.jar:na]
    at skadistats.clarity.event.AbstractInvocationPoint.invoke(AbstractInvocationPoint.java:86) ~[clarity-2.5-SNAPSHOT.jar:na]
    at skadistats.clarity.event.Event.raise(Event.java:37) ~[clarity-2.5-SNAPSHOT.jar:na]
    at skadistats.clarity.processor.runner.AbstractFileRunner.initAndRunWith(AbstractFileRunner.java:39) ~[clarity-2.5-SNAPSHOT.jar:na]
    at skadistats.clarity.processor.runner.ControllableRunner.lambda$runWith$0(ControllableRunner.java:238) ~[clarity-2.5-SNAPSHOT.jar:na]
    at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_242]