Unknown Message Types - Githubissues

spheenik commented 8 years ago

If you come here, you know why. Thanks for your help!

huertasdanny commented 4 years ago

another one but from using combat log example:

unknown embedded message of kind SOURCE2/523. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58
unknown embedded message of kind SOURCE2/540. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58
unknown embedded message of kind SOURCE2/529. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58
unknown embedded message of kind SOURCE2/574. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58
unknown embedded message of kind SOURCE2/556. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58

Noxville commented 4 years ago

These look like user-recorded demos, right? Most of the message types listed are normally not saved in the replay (just broadcast to teammates).

huertasdanny commented 4 years ago

Yes, these are recorded using the DOTA console. The output for the combat_log example though doesn't output any data other than the unknown embedded message which I find odd. I included one of the replayfile I used (small enough to be uploaded)

. match1.log

spheenik commented 4 years ago

@huertasdanny: Enabled the missing messages in clarity-protobuf 4.20, which I uploaded to maven central right now.

PeaXCD commented 4 years ago

match_id: 5305773896 unknown embedded message of kind SOURCE2/571.

spheenik commented 4 years ago

@PeaXCD: Pushing 4.21 now, which has the message enabled.

omorgan7 commented 4 years ago

Since yesterday's battle pass update:

unknown embedded message of kind SOURCE2/597

https://gofile.io/d/s1ywnr

spheenik commented 4 years ago

@omorgan7: Thanks, I suspected such...

I was quick today, and am pushing a new version (4.24) of the protobufs to central as we speak.

latschen commented 4 years ago

unknown usermessage message of kind CSGO/75

damit du was zu tun hast :-*

AadithSrinivasan commented 4 years ago

unknown embedded message of kind SOURCE2/604. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58

p0d3x commented 4 years ago

unknown embedded message of kind SOURCE2/579.

spheenik commented 4 years ago

Sorry for taking so long - I'm a lazy ass.

CSGO/75 as well as SOURCE2/579 and SOURCE2/604 added.
released clarity-protobuf 4.25

pushing to central right now.

p0d3x commented 4 years ago

unknown usermessage message of kind CSGO/69

p0d3x commented 3 years ago

unknown embedded message of kind SOURCE2/605 unknown embedded message of kind SOURCE2/606

nul0m commented 3 years ago

unknown embedded message of kind SOURCE2/144 unknown embedded message of kind SOURCE2/106.

spheenik commented 3 years ago

Version 4.26 released:

add CSGO/69

add SOURCE2/106 add SOURCE2/144 add SOURCE2/605 add SOURCE2/606

Markxy commented 3 years ago

unknown embedded message of kind SOURCE2/610

spheenik commented 3 years ago

Version 4.27 released:

add SOURCE2/612 (DOTA_UM_ChatMessage)

latschen commented 3 years ago

unknown embedded message of kind SOURCE2/611

ghost commented 2 years ago

CSGO/11 on map 3 (dust2) of replay: https://demos.hltv.org//demo/demofiles/demosection/United-Masters-League-Season2-sprout-vs-giants-bo3.rar

p0d3x commented 2 years ago

unknown embedded message of kind SOURCE2/114

Tsury commented 2 years ago

SOURCE2/114 SOURCE2/496 SOURCE2/608 SOURCE2/499 SOURCE2/562 SOURCE2/531

reacheight commented 2 years ago

SOURCE2/154

Dota 2 match 6742168610

STRATZ-Ken commented 1 year ago

[pool-2-thread-4] WARN clarity.runner - unknown embedded message of kind SOURCE2/154. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58

xiaoyangmoa commented 1 year ago

unknown embedded message of kind SOURCE2/154. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58 match is : 6792799128

Dmytro4 commented 1 year ago

6792839364 [unknown embedded message of kind SOURCE2/154. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58]

Dmytro4 commented 1 year ago

updating clarity dependencies in pom.xml seems to be fixed the issue (I missed version release update, sorry)

reacheight commented 1 year ago

SOURCE2/617 SOURCE2/616

6856094991

Whoeza commented 1 year ago

HI hi! 14:18:04.634 WARN unknown embedded message of kind **SOURCE2/154**. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58 14:18:04.673 ERROR Runner thread crashed java.lang.ArrayIndexOutOfBoundsException: Index 5 out of bounds for length 5 Let me know if I can help. Cheers o/

spheenik commented 1 year ago

The exception is unrelated to the warning. Can you provide the replay?

Whoeza commented 1 year ago

The exception is unrelated to the warning. Can you provide the replay?

This is the match ID: 6890930297 Glad if I can help

spheenik commented 1 year ago

Thanks, will look into it

Whoeza commented 1 year ago

Can I ask what is an unknown message type? :)

spheenik commented 1 year ago

At the lowest level, the replay is a collection of "messages", or "packets", which are encoded with potobuf. On disk, only a packet type (as an int) is stored, so clarity needs to know, which protobuf message type has to be decoded how. This is done with the definitions in the package https://github.com/skadistats/clarity-protobuf.

So when clarity encounters a package type, it has no definition for, it warns you with this message. Valve add new message types every once in a while, and I have to update clarity-protobuf accordingly, which I did not do for quite a while, since I am a lazy bastard.

Whoeza commented 1 year ago

So you have to find the definition somewhere, (but where?) and update clarity-protobuf. I see that the repository has not been updated for 2 years, how is that possible? I see that this issue thread is quite active... Does it have to do with Maven Central? Like, clarity-protobuf being updated on Maven and not on GitHub?

Can I help keeping clarity-protobuf up to date?

spheenik commented 1 year ago

There is a project on Github that extracts the protobuf definitions from the game regularly.

https://github.com/SteamDatabase/GameTracking-Dota2

My approach is not to simply copy their definitions completely, since many of them are not needed for parsing demos, and sometimes they remove fields also, and I keep those fields in my definitions, since I want older replays to also be able to be read.

It's true I didn't update for a long time, and should do so. But since the "core definitions" do not change, and there are mostly new UserMessages that will not be of interest for replay parsing anyway, I just figured it's priority B.

Whoeza commented 1 year ago

There is a project on Github that extracts the protobuf definitions from the game regularly.

https://github.com/SteamDatabase/GameTracking-Dota2

My approach is not to simply copy their definitions completely, since many of them are not needed for parsing demos, and sometimes they remove fields also, and I keep those fields in my definitions, since I want older replays to also be able to be read.

I see, your approach has some careful steps to take. I propose you and/or I could make a page in the wiki on this repository (or clarity-protobufs) in GitHub and explain the steps. So if I or someone else who wanted to help keeping clarity-protobufs updated, could follow those steps and produce pull requests.

How does it sound to you?

It's true I didn't update for a long time, and should do so. But since the "core definitions" do not change, and there are mostly new UserMessages that will not be of interest for replay parsing anyway, I just figured it's priority B.

I followed your advice from time ago and started from clarity-examples and clarity-analyzer to be able to understand how to parse game events. Until I learn to do it, I wanted to use Clarity Analyzer to read through a replay quickly, to understand what variables I can parse with clarity. But these unknown message types make Analyzer crash. The next best thing is to use clarity-examples to dump the entire replay, which is not great for a beginner like me, but I make do with what I have.

I'd be glad to help in the little free time I have. PS: when submitting in this issue 58, is it enough to post just one match ID?

spheenik commented 1 year ago

Unknown message ids should not make analyzer crash. The crash is most surely related to something else.

Whoeza commented 1 year ago

Unknown message ids should not make analyzer crash. The crash is most surely related to something else.

Ah, sorry. I did not understand it. That's what you meant when you said the warning and the exception are not related to each other.

Can I help with the crash as well? Should I open a new issue for the crash to separate it from this thread?

spheenik commented 1 year ago

The analyzer crash is most certainly because it uses an old version of clarity. It should work if it uses 2.7.5.

Whoeza commented 1 year ago

The analyzer crash is most certainly because it uses an old version of clarity. It should work if it uses 2.7.5.

Oh thank you, it was that... it works now.

Let me know if I can help with the other stuff we talked about, glad if I can.

darrensapalo commented 1 year ago

Detailing the above details here: faq

I too am experiencing the above mentioned issue:

unknown embedded message of kind SOURCE2/154. 
Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58

Using git repo: clarity-examples @ commit Clarity version: 2.7.5 commit

with replay ID: 6963982486.dem, but as @spheenik explained above, he still needs to (1) extract the new protobuf definitions and (2) tweak the definitions to enable backwards compatible replay parsing.

Let me know if I misunderstood anything 🙇

Whoeza commented 1 year ago

Detailing the above details here: faq

I too am experiencing the above mentioned issue:
unknown embedded message of kind SOURCE2/154. 
Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58
Using git repo: clarity-examples @ commit Clarity version: 2.7.5 commit

with replay ID: 6963982486.dem, but as @spheenik explained above, he still needs to (1) extract the new protobuf definitions and (2) tweak the definitions to enable backwards compatible replay parsing.

Let me know if I misunderstood anything 🙇

Thank you for starting the wiki! I did not know if we were allowed to do it. About your understanding, that's what I got as well.

spheenik commented 1 year ago

New protobufs have been released.

Added SOURCE2/114 (CUserMessageRequestState) SOURCE2/154 (CUserMessageServerFrameTime) SOURCE2/496 (CDOTAUserMsg_SwapVerify) SOURCE2/499 (CDOTAUserMsg_ItemAlert) SOURCE2/531 (CDOTAUserMsg_PlayerMMR) SOURCE2/562 (CDOTAUserMsg_KillcamDamageTaken) SOURCE2/608 (CDOTAUserMsg_AghsStatusAlert) SOURCE2/616 (CDOTAUserMsg_DuelOpponentKilled) SOURCE2/617 (CDOTAUserMsg_DuelAccepted)

My testing capability was limited, since downloading replays through the client seems buggy atm. (Opendota also does not work).

Regarding the Wiki Thing: Doing a FAQ there is probably really a good thing. @darrensapalo Mind if I change your entry to properly explain this?

spheenik commented 1 year ago

Oh, and, the protobufs contain several extra combatlog properties. If you want to use them, I released clarity 2.7.6. Pushed both artifacts to central five minutes ago, so might be an hour or so before you see them.

darrensapalo commented 1 year ago

@spheenik Please do! I'll follow your lead on the docs. Thanks spheenik!

p0d3x commented 7 months ago

unknown top level message of kind CSGO_S2/16

also no decoder for class CCSPlayerController at 19/11! since the update today

lochetti commented 4 months ago

Not sure how to be more helpful, but I got a WARN clarity.runner - unknown top level message of kind CSGO_S2/16. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58

I was trying to parse the first map of this game https://www.hltv.org/matches/2370928/liquid-vs-faze-iem-chengdu-2024 Using the version 3.0.3

BusyLivingDota commented 4 months ago

unknown top level message of kind CSGO_S2/16. Please report this in the corresponding issue: https://github.com/skadistats/clarity/issues/58 on tick 152 of the cs2 replay: https://www.hltv.org/matches/2370727/faze-vs-natus-vincere-pgl-cs2-major-copenhagen-2024 map-3-Inferno

spheenik commented 4 months ago

Thank you guys, CSGO_S2/16 noted, I'll put updating the protobufs to my not so small list of todos.

skadistats / clarity

Unknown Message Types #58