Streaming: GTA5_b3095.exe!sub_14131068C (0xc8)

jxnnyo commented 1 week ago

What happened?

I experienced a crash after adding a new clothing pack that was significantly larger than the previous one. This crash consistently occurred upon loading the game with the new pack. The issue is not limited to clothing; adding a large number of other streaming files, such as new cars, also results in the same crash.

CfxCrashDump_2024_06_26_14_03_02.zip

Expected result

The game should be able to handle the new streaming files, regardless of their type or size, without crashing.

Reproduction steps

Add a large amount of streaming files to the server (in my case a large clothing pack).
Start the server and attempt to load the game.
Observe the crash occurring after the new pack is added.
Stop other streaming resources (such as cars, weapons, or other clothing packs).
Observe that the game no longer crashes.
Start the same streaming resources back up
Observe the same crash occurring.

Importancy

Crash

Area(s)

FiveM

Specific version(s)

FiveM 8695, Server 8695/Windows

Additional information

The crash does not occur when stopping other streaming resources such as cars, weapons, or other clothing items, indicating a possible limit on the number or size of streaming files.
The issue seems to be related to an overall limit on streaming files, not specific to any particular type of file.
TechDemo asked me to create this GitHub issue to forward the problem to the correct team for further investigation.
It does not seem to be localized to me, I have spoken with 2 people with the same/similar issues.

Please let me know if any additional information is needed to assist in debugging this issue. Thank you!

Bossman02 commented 1 week ago

if I remember correctly there is a limit on the number of ytd you can have streamed, also the clothing packs have their own limits, which if not created with the correct tool are not respected and generate crashes

DemmyDemon commented 1 week ago

if I remember correctly there is a limit on the number of ytd you can have streamed, also the clothing packs have their own limits, which if not created with the correct tool are not respected and generate crashes

Sure, but it should generate an error message to that effect, not just crash outright? The issue is the crash, not the limitation.

Bossman02 commented 1 week ago

no, it make a crash in the case of clothing limits https://github.com/DurtyFree/durty-cloth-tool/wiki/YMT-game-limit-and-crash-issues

Bossman02 commented 1 week ago

also can be any of the streaming files is corrupted (normally some 3dmodel)

DemmyDemon commented 1 week ago

no, it make a crash

Yes, I know. That's what I'm saying. That is the problem. That is what the actual issue is. It crashes rather than inform anyone of any limitation, making it hard for server operators to figure out what the problem is.

DemmyDemon commented 1 week ago

also can be any of the streaming files is corrupted (normally some 3dmodel)

No, that is a different crash.

Bossman02 commented 1 week ago

no, it make a crash

Yes, I know. That's what I'm saying. That is the problem. That is what the actual issue is. It crashes rather than inform anyone of any limitation, making it hard for server operators to figure out what the problem is.

I don't think this can be solved, as I have understood it is a limitation of the gta engine itself and not of fivem (if you generate the clothing pack exceding limits in singleplayer mode and load it from the single player it also crashes)

DemmyDemon commented 1 week ago

no, it make a crash

Yes, I know. That's what I'm saying. That is the problem. That is what the actual issue is. It crashes rather than inform anyone of any limitation, making it hard for server operators to figure out what the problem is.

I don't think this can be solved, as I have understood it is a limitation of the gta engine itself and not of fivem (if you generate the clothing pack exceding limits in singleplayer mode and load it from the single player it also crashes)

Extending the clothing pool is possible, but that's not what this is about. This is about giving a proper error message when a limitation is met, rather than sending server owners on a wild chase through the everything looking for solutions.

"You are streaming too much shit. Cut it out."

jxnnyo commented 1 week ago

As I said in the OP after speaking to staff in the discord it was mentioned that this crash is not intentional and the streaming handler should be able to take any quantity of files. Anyone saying the issue is related to the clothing pack is incorrect due to the fact me stopping a few cars or a few MLOs lets me in game fine.

DemmyDemon commented 1 week ago

after speaking to staff in the discord

I am TechDemo in Discord. I am not "staff", I'm just a volunteer community moderator.

jxnnyo commented 1 week ago

Ah sorry there was a miscommunication, but I also believe that there shouldn’t be a limit.

NB-FiveM commented 1 week ago

This crash is related to the amount of streamed files (collectively across all resources, not in a single resource). It is the same as an issued I've opened in the past and closed simply because after doing some research it appeared nothing could really be done about it to mitigate it other than removing an excessive amount of streamed files (which resolved the issue).

jxnnyo commented 1 week ago

As I have already mentioned, I am aware that this is the issue however multiple people have stated that is not intentional and an issue with the method FiveM use to handle streaming files.

jxnnyo commented 1 week ago

Any update on this?

Legacy-TacticalGamingInteractive commented 6 days ago

Any update on this?

We just got this error as well on the latest artifacts when trying to add new stuff. (Weapons) It is not only related to clothing. We have not touched .ymt limit or clothing on this update. The log and client crash seems to end when loading handling.meta files. However, we didn't touch any cars or handling on this update. What typically loads after handling metas for the client? That might help narrow it down.

jxnnyo commented 6 days ago

The exact same issue im having, crashes straight after handling metas. Doesn't make a difference going back on artifacts tho as I already tried that, only thing I can think of is something todo with the client build. @Legacy-TacticalGamingInteractive

Legacy-TacticalGamingInteractive commented 6 days ago

The exact same issue im having, crashes straight after handling metas. Doesn't make a difference going back on artifacts tho as I already tried that, only thing I can think of is something todo with the client build. @Legacy-TacticalGamingInteractive

What is your amount of .ytd in your server right now?
What is your amount of .ydr in your server right now?
Total number of ensured resources? (Perhaps search for ensure and start in your server.cfg to see)

If there are indeed magical limits, which I have yet to see ANYONE document ANYWHERE other than just heresay. Then maybe this can help us solve that.

jxnnyo commented 6 days ago

My current YTD count including replacements and addons is at 20,150. My current YDR count including replacements and addons is at 12,923. Total number of resources is at 466 (a lot are scripts and unrelated to streaming assets).

I hope this can assist Cfx in investigating this issue.

Legacy-TacticalGamingInteractive commented 6 days ago

Heres mine as you can see:

[    100047] [b3095_GTAProce]             MainThrd/ Loading 23 handling entries from dlc_mp2023_02CRC:/common/data/handling.meta
[    100328] [b3095_GTAProce]             MainThrd/ Loading content XML: dlc_MP2023_02_G9ECCRC:/content.xml
[    100328] [b3095_GTAProce]             MainThrd/ Loading content XML: dlcSPUpgradeCRC:/content.xml
[    100375] [b3095_GTAProce]             MainThrd/ Loading 12 handling entries from dlcSPUpgrade:/common/data/handling.meta
[    104015] [b3095_GTAProce]             MainThrd/ rage::gameSkeleton::RunInitFunctions: Invoking CDLCScript INIT_SESSION init (17 out of 18)
[    104015] [b3095_GTAProce]             MainThrd/ rage::gameSkeleton::RunInitFunctions: Invoking audNorthAudioEngineDLC INIT_SESSION init (18 out of 18)
[    109640] [b3095_DumpServ]                22204/ Process crash captured. Crash dialog content:
[    109640] [b3095_DumpServ]                22204/ GTA5_b3095.exe!sub_14131068C (0xc8)
[    109640] [b3095_DumpServ]                22204/ An error at GTA5_b3095.exe!sub_14131068C (0xc8) caused FiveM to stop working. A crash report is being uploaded to the FiveM developers.
[    109640] [b3095_DumpServ]                22204/ 
[    109640] [b3095_DumpServ]                22204/ Legacy crash hash: princess-video-alanine
[    109640] [b3095_DumpServ]                22204/ Stack trace:
[    109640] [b3095_DumpServ]                22204/   GTA5_b3095.exe!sub_14131068C (0xc8)
[    109640] [b3095_DumpServ]                22204/   gta-streaming-five.dll!LoadStreamingFiles (0x14f6) (<A HREF="https://sourcegraph.com/github.com/citizenfx/fivem/-/blob/production/fivem/code/components/gta-streaming-five/src/LoadStreamingFile.cpp?L1951">LoadStreamingFile.cpp:1951</A>)
[    109640] [b3095_DumpServ]                22204/   gta-streaming-five.dll!<lambda>::()::__l2::<lambda>::operator() (0x2a) (<A HREF="https://sourcegraph.com/github.com/citizenfx/fivem/-/blob/production/fivem/code/components/gta-streaming-five/src/LoadStreamingFile.cpp?L3724">LoadStreamingFile.cpp:3724</A>)
[    109640] [b3095_DumpServ]                22204/   gta-streaming-five.dll!std::invoke (0x2c) (type_traits:1586)
[    109640] [b3095_DumpServ]                22204/   gta-streaming-five.dll!fwEvent<enum rage::InitFunctionType>::Connect::__l7::<lambda>::operator() (0x2c) (<A HREF="https://sourcegraph.com/github.com/citizenfx/fivem/-/blob/production/fivem/code/client/shared/EventCore.h?L299">EventCore.h:299</A>)
[    109640] [b3095_DumpServ]                22204/   gta-streaming-five.dll!std::invoke (0x2c)
[    109640] [b3095_DumpServ]                22204/   gta-streaming-five.dll!std::_Invoker_ret<bool>::_Call (0x2c) (functional:659)
[    109640] [b3095_DumpServ]                22204/ 
[    111125] [b3095_DumpServ]                22204/ Crash report service returned si-b9d9d200908142f1b166690a85bb5a53

From my DMP analysis

EXCEPTION_RECORD:  ffffffffffffffff -- (.exr 0xffffffffffffffff)
ExceptionAddress: 00007ff6727a0754 (FiveM_b3095_GTAProcess+0x0000000001310754)
   ExceptionCode: c0000005 (Access violation)
  ExceptionFlags: 00000000
NumberParameters: 2
   Parameter[0]: 0000000000000001
   Parameter[1]: 000000000001f2fb
Attempt to write to address 000000000001f2fb

APP:  fivem_b3095_gtaprocess.exe

FAULTING_THREAD:  0000000000004820

PRIMARY_PROBLEM_CLASS:  INVALID_POINTER_WRITE

BUGCHECK_STR:  APPLICATION_FAULT_INVALID_POINTER_WRITE

LAST_CONTROL_TRANSFER:  from 00007ff903939f07 to 00007ff6727a0754

STACK_TEXT:  
000000d8`0bf7dd30 00007ff9`03939f07 : 00007ff6`744464f0 00000000`00000000 00000212`ce930bc0 00000212`ce930df0 : FiveM_b3095_GTAProcess+0x1310754
000000d8`0bf7de60 00007ff9`0391bad1 : 00000211`84c65af0 00000000`00000008 00000000`00000003 00000000`00000003 : gta_streaming_five!streaming::IsStreamerShuttingDown+0x4787
000000d8`0bf7e310 00007ff9`4bc74420 : 000000d8`0bf7e410 00000000`00000008 00003c54`9c6168bd 00000000`6c236830 : gta_streaming_five!rage::fwScriptGuid::GetGuidFromBase+0x2ff1
000000d8`0bf7e340 00007ff6`714a64fb : 00000778`23e12fe7 00007ff6`714a6472 00007ff6`733da300 00000778`0fff1f84 : gta_core_five!rage::gameSkeleton::RunInitFunctions+0x4e0
000000d8`0bf7e560 00007ff6`714a8e99 : 00000000`00000000 00000000`00000002 00000000`00000000 00000000`6c236830 : FiveM_b3095_GTAProcess+0x164fb
000000d8`0bf7e5a0 00007ff6`714b7d75 : 00000000`0000000b 00000000`00000002 00000000`00000013 00000000`6c236830 : FiveM_b3095_GTAProcess+0x18e99
000000d8`0bf7e5d0 00007ff6`714b141b : 00000000`00000000 00000211`d8a38720 00000211`d0d90000 00000000`00000000 : FiveM_b3095_GTAProcess+0x27d75
000000d8`0bf7e600 00007ff6`72abeec0 : 00007ff6`733da3c0 00007ff9`6c236830 00007ff9`6c236830 00000000`00000001 : FiveM_b3095_GTAProcess+0x2141b
000000d8`0bf7e630 00007ff6`71491494 : 00000000`00000000 000000d8`0bf7e800 00000000`00000001 00007ff9`6c236830 : FiveM_b3095_GTAProcess+0x162eec0
000000d8`0bf7e660 00007ff9`4bc7345e : 00007ff6`00000001 00000211`84d30b20 00000000`00000000 00007ff6`72e7ac70 : FiveM_b3095_GTAProcess+0x1494
000000d8`0bf7e690 00007ff6`72cb1344 : 00000000`00000000 000000d8`0bf7e800 00007ff9`6c236830 00007ff9`6c236830 : gta_core_five!game::SetBindingTagActive+0x61de
000000d8`0bf7e6c0 00007ff6`775e110e : 00007ff6`00000000 00007ff6`72cb1200 00007ff9`6c236830 00007ff9`73b07020 : FiveM_b3095_GTAProcess+0x1821344
000000d8`0bf7e700 00007ff6`7760659f : 00007ff6`775b7c18 00007ff6`775b7c18 00000211`d0a80ad0 00000000`00000008 : FiveM_b3095_GTAProcess!GetErrorData+0x1f3e
000000d8`0bf7eaa0 00007ff6`7762398a : 00000000`0000000a 00000000`00000000 00000000`00000001 00007ff8`e3370d46 : FiveM_b3095_GTAProcess!AsyncTrace+0x1a87f
000000d8`0bf7fa40 00007ff9`8c917344 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : FiveM_b3095_GTAProcess!TryCollectCrashLog+0x1357a
000000d8`0bf7fa80 00007ff9`8d0bcc91 : 00007ff9`8d111a85 00000000`00000000 00000000`00000000 00000000`00000000 : kernel32!BaseThreadInitThunk+0x14
000000d8`0bf7fab0 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x21

STACK_COMMAND:  ~0s; .ecxr ; kb

SYMBOL_STACK_INDEX:  0

SYMBOL_NAME:  fivem_b3095_gtaprocess+1310754

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: FiveM_b3095_GTAProcess

IMAGE_NAME:  FiveM_b3095_GTAProcess.exe

DEBUG_FLR_IMAGE_TIMESTAMP:  667aeb9b

FAILURE_BUCKET_ID:  INVALID_POINTER_WRITE_c0000005_FiveM_b3095_GTAProcess.exe!Unknown

BUCKET_ID:  APPLICATION_FAULT_INVALID_POINTER_WRITE_fivem_b3095_gtaprocess+1310754

WATSON_STAGEONE_URL:  http://watson.microsoft.com/StageOne/FiveM_b3095_GTAProcess_exe/2_0_0_8691/667aeb9b/FiveM_b3095_GTAProcess_exe/2_0_0_8691/667aeb9b/c0000005/01310754.htm?Retriage=1

jxnnyo commented 6 days ago

Mine also crashes just after the following

[ 104015] [b3095_GTAProce] MainThrd/ rage::gameSkeleton::RunInitFunctions: Invoking audNorthAudioEngineDLC INIT_SESSION init (18 out of 18)

jxnnyo commented 6 days ago

Seems like its a common issue, I thought it was a select few people.

jxnnyo commented 6 days ago

The exact same issue im having, crashes straight after handling metas. Doesn't make a difference going back on artifacts tho as I already tried that, only thing I can think of is something todo with the client build. @Legacy-TacticalGamingInteractive

What is your amount of .ytd in your server right now?

What is your amount of .ydr in your server right now?

Total number of ensured resources? (Perhaps search for ensure and start in your server.cfg to see)

If there are indeed magical limits, which I have yet to see ANYONE document ANYWHERE other than just heresay. Then maybe this can help us solve that.

What's your answers to this question out of curiosity? See if we have any similarities.

Legacy-TacticalGamingInteractive commented 6 days ago

[   1653422] [b3095_GTAProce]             MainThrd/ Creating script environments for w_pocketlight
[   1653578] [b3095_GTAProce]             MainThrd/ failed to reg w_me_pocketlight_camo_02.ydr? -1
[   1656250] [b3095_DumpServ]                26804/ Process crash captured. Crash dialog content:
[   1656250] [b3095_DumpServ]                26804/ GTA5_b3095.exe!sub_14131068C (0x7e)
[   1656250] [b3095_DumpServ]                26804/ An error at GTA5_b3095.exe!sub_14131068C (0x7e) caused FiveM to stop working. A crash report is being uploaded to the FiveM developers.
[   1656250] [b3095_DumpServ]                26804/ 
[   1656250] [b3095_DumpServ]                26804/ Legacy crash hash: pip-jig-victor
[   1656250] [b3095_DumpServ]                26804/ Stack trace:
[   1656266] [b3095_DumpServ]                26804/   GTA5_b3095.exe!sub_14131068C (0x7e)
[   1656266] [b3095_DumpServ]                26804/   GTA5_b3095.exe!sub_141645EEC (0x91)
[   1656266] [b3095_DumpServ]                26804/   gta-streaming-five.dll!hook::details::cdecl_stub_<unsigned int *,unsigned int *,char const *,bool,char const *,bool>::operator() (0x1e) (<A HREF="https://sourcegraph.com/github.com/citizenfx/fivem/-/blob/beta/fivem/code/client/shared/Hooking.Invoke.h?L113">Hooking.Invoke.h:113</A>)
[   1656266] [b3095_DumpServ]                26804/   gta-streaming-five.dll!streaming::RegisterRawStreamingFile (0x7b) (<A HREF="https://sourcegraph.com/github.com/citizenfx/fivem/-/blob/beta/fivem/code/components/gta-streaming-five/src/Streaming.cpp?L233">Streaming.cpp:233</A>)
[   1656266] [b3095_DumpServ]                26804/   gta-streaming-five.dll!LoadStreamingFiles (0x1678) (<A HREF="https://sourcegraph.com/github.com/citizenfx/fivem/-/blob/beta/fivem/code/components/gta-streaming-five/src/LoadStreamingFile.cpp?L1976">LoadStreamingFile.cpp:1976</A>)
[   1656266] [b3095_DumpServ]                26804/   gta-streaming-five.dll!streaming::AddDataFileToLoadList (0x258) (<A HREF="https://sourcegraph.com/github.com/citizenfx/fivem/-/blob/beta/fivem/code/components/gta-streaming-five/src/LoadStreamingFile.cpp?L1682">LoadStreamingFile.cpp:1682</A>)
[   1656266] [b3095_DumpServ]                26804/   citizen-resources-gta.dll!<lambda>::operator() (0xe57) (<A HREF="https://sourcegraph.com/github.com/citizenfx/fivem/-/blob/beta/fivem/code/components/citizen-resources-gta/src/ResourcesTest.cpp?L224">ResourcesTest.cpp:224</A>)
[   1656266] [b3095_DumpServ]                26804/ 
[   1658703] [b3095_DumpServ]                26804/ Crash report service returned si-33999b6da72343e2b62ff1bb3e3f53ec

[     83875] [b3095_GTAProce]             MainThrd/ rage::gameSkeleton::RunInitFunctions: Invoking CDLCScript INIT_SESSION init (17 out of 18)
[     83890] [b3095_GTAProce]             MainThrd/ rage::gameSkeleton::RunInitFunctions: Invoking audNorthAudioEngineDLC INIT_SESSION init (18 out of 18)
[     86484] [b3095_GTAProce]             MainThrd/ failed to reg mlo_gravestone.ytyp? -1
[     88843] [b3095_DumpServ]                 9408/ Process crash captured. Crash dialog content:
[     88843] [b3095_DumpServ]                 9408/ GTA5_b3095.exe!sub_14131068C (0x7e)
[     88843] [b3095_DumpServ]                 9408/ An error at GTA5_b3095.exe!sub_14131068C (0x7e) caused FiveM to stop working. A crash report is being uploaded to the FiveM developers.
[     88843] [b3095_DumpServ]                 9408/ 
[     88843] [b3095_DumpServ]                 9408/ Legacy crash hash: pip-jig-victor
[     88843] [b3095_DumpServ]                 9408/ Stack trace:
[     88843] [b3095_DumpServ]                 9408/   GTA5_b3095.exe!sub_14131068C (0x7e)
[     88843] [b3095_DumpServ]                 9408/   GTA5_b3095.exe!sub_141645EEC (0x91)
[     88843] [b3095_DumpServ]                 9408/   gta-streaming-five.dll!hook::details::cdecl_stub_<unsigned int *,unsigned int *,char const *,bool,char const *,bool>::operator() (0x1e) (<A HREF="https://sourcegraph.com/github.com/citizenfx/fivem/-/blob/master/fivem/code/client/shared/Hooking.Invoke.h?L113">Hooking.Invoke.h:113</A>)
[     88859] [b3095_DumpServ]                 9408/   gta-streaming-five.dll!streaming::RegisterRawStreamingFile (0x7b) (<A HREF="https://sourcegraph.com/github.com/citizenfx/fivem/-/blob/master/fivem/code/components/gta-streaming-five/src/Streaming.cpp?L233">Streaming.cpp:233</A>)
[     88859] [b3095_DumpServ]                 9408/   gta-streaming-five.dll!LoadStreamingFiles (0x1678) (<A HREF="https://sourcegraph.com/github.com/citizenfx/fivem/-/blob/master/fivem/code/components/gta-streaming-five/src/LoadStreamingFile.cpp?L1976">LoadStreamingFile.cpp:1976</A>)
[     88859] [b3095_DumpServ]                 9408/   gta-streaming-five.dll!<lambda>::()::__l2::<lambda>::operator() (0x2a) (<A HREF="https://sourcegraph.com/github.com/citizenfx/fivem/-/blob/master/fivem/code/components/gta-streaming-five/src/LoadStreamingFile.cpp?L3724">LoadStreamingFile.cpp:3724</A>)
[     88859] [b3095_DumpServ]                 9408/   gta-streaming-five.dll!std::invoke (0x2c) (type_traits:1586)
[     88859] [b3095_DumpServ]                 9408/ 
[     90875] [b3095_DumpServ]                 9408/ Crash report service returned si-07669a1921144223b63e1cf387a307ed

Adding this an example as well we tested this with CANARY build.

We have been testing for the past 9 hours to get the exact limit.

This was a different crash message however its directly related. Different crash information as its a different client build but the same symptom.

At the moment we have been testing, and soon as you get close to the limit you start to see random crashes on the client side for some people only. Such as when they try to spawn a car or a new gun, or a prop of some kind.

PS. This is without any client mods loaded. Completely fresh installation of FiveM and GTA5.

    "crashometry_load_game_first_launch": "true",
    "crashometry_mod_package_count": "1",
    "crashometry_onesync_big": "true",
    "crashometry_onesync_enabled": "true"

    uint32_t* RegisterRawStreamingFile(uint32_t* fileId, const char* fileName, bool unkTrue, const char* registerAs, bool errorIfFailed)
    {
        rage::fiCollection* rawStreamer = nullptr;

        if (GetRawStreamerForFile(fileName, &rawStreamer))
        {
            auto fileIdx = rawStreamer->GetEntryByName(fileName);
            if (fileIdx != uint16_t(-1))
            {
                const char* registerName = fileName;

                if (registerAs)
                {
                    registerName = registerAs;
                }

                uint8_t unkVal;
                rage::strStreamingEngine::GetInfo()->RegisterObject(fileId, registerName, fileIdx | (1 << 16), 1, &unkVal, unkTrue, nullptr, false);
            }

            return fileId;
        }

        return g_registerRawStreamingFile(fileId, fileName, unkTrue, registerAs, errorIfFailed);
    }
}

jxnnyo commented 4 days ago

Any update?

prikolium-cfx commented 4 days ago

From what I found the limit for total assets count in pgRawStreamer is 65535.

Legacy-TacticalGamingInteractive commented 4 days ago

From what I found the limit for total assets count in pgRawStreamer is 65535.

Is there a way to check this number easily via console or some overlay, or must this be done by manually counting every asset in a servers resources folder or some such via streamed file types (would data/meta files also count towards this or just things in stream folders). At the moment it does not appear I have met this limit on quick glance of total ytd, ydr, ymt, ycd, ytyp, ybn, ydd etc. Not running client sided mods either which I assume add to that number as well if someone has mod packs loaded. Not sure about the others.

prikolium-cfx commented 4 days ago

To fix it and extend available amount of assets to load we need to completely change the way how custom assets loaded. It will take much time. For now I can try to prevent it from crashing and make it possible to check how many assets in total loaded right now.

NB-FiveM commented 4 days ago

A check to help server owners / developers understand that the limit has been reached I think is sufficient. While i do think 65535 is an excessive amount of files and likely could be reduced in most cases as I have done with my server... it does however seem to be much easier to reach over time especially with some of the larger map modders and custom clothing. It could also allow for some breathing room for additional map additions/mods

Legacy-TacticalGamingInteractive commented 4 days ago

To fix it and extend available amount of assets to load we need to completely change the way how custom assets loaded. It will take much time. For now I can try to prevent it from crashing and make it possible to check how many assets in total loaded right now.

That would be good for now and at least allow some way to work with the issue.

Legacy-TacticalGamingInteractive commented 4 days ago

A check to help server owners / developers understand that the limit has been reached I think is sufficient. While i do think 65535 is an excessive amount of files and likely could be reduced in most cases as I have done with my server... it does however seem to be much easier to reach over time especially with some of the larger map modders and custom clothing. It could also allow for some breathing room for additional map additions/mods

Unfortunately with the massive amount of encrypted resources also all being separate, and devs adding locked vanilla streamed replacement files to them on top of custom files - especially in the case of maps and eup - we cannot even condense resources like we used to by integrating some modified vanilla files to our own needs since we cannot edit anything like ydd ydr etc now too Making things even more difficult.

Theres so many things we still need to put into our server that we havent even been able to.

Given how many years FiveM has been around and longer running servers will eventually, like in our immediate case, need more than 65k files it will continue to be a problem more and more.

Is it specific file types only that count towards this limit?

jxnnyo commented 4 days ago

I agree we need more context to whatever is causing this, and I don't believe it should be intentional as said above many servers who have ran for a while is constantly adding new things to the servers to keep players attracted and its not as easy as 'removing' things. A lot of them are for player businesses etc.

AvarianKnight commented 4 days ago

To fix it and extend available amount of assets to load we need to completely change the way how custom assets loaded. It will take much time. For now I can try to prevent it from crashing and make it possible to check how many assets in total loaded right now.

Would it not be better to just throw a proper error?

I'm assuming this could also lead to player assets not loading too, which would lead to invisible players and such, along side other broken content if assets don't stream properly (invisible cars, parts of the players bodys missing, invisible interiors, etc).

This would just lead to people instead making reports saying "assets don't load properly plz fix" which would be counter-productive.

tens0rfl0w commented 4 days ago

I had something like this on my mind: https://github.com/citizenfx/fivem/blob/4755fafe655c02d0f63b570bb99d005b09ad963e/code/components/gta-streaming-five/src/PatchStreamingPreparation.cpp#L289

Preventing assets from loading when over the limit kinda feels counter intuitive.

Legacy-TacticalGamingInteractive commented 4 days ago

It certainly hurts. Id rather have invisible assets with notification and a better error message with a total streamed asset count included perhaps as well as passive way to also check the total streamed assets at will with a command or on client startup, using the same logic that would display the number in the error message.

That would suffice, for now. At minimum. But ultimately more of a long term fix will be needed as more servers will indeed start having this issue now as we see already with a few new issues popping up.

AvarianKnight commented 4 days ago

I don't think that it would be a proper solution, even if temporary.

Who knows what can (and probably will break) when just ignoring any new asset streaming.

If players want to opt into this I think it would be fine, but having it as the default just seems like a bad idea.

Legacy-TacticalGamingInteractive commented 4 days ago

Yeah either way. Start first with an error and a count and we can go from there to start off.

prikolium-cfx commented 3 days ago

Can you please try to reproduce it on feature branch for me? To change your client to feature branch you need to open CitizenFx.ini and change UpdateChannel to specific/feature/pgrawstreamer-full-error. Tell me if it works or no

Legacy-TacticalGamingInteractive commented 2 days ago

Do you know which file types count towards the limit? Is it just .ydr and .yft?

prikolium-cfx commented 2 days ago

Do you know which file types count towards the limit? Is it just .ydr and .yft?

All the assets: ydr, yft, ytd, ybn, ydd...

Legacy-TacticalGamingInteractive commented 2 days ago

Ah true. The reason I asked that one was just because I was wondering if things like .html .js and big resources like ox library related things and such using yarn and all that if those types of files count towards it as well or if its just hard GTAV assets like the ones you mentioned and for figuring out which to remove a little easier.

[    131468] [b3095_GTAProce]             MainThrd/ Loading content XML: dlc_MP2023_02_G9ECCRC:/content.xml
[    131484] [b3095_GTAProce]             MainThrd/ Loading content XML: dlcSPUpgradeCRC:/content.xml
[    131515] [b3095_GTAProce]             MainThrd/ Loading 12 handling entries from dlcSPUpgrade:/common/data/handling.meta
[    134656] [b3095_GTAProce]             MainThrd/ rage::gameSkeleton::RunInitFunctions: Invoking CDLCScript INIT_SESSION init (17 out of 18)
[    134656] [b3095_GTAProce]             MainThrd/ rage::gameSkeleton::RunInitFunctions: Invoking audNorthAudioEngineDLC INIT_SESSION init (18 out of 18)
[    137156] [b3095_GTAProce]             MainThrd/ Error: ERR_STR_FAILURE: trying to add more assets to pgRawStreamer when it's already full (65535).
[    141218] [b3095_DumpServ]                20492/ Process crash captured. Crash dialog content:
[    141218] [b3095_DumpServ]                20492/ ERR_STR_FAILURE: trying to add more assets to pgRawStreamer when it's already full (65535).
[    141218] [b3095_DumpServ]                20492/  
[    142843] [b3095_DumpServ]                20492/ Crash report service returned si-6daf282316d848f99fe548f01eaeddf5

✔ Can confirm that this is now working as intended. Nice! I suppose then that leaves a final question at least from my end at least.

Is there a method to see the total number of assets current loaded on client or from server? This can have some effects for these use cases:

1) So server owners will know how much we will have to remove rather than guessing. Such as when we have vehicle packs or clothing or things we want to add to the server that we need to plan ahead for.

2) Helping players who have client sided mods, they will undoubtedly also need to know so they can adjust their own personal mods to be less to be able to join some servers close to the limit already.

3) Can allow for developers of resources to be a little more mindful when creating things, and be aware that there actually is a limit they need to be careful of. And perhaps not fill their resource stream folders with tons of not really needed files to stream.

prikolium-cfx commented 2 days ago

Ah true. The reason I asked that one was just because I was wondering if things like .html .js and big resources like ox library related things and such using yarn and all that if those types of files count towards it as well or if its just hard GTAV assets like the ones you mentioned and for figuring out which to remove a little easier.
[    131468] [b3095_GTAProce]             MainThrd/ Loading content XML: dlc_MP2023_02_G9ECCRC:/content.xml
[    131484] [b3095_GTAProce]             MainThrd/ Loading content XML: dlcSPUpgradeCRC:/content.xml
[    131515] [b3095_GTAProce]             MainThrd/ Loading 12 handling entries from dlcSPUpgrade:/common/data/handling.meta
[    134656] [b3095_GTAProce]             MainThrd/ rage::gameSkeleton::RunInitFunctions: Invoking CDLCScript INIT_SESSION init (17 out of 18)
[    134656] [b3095_GTAProce]             MainThrd/ rage::gameSkeleton::RunInitFunctions: Invoking audNorthAudioEngineDLC INIT_SESSION init (18 out of 18)
[    137156] [b3095_GTAProce]             MainThrd/ Error: ERR_STR_FAILURE: trying to add more assets to pgRawStreamer when it's already full (65535).
[    141218] [b3095_DumpServ]                20492/ Process crash captured. Crash dialog content:
[    141218] [b3095_DumpServ]                20492/ ERR_STR_FAILURE: trying to add more assets to pgRawStreamer when it's already full (65535).
[    141218] [b3095_DumpServ]                20492/  
[    142843] [b3095_DumpServ]                20492/ Crash report service returned si-6daf282316d848f99fe548f01eaeddf5
✔ Can confirm that this is now working as intended. Nice! I suppose then that leaves a final question at least from my end at least.

Is there a method to see the total number of assets current loaded on client or from server? This can have some effects for these use cases:

So server owners will know how much we will have to remove rather than guessing. Such as when we have vehicle packs or clothing or things we want to add to the server that we need to plan ahead for.

Helping players who have client sided mods, they will undoubtedly also need to know so they can adjust their own personal mods to be less to be able to join some servers close to the limit already.

Can allow for developers of resources to be a little more mindful when creating things, and be aware that there actually is a limit they need to be careful of. And perhaps not fill their resource stream folders with tons of not really needed files to stream.

We've added command assetscount to F8 console and also pgRawStreamer assets monitor. Will be available in feature branch in ~1 hour.

Test it please, provide feedback, and we will merge it to canary after.

jxnnyo commented 2 days ago

Are there any plans in the coming future to refactor pgRawStreamer to be able to handle more assets than 65535?

Legacy-TacticalGamingInteractive commented 1 day ago

Are there any plans in the coming future to refactor pgRawStreamer to be able to handle more assets than 65535?

I hope so. This would allow us to continue buying more assets on Tebex as well. At this point we cannot really go anywhere else or buy anything new since its maxed for us.

prikolium-cfx commented 1 day ago

Yes, but not in next 3 months.

Are there any plans in the coming future to refactor pgRawStreamer to be able to handle more assets than 65535?

Legacy-TacticalGamingInteractive commented 1 day ago

Yes, but not in next 3 months.

Are there any plans in the coming future to refactor pgRawStreamer to be able to handle more assets than 65535?

Do you want us to reopen/revisit this issue then. Or make a new one to remind the team.

prikolium-cfx commented 1 day ago

Yes, but not in next 3 months.

Are there any plans in the coming future to refactor pgRawStreamer to be able to handle more assets than 65535?

Do you want us to reopen/revisit this issue then. Or make a new one to remind the team.

I've added this to our backlog and we will take care of that when we will be ready

citizenfx / fivem