ValveSoftware / Fossilize

A serialization format for various persistent Vulkan object types.
MIT License
556 stars 46 forks source link

Can I run fossilize manually? #151

Open letharion opened 3 years ago

letharion commented 3 years ago

Fossilize can take a bit of time when I want to start a game. Is there a way I can stick a call to fossilize into a cron job and have it run in the background regularly? I tried the naive approach of copying the call from ps, resulting in the following:

~/.local/share/Steam/ubuntu12_64/fossilize_replay \
  ~/.local/share/Steam/steamapps/shadercache/570/fozpipelinesv5/steamapp_pipeline_cache.foz \
  ~/.local/share/Steam/steamapps/shadercache/570/fozpipelinesv5/steam_pipeline_cache.foz \
  --on-disk-validation-whitelist ~/.local/share/Steam/steamapps/shadercache/570/fozpipelinesv5/steam_pipeline_cache_whitelist \
  --replayer-cache ~/.local/share/Steam/steamapps/shadercache/570/fozpipelinesv5/replay_cache \
  --master-process --quiet-slave --shmem-fd 132 --control-fd 145 --spirv-val --num-threads 3 \
  --device-index 0 --timeout-seconds 10 --implicit-whitelist 0
Fossilize INFO: Attempting to map shmem block.
Fossilize INFO: Using control FD as sentinel.
Fossilize ERROR: Failed to add control_fd to epoll.

Is the file descriptor created outside of fossilize? If so can I manually do it?

kakra commented 3 years ago

You probably want to skip all the fd related options, those are created by the Steam client to communicate with the processes. Also, maybe skip quiet-slave while testing and look for obvious problems.

letharion commented 3 years ago

Oh I see, that explains the missing file descriptor. There were plenty of options passed, so I didn't notice them there. I should of course have realized that's how the missing FD was passed in.

Dropping --shmem-fd --control-fd --quite-slave fossilize does indeed run. It yields loads of

Fossilize WARN: Did not replay blob (tag: 7, hash: 0x31a65b0417b7c55f). See previous logs for context. Fossilize WARN: Referenced Pipeline layout 1443983c1d9866f5, but it does not exist. This can be expected when replaying an archive from Steam. If replaying just the application cache, make sure to replay together with the common cache, as application cache can depend on common cache.

But maybe that's not an issue, as it says This can be expected. I'll try this out for a while, gonna need some updates to the game to see if it works well.

kakra commented 3 years ago

@letharion Be aware that IO/CPU pressure auto-adjust may not work when running directly from command line or cron... I think --master-process did the trick for enabling background mode but I'm not sure. If auto-adjust works, you may want to bump up the thread count.

ghost commented 2 years ago

That's for replaying fossils, but how do you record fossils manually though?