only captured json files, where is the captured images?

[yulong314]

I somehow let the gta v loaded gta5.hk. But, after F12 pressed in game, I only see a lot of *_state.json file in cap directory. How can i capture images?

[philkr]

The capture plugin (https://github.com/philkr/gamehook/blob/master/plugins/capture/capture.cpp) will not trigger any image captioning by itself. The easiest way to capture images is through the webserver. Call localhost:8766/settings?fps=1, which should trigger the recording. You can also use python to trigger it manually, see pyhookv (https://github.com/philkr/pyhookv/blob/master/examples/gtapilot.py) and the python plugin (https://github.com/philkr/gamehook/tree/master/plugins/python) for examples.

[bradyz]

To check if your capture plugin is working (make sure you have capture.hk and server.hk in the GTA folder) -

When the game fully loads, call http://localhost:8766/settings?fps=1&targets=flow&targets=final&targets=albedo&targets=disparity&targets=object_id Then inspect http://localhost:8766/bmp?t=flow http://localhost:8766/bmp?t=final ... to check if they're being captured correctly.

[yulong314]

@bradyz thanks, by your settings I can see albedo, disparity, object_id and flow images in browser. But I cann't see "final". I don't know why?

[bradyz]

Okay that's what I thought - I had to make some small tweaks to get the "final" (this is just raw image frames) working.

Replace gta.cpp, line 81 with

std::unordered_set<ShaderHash> final_shader = { ShaderHash("6eef7895:a8818cdd:d19840f5:68c224f4") };

This fix is not the best (the logger spews out warnings), but it gets the job done for now and you should be able to retrieve the images (view in browser, and saves to disk). I'm currently looking for a better fix.

Note: if this doesn't work, you'll have to use RenderDoc to find the correct shader hash. The way I did this was to

select the final color pass (Colour Pass #15 (1 Targets + Depth)) click on the pipeline state tab select the PS (pixel shader) block and click view the shader hash should be at the very top

[yulong314]

@bradyz, it works, my shader hash is: final_shader = { ShaderHash("e5807043:550206a0:a0b79a2f:a142d658") }

[philkr]

It looks like v1.0.1365 actually changed some of the post processing shaders out. 12c5bc288e10b5d4a2a0bc0ef01b0be47bdee448 contains a new logic to find the output before the final distortion shader. Closing this for now, please reopen of not fixed.

[benjamintanweihao]

@yulong314 Do you have an example of the flow image? All I get is a black image. Which GTA version are you using?

[annopackage]

@bradyz, it works, my shader hash is: final_shader = { ShaderHash("e5807043:550206a0:a0b79a2f:a142d658") }

@yulong314 @bradyz @benjamintanweihao Hi, do you know how to capture the raw data and image automatically rather than triggering through the webserver?

[benjamintanweihao]

@annopackage I haven't touched this in a few years but I managed to dig up some notes I had:

GTA

The setup for GTA is pretty involved. These are roughly the setup steps involved:

Download GTA V
Compile the Gamehook and Gamehook GTA Plugin in Visual Studio (This results in the dxgi.dll, server.hk, capture.hk, and gta.hk files)
Copy the compiled assets into the GTA V folder
Copy the ScriptHook V files into the GTA V folder

    After this one time setup, the capture process is roughly:

When GTA V starts, the plugins should be loaded, along with the server and capture plugins
Enter http://localhost:8766/settings?fps=1&targets=flow&targets=final&targets=albedo&targets=disparity&targets=object_id into a browser window to trigger capturing. Each target corresponds to a type of capture. 

Object IDs are for instance segmentation. Class segments can be recovered from the object id frames via some bit manipulation. In particular, the last 4 bits encode the class labels.
Disparity captures the depth information.
Final is the original image.
Flow captures the optical flow.

Frames are stored in the cap directory in the GTA V folder.