Copy field back to host only on request

[kynan]

Currently, fields written back to state are copied back to the host at the end of every time step. A call to state->returnFieldToHost is generated at the end of the generated run_model function. Instead, this should be made user-configurable i.e. via a custom UFL command (e.g. dump) that would request returning the field to the host.

[gmarkall]

Isn't it up to the dump_period_in_timesteps option in the flml (or something with a similar name) to choose when to write to disk, and by extension, return fields to the host?

I think there's a function in fluidity called do_write_state() that tells you if you're supposed to write the state - I think we could just make the call to return the fields to the host when this returns true.

[kynan]

Yes, this makes much more sense. It felt awkward anyway suggesting it should be a UFL command.

[dham]

More generally, a field may be required at some future date inside Fluidity but Fluidity needs to ask for it.

It definitely should not be a user command.

I think the short term answer is to trigger copy back for all fields inserted into state each time it happens. I realise this is stupid from a performance perspective but I think correctness by the easiest possible route is the right first step. Copying back only on demand is something we can do next.

Currently every field is needed every timestep in order to write the stat file but it would be quite easy to code the stat file diagnostics for fields currently on the device to happen on the device.

[gmarkall]

This now works in the CUDA backend, to the extent that things aren't transferred back at the end of a time step, but are only transferred when output_state is called.