Network message v2

[Robadob]

Copied from https://github.com/FLAMEGPU/FLAMEGPU2/pull/883, working in a new branch to avoid breaking FRE project's dependency during development.

Remaining tasks:

• [x] Update current ID code to not enforce contiguous IDs
• [x] Validate IDs
• [x] Fix traversal
  • [x] Validate early, translate at the end
  • [x] Update pbm/ipbm to construct keys/vals with idmap
  • [x] fix tests
• [x] Add device API edge src-dest to index method
• [x] Duplicate host API to offer both index and ID (This will necessitate a host ID map)
• [x] (more) tests
• [x] python/rtc API tests (particularly around no edges)
• [x] native python/codegen tests (api has change a touch since they were setup)
• [x] Improve HostAPI auto sync
• [x] docs (https://github.com/FLAMEGPU/FLAMEGPU2-docs/pull/167)

[Robadob]

Todo

Agree whether vertex IDs can be "sparse" (@ptheywood in favour because everyone else does it, me not in favour because it adds an additional indirection (if not more) when accessing the graph via ID)

[Robadob]

Sparse IDs, limited by memory. Users encouraged to keep them contiguous to avoid memory issues.

[Robadob]

Hard stuff is all done (unless i find more edge cases along the way).

I've agreed with @ptheywood that we should offer both index and ID access API, and just document index as being preferable for performance.

Remaining tasks:

• device/host id/index api duplication+ tests
• rtc device api tests
• docs
• update fujitsu

Should be achievable before the end of next week.

[Robadob]

Current discussion is for HostAPI to emulate C++ std::map. Albeit with a fixed size

HostStaticVertexMap HostEnvironmentDirectedGraph::vertices()

User first must resize it with existing HostEnvironmentDirectedGraph::setVertexCount().

They can then insert and update elements, using operator[]. Similar to a regular std::map, or use iteration mechanics.

insert(), erase(), emplace() would not be supported to avoid the need to define a host vertex structure (which would essentially be a wrapper for std::map<std::string, std::any> 🤮).

[Robadob]

HostAPI refactor should be complete, with all tests working on C++/Py.

Going to make a start on the documentation.

[Robadob]

Final changes to do next week are adding python tests around device API edge cases and checking whether there needs to be some IO documentation added.

[ptheywood]

seatbelts=off builds fail without a rebase, due to the strings bug fixed elsewhere.

seatbelts=on builds pass all tests (c++ and python) under linux as it stands.

I think the conflict wants to be resolved as follows (and changed to avoid the py3.8? bug that has been fixed in master since) but I am not 100% on this, so not force pushing.

        if t.targets[0].id not in self._locals :
            # Special case, catch message.at() where a message is returned outside a message loop
            if hasattr(t.value, "func") and isinstance(t.value.func, ast.Attribute) and t.value.func.attr == 'at' :
                if t.value.func.value.id == self._input_message_var :
                    self._standalone_message_var.append(t.targets[0].id)
            # Special case, track which variables hold directed graph handles
            elif hasattr(t.value, "func") and isinstance(t.value.func, ast.Attribute) and t.value.func.attr == 'getDirectedGraph' :
                if t.value.func.value.value.id == "pyflamegpu" and t.value.func.value.attr == "environment" :
                    self._directed_graph_vars.append(t.targets[0].id)
            # Special case, definitions outside of agent fn are made const
            if self._indent == 0:
                self.write("constexpr ")

Edit: That results in a test failure, also needs the test changing to account for the addition of constexpr (I think just to the expected result, but not 100%)

[Robadob]

Windows Release SEATBELTS=OFF C & Python tests now pass.

[ptheywood]

Linux release seatbelts=ON C tests all pass post-rebase.