search

charmplusplus / charm

The Charm++ parallel programming system. Visit https://charmplusplus.org/ for more information.
Apache License 2.0
206 stars 50 forks source link

objid_t: Basic definitions and use in envelope #170

Closed heyram closed 8 years ago

heyram commented 11 years ago

Original issue: https://charm.cs.illinois.edu/redmine/issues/170

heyram commented 5 years ago

Original date: 2013-04-12 16:30:00

Synchronize enums in charm.h with those defined in charmdebug. (MsgInfo.java)

heyram commented 5 years ago

Original date: 2013-04-12 16:42:31

Audit and fix all locations where ForIDedObjMsg needs to have a presence. git grep ForArrayEltMsg should bring these locations to light.

heyram commented 5 years ago

Original date: 2013-04-12 17:19:15

envelope setArrayMgr() only caters to the ForArrayEltMsg case. It is mostly used in scenarios that do not currently affect the p2p msging case. There are a few uses on the msg send path, but these should be dealt with in the send path code.

PhilMiller commented 5 years ago

Original date: 2013-04-13 20:11:46

I just realized a quirk in how we've been thinking of expressing and tracking indices for bound arrays (those sharing a common location manager). Corresponding elements of those arrays should have corresponding element IDs, and their overall object IDs should differ only in the collection ID. That implies that any size limits or explicit compression function specified for an array should actually apply to all arrays bound to it as well. That suggests two possible designs:

heyram commented 5 years ago

Original date: 2013-04-30 17:06:05

-debug-charm.C: CpdPupMessage() needs to know how to pup ForIDedObjMsg-

Opened as #193

heyram commented 5 years ago

Original date: 2013-04-30 17:34:21

-xi-symbol.C: generated tracing code uses ForArrayEltMsg, which is fine for now (so it won't get confused), but should eventually distinguish the cases.-

Noted on #176

heyram commented 5 years ago

Original date: 2013-04-30 17:37:51

ck-com/MsgPacker.h contains stuff that assumes messages sent to array elements are aimed by CkArrayIndex. Since this is essentially dead code, we'll leave it alone.

PhilMiller commented 5 years ago

Original date: 2013-04-30 18:31:24

In the array bcast ctor case, the size is already stored in numInitial.

PhilMiller commented 5 years ago

Original date: 2013-05-01 22:39:24

Mapped the bcast constructor case to the solution to #171, since we'll need to follow that path for bulk-constructed arrays to which something larger may be bound.

PhilMiller commented 5 years ago

Original date: 2013-08-23 00:45:49

I now have messages passing through the ID code path, with the envelope being converted before sending, and index extracted from ID on receipt. It's rather ugly, so I won't check it in, but the necessary bits are functional.

PhilMiller commented 5 years ago

Original date: 2013-08-23 01:02:59

-Eh, it fails on megatest. Still something broken.-

PhilMiller commented 5 years ago

Original date: 2013-08-27 17:28:49

Don't try to recompress the destination index of a message that's already compressed. Almost all the way through megatest now - failing on synctest.

PhilMiller commented 5 years ago

Original date: 2013-08-27 23:24:25

Compression of array IDs constructed on non-zero PEs is a bit tricky, because they're intentionally generated in the negative range. Will need to compact their 2's-complement representation down to the available bits, if possible.

For now, simply disable compression for such arrays.

PhilMiller commented 5 years ago

Original date: 2013-08-27 23:25:16

And with that, make test in tests/charm++ and examples/charm++ runs on net-linux-x86_64.

PhilMiller commented 5 years ago

Original date: 2013-10-07 19:05:50

This won't be ready for 6.6, so push it back.

PhilMiller commented 5 years ago

Original date: 2013-12-24 22:05:39

With some additional minor fixes, make test runs in full on net-linux-x86_64, as well as benchmarks/leanmd.

PhilMiller commented 5 years ago

Original date: 2013-12-24 22:20:58

Of course, once I actually make leanMD use the new code path, it crashes. More to fix, I guess.

The crash is in deliverUnknown. If I comment out the code that places patches on non-default PEs initially, it runs successfully.

PhilMiller commented 5 years ago

Original date: 2013-12-28 22:46:08

The fixes to get leanMD running, across multiple PEs, with explicit initial placement and load balancing, were all pretty simple. They mostly boiled down to consistently using the decompression function as defined.

PhilMiller commented 5 years ago

Original date: 2013-12-28 23:22:34

I just looked at Projections traces of the runs with and without array size bounds set, which during debugging very clearly enabled/disabled the object ID delivery path. Despite that, I'm not seeing any difference in reported message sizes. The trace generation calls env->getTotalsize(), which should reflect the difference in extra envelope data size. So, something's up, and I don't know what.

I've also confirmed by direct assertion that at the point of delivery to each compute object, the message is ForIDedObjMsg only when bounds are set. So, the new form is making it all the way through the stack, but its reported size is no different.

PhilMiller commented 5 years ago

Original date: 2013-12-30 23:08:42

Moved the code to pack array-element messages down to use 64-bit IDs earlier, so that the difference is accurately reflected in LB communication data and Projections traces. There's now an 8 byte per message difference between ForArrayEltMsg and ForIDedObjMsg. If we enabled packing for the variable-envelope structures, this should increase, along with an across-the-board improvement (see #387 for work on this - the difference with that is 22 bytes vs 10).

PhilMiller commented 5 years ago

Original date: 2013-12-30 23:37:57

All seems to be working, so this should be ready to merge once the mainline is open again.

PhilMiller commented 5 years ago

Original date: 2014-03-20 22:02:54

Core implementation plans

Need to implement along three paths:

As a very rigid test of proper function, we can initially add the ID alongside existing array index fields in both chare instances and messages. We can then assert that in all cases, the ID and index in the message match that of the object to which the message is being delivered. Once we've tested thoroughly enough in that mode, even with forwarding and such based on the ID rather than the index, we can drop the index field in the messages.

Construction has two cases:

Given the above test methodology, it should be possible to forward-port and adapt the already-written but rotting code for the compression case, and more strongly confirm that it works. Non-compressible cases should just hang on to the sentinel value (in both construction and transmission) until distributed construction and lookup is implemented. Along the delivery path, I'd need to at least transiently carry both the existing code for the distributed case using sentinels and the short-circuit code for the new case using IDs, with a temporary check against the sentinel to decide which path to follow.

The exact construction behavior for the compressible case is a bit of an open question. The generated IDs will result in a different notion of home PE than the array's map function applied to the object index. Since compressible arrays guarantee that every processor can compute the index <-> ID mapping bidirectionally, we could conditionalize relevant code paths accordingly. It looks like this would only have to be done in CkLocMgr::homePe({arrayIndex,ck::objid}), as everything calls that. That would suggest there's no reason to specially contact the home PE regarding the ID, since it will never be relevant in that setting.

The above test will also work for building up the distributed case, since a given message either has the field filled in or not. Once an object has the field filled, messages to it will also need to be filled. When we think implementation has 100% path coverage, we can start asserting that the message field is always filled, and similarly during object construction. Once that passes, then we can relegate the delivery path through the array manager / location manager stack to only messages whose recipient IDs don't match local objects.