[Question:] GC event notification for Memory-Compaction/Compression inside an array!

[CodingMadness]

I could not find any valuable information online and i asked on so many places but noone seems to know that particular stuff:

As is the title, is there any way to get Information from the (perhaps Background GC) IF and WHEN, and WHEN it is done a memory-compression/compaction of array elements happen?

consider this example:

I hope i could make clear in what I would like to do, again I could not find sources either in msdn nor by talking to ppl about this.

Thanks for any info regarding this.

Cheers!

[dotnet-issue-labeler[bot]]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[CodingMadness]

BTW: This question is more important on array of Class, or array of Struct

Cause i wanna do stuff based on the offset of each of the elements like: "myArray[0].FullName[0]" and for them i need to always have the right offset between: "myArray[0].FullName[0] <--------> myArray[1].FullName[0]"

[ghost]

Tagging subscribers to this area: @dotnet/gc See info in area-owners.md if you want to be subscribed.

Issue Details

---

I could not find any valuable information online and i asked on so many places but noone seems to know that particular stuff: As is the title, is there any way to get Information from the (perhaps Background GC) **IF and WHEN, and WHEN it is done** a memory-compression/compaction of array elements happen? consider this example:  I hope i could make clear in what I would like to do, again I could not find sources either in msdn nor by talking to ppl about this. Thanks for any info regarding this. Cheers!

Author:	Shpendicus
Assignees:	-
Labels:	`area-GC-coreclr`, `untriaged`
Milestone:	-

[CodingMadness]

Note:

the indeed more relevant info is if the GC ever compacts the room between array of classes/structs like:

class Test {
  public int A;
  public double B;
  public string C;
}

Test[] tests = new[20]; 

ref char first = ref Unsafe.AsRef(tests[0].C[0]);
ref char next = ref Unsafe.AsRef(tests[1].C[0]);

nint diff = Unsafe.ByteOffset(ref first, ref next); //can this value be different when GC compacts or does he eveb compact such?

[jkotas]

the indeed more relevant info is if the GC ever compacts the room between array of classes/structs like:

Yes, the GC can re-arrange classes like these arbitrarily. It can move them around arbitrary and the room between them can expand or shrink arbitrarily.

If you would like to prevent GC from moving the instances around, you would need to pin each of the class instance using Pinned GCHandle. A lot of long-term pinning comes with significant performance penalty.

[cshung]

Have you considered using an array of struct?

[CodingMadness]

@jkotas Hi, first thx for your answer !

If that is the case, is there an event I can wait for or get notified by the GC when exactly the memory-compaction happens and when he is done compacting? so I can re-calculate offsets not mess up my algorithm?

The moving around of the array/classes is not a problem cause i am working with Span<CLASS/STRUCT> and that span is created via a "ref T" and a computed valid Length, so the GC should update that automatically, however, when i compute offsets this happens in my constructor and the "nint" offsets are valuetypes so the difference is not being preserved i think, so when he compacts the size inside the array the "nint offset" would change!

So my last question: Would pinning the entire array also avoid the memory-compacttion of the said array? and also, is there a GC event which notifies you exactly on , when he moves and shrinks/expands the array in size?

Thx!

[CodingMadness]

Have you considered using an array of struct?

i am writing a ref-struct module which should work for all types actually, atleast i hoped for that, so i would either

a) await the compact-GC event if that is possible or b) Pin the array for the duration of the method he is using, not sure about the data regaridng the runtime of any method my type would then be used.

[CodingMadness]

Also I did read this article from on of your developers: https://tooslowexception.com/pinned-object-heap-in-net-5/ which seemed interesting to me, since i do not have control of the creation of the array, is there an option of let the GC move the entire array maybe for the duration of the pinning to the "POH"? (Pinned Object Heap)? or would this be overkill?

[jkotas]

Would pinning the entire array also avoid the memory-compacttion of the said array?

No. Pinning is not transitive. Pinning an array does not automatically pin object instances that are stored in the array.

await the compact-GC event if that is possible or

There is no way to await or otherwise synchronize with when the GC does heap compaction.

is there an option of let the GC move the entire array maybe for the duration of the pinning to the "POH"?

There is no such option.

[CodingMadness]

Would pinning the entire array also avoid the memory-compacttion of the said array?

No. Pinning is not transitive. Pinning an array does not automatically pin object instances that are stored in the array.

await the compact-GC event if that is possible or

There is no way to await or otherwise synchronize with when the GC does heap compaction.

is there an option of let the GC move the entire array maybe for the duration of the pinning to the "POH"?

There is no such option.

Hmm, okay thx for the clarification. I think i will then just fix the all string references of the array for the entirety of the lifetime of my ref struct, i dont see any other chance.

[jkotas]

What are you trying to achieve with this whole scheme? If you are trying to do this for performance, pinning all string references in the array is likely to be much more expensive that the savings you get.

Note that pinning has both direct and indirect costs: the direct costs are easy to measure by microbenchmarks, the indirect costs are much harder to measure - they only show up in longer running apps where the pinning interacts poorly with the rest of the app activity.

[CodingMadness]

What are you trying to achieve with this whole scheme? If you are trying to do this for performance, pinning all string references in the array is likely to be much more expensive that the savings you get.

Note that pinning has both direct and indirect costs: the direct costs are easy to measure by microbenchmarks, the indirect costs are much harder to measure - they only show up in longer running apps where the pinning interacts poorly with the rest of the app activity.

Well, my intention was actually to create a custom view on Span or string[] of classes, to avoid usingf LINQ on hotpaths without any allocation but generating the same results, i am successful right now with it, but the GC fragmentation can ruin all of this intent sadly :/

[CodingMadness]

for instacne you can use my type like this:

var sequence = new Sequence(someArray)

sequence.GetNames() //gives you: "hello dear world how are you" 

"for instance:

 Person[] persons = new 
{
      Name: "Mike", Age: 19,  Birth: 1995,
      Name: "John", Age: 35,  Birth: 1987
      Name: "Tom", Age: 92,  Birth: 1930
      Name: "Homer", Age: 57,  Birth: 1968
}

var names = Sequence(persons);

names.GetSequence(); //gives you a ROS of "Mike, John, Tom, Homer"

//i am currently accomplishing this by destroying the internal layout of the entire array, BUT when the user has what he wanted , for isntance to pass that thingg onto another function, he can pass it  and then right after he needs his array healthy back i do restore its layout and i tested it mutliple times, it works. 

ofc this works only right now for strings only also only the first i can get, but its still on testing and stuff, thats why i wanted to know if the GC somehow lets me re-compute the offsets because in order for me to get those i need to do some Unsafe.ByteOffsets between the strings and so on.

But the GC stuff is kinda making this not so cool.