rustc consumes > 50GB memory, completes eventually but def. stuck on something

[GlenDC]

First of all, I am convinced it is more the fault of me doing something wrong. My hope and guess is that this is not a bug in rust, but just me misusing the language. I can however not find why it is happening, but I imagine it is something to do with an accidantal recursive trait bound or other generic related issue...

• Problem is specifically with: https://github.com/plabayo/rama/blob/09c8bc0fea80560763c12149aceae3d9ddae87f5/examples/http_high_level_client.rs
• And it seemed to have been introduced the "problem" in https://github.com/plabayo/rama/pull/297/commits/aa4ea25cd2e4d624e8f82d1c48d375f285e3266b (that's when CI stage timings increased from 12m to 85m...)

Of course it might be that already before aa4ea25cd2e4d624e8f82d1c48d375f285e3266b I was defining trait bounds or using generics in a "wrong" manner.

Logs attached for timings I was able to produce with nightly (in stable it was taking too long for me to be able to wait):

rustc.logs.zip

[GlenDC]

When you apply the patch below to my current feature branch code
at https://github.com/plabayo/rama/tree/09c8bc0fea80560763c12149aceae3d9ddae87f5 then cargo check --workspace --timings --all-features --examples works again...

This surprises me as I thought maybe the introduction of ConnectorService would have caused this issue, but no... So only other difference is the introduction of an additional layer. Being that the HttpConnector is introduced as a way to have that logic for good reasons be taken out of HttpClient...

Then again, if the reason of all this is that I'm in generally misusing the typesystem then it might also just be that that one additional layer is the drop in the bucket to make it all go complete meltdown... That might explain why in my branched code (https://github.com/plabayo/rama/tree/09c8bc0fea80560763c12149aceae3d9ddae87f5) commenting out the use of RetryLayer from the http_high_level_client.rs example also makes it check...

FWIW despite these checks fail on some integration tests and that specific example, running actual binaries and examples do work... Somehow checking gets stuck but building not... 🤷

[jieyouxu]

The invocation is

RUSTFLAGS='-Ztime-passes -Z threads=8 -Z share-generics' cargo +nightly check --workspace --all-features --examples

where -Z threads=8 -Z share-generics may be significant. In any case

time: 148.279; rss:   85MB -> 10809MB (+10724MB)    type_check_crate

looks like a pathological type checking case.

[GlenDC]

Well normally i run on stable without any of these -z flags. There it takes a lot longer to complete. Noticed that indeed type_check_system was the big step here.

last night I let my machine running it by accident, which is how it reached 50gb of mem consumed by rustc process

[GlenDC]

Not an expert, but https://github.com/rust-lang/rust/issues/125267 might be related.

[GlenDC]

For now I am no longer blocked on this FWIW. I realized my design could be better. There was no reason to add these generics to HttpClient. Much better to have that just as an example that works for most normal cases and have users create their own HttpClient service for anything else. Doing this has made it that I am no longer getting these issues.