francesca64
commented
6 years ago Thanks for the issue report!
- Which platform does this happen on?
- Does this also happen when using winit without glutin?
Closed fasihrana closed 2 years ago
francesca64
commented
6 years ago Thanks for the issue report!
fasihrana
commented
6 years ago This happens on both Windows and Ubuntu.
I have not checked winit on its own since i've been working on webrender based project which uses glutin.
fasihrana
commented
6 years ago So I just confirmed that it also happens on winit although the amount of memory held on to is about 76K as per windows task manager. Though this effect is definitely pronounced when using glutin and gets even worse when you add webrender on top of it.
goddessfreya
commented
5 years ago Hey @fasihrana sry for the late reply...
Soooo, I was able to reproduce this on linux.
So, I used this modified example:
extern crate glutin;
use glutin::ContextTrait;
use std::thread;
use std::time::Duration;
struct Win {
ctx: glutin::CombinedContext,
ev: glutin::EventsLoop,
}
impl Win {
fn new() -> Win {
let ev = glutin::EventsLoop::new();
let wb = glutin::WindowBuilder::new()
.with_dimensions(glutin::dpi::LogicalSize::new(300.0, 200.0));
let ctx = glutin::ContextBuilder::new()
.with_gl(glutin::GlRequest::Specific(glutin::Api::OpenGl, (3, 2)))
.build_combined(wb, &ev)
.unwrap();
unsafe {
ctx.make_current().ok();
}
Win {
ctx,
ev,
}
}
fn event(&mut self) -> (bool, bool) {
let mut exit = false;
let mut add = false;
self.ev.poll_events(|e| match e {
glutin::Event::WindowEvent { event, .. } => match event {
glutin::WindowEvent::CloseRequested => {
exit = true;
}
glutin::WindowEvent::MouseInput { state, .. } => {
if state == glutin::ElementState::Released {
add = true;
}
}
_ => (),
},
_ => (),
});
(exit, add)
}
}
fn main() {
let mut wv = vec![];
wv.push(Win::new());
loop {
let mut i = 0;
while i < wv.len() {
let (close, add) = wv[i].event();
if close {
wv.remove(i);
} else {
i += 1;
}
if add {
wv.push(Win::new());
}
}
if wv.len() == 0 {
wv.push(Win::new());
}
thread::sleep(Duration::from_millis(1000));
}
}And I experienced a growth in mem usage. I then ran rm massif.out.*; valgrind --tool=massif --max-snapshots=1000 --detailed-freq=1 --depth=200 --trace-children=yes --peak-inaccuracy=0.5 --stacks=yes target/debug/examples/test && massif-visualizer massif.out.* and noticed nothing abnormal.
Running massif with --pages-as-heap=yes instead of stacks=yes results in this abnormal graph: https://i.imgur.com/oRtBjNd.png, and reveals that the source of the page usage is start_thread in libpthread.
Further investigation reveals that each time we make a new window, we make at least one thread. Stepping through the code, line by line, reveals that the source is this:
#0 u_thread_create (param=param@entry=0x1013c6140, routine=0x7ffff5ab7df0 <util_queue_thread_func>) at ../mesa/src/util/u_thread.h:39
#1 0x00007ffff5ab8305 in util_queue_init (queue=queue@entry=0x1013d1cf0, name=name@entry=0x7ffff5d90a57 "gdrv", max_jobs=max_jobs@entry=8,
num_threads=num_threads@entry=1, flags=flags@entry=0) at ../mesa/src/util/u_queue.c:372
#2 0x00007ffff5beabb3 in threaded_context_create (pipe=0x101425200, parent_transfer_pool=0x1006aaa90, replace_buffer=0x7ffff55b3ec0 <si_replace_buffer_storage>,
create_fence=0x7ffff55c5030 <si_create_fence>, out=0x1014255f8) at ../mesa/src/gallium/auxiliary/util/u_threaded_context.c:2609
#3 0x00007ffff58037e6 in st_api_create_context (stapi=<optimized out>, smapi=0x1006a38a0, attribs=0x7fffffff02c0, error=0x7fffffff02bc, shared_stctxi=0x0)
at ../mesa/src/mesa/state_tracker/st_manager.c:923
#4 0x00007ffff53cd88a in dri_create_context (api=<optimized out>, visual=<optimized out>, cPriv=0x1013814b0, ctx_config=<optimized out>, error=0x7fffffff04d4,
sharedContextPrivate=<optimized out>) at ../mesa/src/gallium/state_trackers/dri/dri_context.c:161
#5 0x00007ffff53c83a6 in driCreateContextAttribs (screen=0x1006a2520, api=<optimized out>, config=0x1009c3ea0, shared=<optimized out>, num_attribs=<optimized out>,
attribs=<optimized out>, error=0x7fffffff04d4, data=0x101381310) at ../mesa/src/mesa/drivers/dri/common/dri_util.c:473
#6 0x00007ffff6820f6b in dri3_create_context_attribs (base=0x1005d4ca0, config_base=0x1009e1a50, shareList=<optimized out>, num_attribs=<optimized out>,
attribs=<optimized out>, error=0x7fffffff04d4) at ../mesa/src/glx/dri3_glx.c:308
#7 0x00007ffff6808b8a in glXCreateContextAttribsARB (dpy=0x100584f70, config=0x1009e1a50, share_context=0x0, direct=1, attrib_list=0x101381f30)
at ../mesa/src/glx/create_context.c:78
#8 0x00007ffff7b8516c in glXCreateContextAttribsARB (dpy=0x100584f70, config=0x1009e1a50, share_list=0x0, direct=1, attrib_list=0x101381f30) at libglx.c:305
#9 0x000000010008c509 in glutin::api::glx::ffi::glx_extra::Glx::CreateContextAttribsARB (self=0x7fffffff0b08, dpy=0x100584f70, config=0x1009e1a50, share_context=0x0,
direct=1, attrib_list=0x101381f30) at /home/gentz/Documents/gfx/glutin/target/debug/build/glutin-6f7522a6498d80d0/out/glx_extra_bindings.rs:528
#10 0x00000001000882a4 in glutin::api::glx::create_context (glx=0x7fffffff6cb0, extra_functions=0x7fffffff0b08, extensions=..., xlib=0x100599f80, version=...,
profile=..., debug=true, robustness=glutin::Robustness::NotRobust, share=0x0, display=0x100584f70, fb_config=0x1009e1a50, visual_infos=0x7fffffff6f50)
at src/api/glx/mod.rs:490
#11 0x00000001000873fb in glutin::api::glx::ContextPrototype::finish (self=..., window=33554483) at src/api/glx/mod.rs:289
#12 0x000000010008b55f in glutin::platform::platform::x11::Context::new (wb=..., el=0x7fffffffa6c8, pf_reqs=0x7fffffff9dd0, gl_attr=0x7fffffff9800)
at src/platform/linux/x11.rs:262
#13 0x000000010009e94d in glutin::platform::platform::Context::new (wb=..., el=0x7fffffffa6c8, pf_reqs=0x7fffffff9dd0, gl_attr=0x7fffffff9e18)
at src/platform/linux/mod.rs:109
#14 0x0000000100089741 in glutin::combined::CombinedContext::new (wb=..., cb=..., el=0x7fffffffa6c8) at src/combined.rs:58
#15 0x0000000100093168 in glutin::ContextBuilder::build_combined (self=..., wb=..., el=0x7fffffffa6c8) at src/lib.rs:286
#16 0x000000010006aef6 in test::Win::new () at examples/test.rs:18
#17 0x000000010006b276 in test::main () at examples/test.rs:69I'm afraid I don't have the source file ../mesa/src/util/u_thread.h so I couldn't go deeper. Interestingly, mesa does destroy the thread it makes when we destroy the context. Now, if I recall correctly, pthreads will "cache" the threads it makes, so the extra gigabytes worth of pages might not really be there, so I think we can safely ignore this.
However, I did not stop there.
If we replace line 71 with a break instead of a wv.push(...) and run valgrind --tool=memcheck --leak-check=full --track-origins=yes --show-leak-kinds=all ./target/debug/examples/test, we reveal 8.5 mbs of complaints, mostly from mesa-related stuff (likely all false-positives). If we strip out everything which doesn't mention either glutin or winit, we get this (stripped via grep "winit\|glutin" -C 13, so not exactly accurate): https://termbin.com/75ku
Let's take a look at the candidates in glutin:
dlopen but no dlclose! Fixed: #1058 goddessfreya
commented
5 years ago The main cause of this issue is going to be however: https://i.imgur.com/1xhOKQl.png
Produced with:
extern crate glutin;
use std::thread;
use std::time::Duration;
struct Win {
win: glutin::Window,
ev: glutin::EventsLoop,
}
impl Win {
fn new() -> Win {
let ev = glutin::EventsLoop::new();
let win = glutin::WindowBuilder::new()
.with_dimensions(glutin::dpi::LogicalSize::new(300.0, 200.0))
.build(&ev)
.unwrap();
Win { win, ev }
}
fn event(&mut self) -> (bool, bool) {
let mut exit = false;
let mut add = false;
self.ev.poll_events(|e| match e {
glutin::Event::WindowEvent { event, .. } => match event {
glutin::WindowEvent::CloseRequested => {
exit = true;
}
glutin::WindowEvent::MouseInput { state, .. } => {
if state == glutin::ElementState::Released {
add = true;
}
}
_ => (),
},
_ => (),
});
(exit, add)
}
}
fn main() {
let mut wv = vec![];
wv.push(Win::new());
loop {
let mut i = 0;
while i < wv.len() {
let (close, add) = wv[i].event();
if close {
wv = vec![]
} else {
i += 1;
}
if add {
wv.push(Win::new());
wv.push(Win::new());
wv.push(Win::new());
wv.push(Win::new());
wv.push(Win::new());
wv.push(Win::new());
wv.push(Win::new());
wv.push(Win::new());
wv.push(Win::new());
wv.push(Win::new());
wv.push(Win::new());
}
}
if wv.len() == 0 {
// wv.push(Win::new());
break;
}
thread::sleep(Duration::from_millis(1000));
}
}That is some excellent investigation!
Although this still leaves us with Windows issue unaddressed. Not sure if correcting X11 issue referenced here will also deal with Windows platform.
And I haven't even tested this on a Mac.
goddessfreya
commented
5 years ago I didn't feel satisfied with "this is an upstream issue" when it came to the X11 backend, so I did some further digging (as was part of the 0.21 milestone).
I had to compile libx11 (wo/opts and w/debug symbols) and glibc (w/ debug symbols) to confirm my suspicions, but here we are:
When winit calls XOpenIM, it parses some sort of (config?) file (not sure what exactly) to populate some sort of (token?) tree. This tree is allocated with realloc. When free is called on this memory, glibc doesn't free it, as it's part of some arena (sorry, best description I can give. Not that familiar with the glibc source code). This can be confirmed in two ways:
First, one can simply drop all the windows midway into the exe, then start making new ones, and note how mem consumption doesn't rise any more:
extern crate glutin;
use std::thread;
use std::time::Duration;
struct Win {
win: glutin::Window,
ev: glutin::EventsLoop,
}
impl Win {
fn new() -> Win {
let ev = glutin::EventsLoop::new();
let win = glutin::WindowBuilder::new()
.with_dimensions(glutin::dpi::LogicalSize::new(300.0, 200.0))
.build(&ev)
.unwrap();
Win { win, ev }
}
fn event(&mut self) -> (bool, bool, bool) {
let mut exit = false;
let mut add = false;
let mut running = true;
self.ev.poll_events(|e| match e {
glutin::Event::WindowEvent { event, .. } => match event {
glutin::WindowEvent::CloseRequested => running = false,
glutin::WindowEvent::MouseInput { state, .. } => {
if state == glutin::ElementState::Released {
add = true;
}
}
glutin::WindowEvent::KeyboardInput {
input:
glutin::KeyboardInput {
virtual_keycode:
Some(glutin::VirtualKeyCode::Q),
..
},
..
} => exit = true,
_ => (),
},
_ => (),
});
(exit, add, running)
}
}
fn main() {
let mut wv = vec![];
wv.push(Win::new());
'outer: loop {
let mut i = 0;
while i < wv.len() {
let (close, add, running) = wv[i].event();
if !running {
break 'outer;
}
if close {
wv = vec![]
} else {
i += 1;
}
if add {
wv.push(Win::new());
wv.push(Win::new());
wv.push(Win::new());
wv.push(Win::new());
wv.push(Win::new());
wv.push(Win::new());
wv.push(Win::new());
wv.push(Win::new());
wv.push(Win::new());
wv.push(Win::new());
wv.push(Win::new());
}
}
if wv.len() == 0 {
wv.push(Win::new());
}
thread::sleep(Duration::from_millis(1000));
}
}Second, if one has a loop which drops the prior window before making a new one, memory will not rise. See this modified main fn:
fn main() {
let mut wv = Win::new();
'outer: loop {
let (close, add, running) = wv.event();
std::mem::drop(wv);
wv = Win::new();
if !running {
break 'outer;
}
thread::sleep(Duration::from_millis(1000));
}
}Anyways, this is glibc just functioning as (un)expected.
kchibisov
commented
2 years ago There're no windows in glutin anymore.
Based on the result of opening and closing more than one window I'm finding that a little extra memory is left assigned to the process than the memory the process started with. I've created a small multi window sample here.
Are there any know memory leak issues in glutin/winit when closing windows?