How to show image subsequently like video?

[dbsxdbsx]

Just like with opencv, in python it would be like:

    def show_image(self,
                   np_frame,
                   image_title="default title",
                   stop_interval=1):
        """return whether break
        stop_interval: if 0, the img would be held without auto update,except manully destroy the current img window
        """
        cv2.imshow(image_title, np_frame)
        if cv2.waitKey(
                stop_interval
        ) == 27:  # 27 is "esc"    another option: key == ord('q') or key == ord('Q') or
            print("get exit key")
            return True
        return False

Users can show_image as possible as they can, the later image would replace the former one in the same window, and the last image would not disappear until users press esc.

Now I want to do it in rust with minifb, with code like this:

mod image_tool;
use crate::image_multi::image_tool::Image;
use minifb::{Key, Window, WindowOptions};
use std::cell::RefCell;

use std::sync::{Arc, Mutex};

// Change the thread_local! to a global static variable
lazy_static::lazy_static! {
    static ref WINDOW: Arc<Mutex<Option<Window>>> = Arc::new(Mutex::new(None));
}

#[derive(PartialEq, Debug, Clone)]
pub enum ImageType {
    Screenshot { x: i32, y: i32, w: usize, h: usize },
    ImagePath(String),
}

pub fn test_show_image(title: Option<String>, image_type: ImageType) {
    let image = match image_type {
        ImageType::Screenshot { x, y, w, h } => Image::from_screen(x, y, w, h),
        ImageType::ImagePath(path) => Image::from_path(path),
    };
    let (width, height) = image.get_dims();

    // Pass the WINDOW variable to the show_window function
    show_window(title, width, height, image, Arc::clone(&WINDOW));
}

fn show_window(
    title: Option<String>,
    width: u32,
    height: u32,
    image: Image,
    window_arc: Arc<Mutex<Option<Window>>>,
) {
    let mut window_guard = window_arc.lock().unwrap();

    if window_guard.is_none() {
        let new_window = Window::new(
            &title.clone().unwrap_or_default(),
            width as usize,
            height as usize,
            WindowOptions {
                resize: true,
                ..WindowOptions::default()
            },
        )
        .unwrap();
        *window_guard = Some(new_window);
    }

    let window = window_guard.as_mut().unwrap();

    let buffer = image.get_buffer_u32();

    window
        .update_with_buffer(&buffer, width as usize, height as usize)
        .unwrap();

    // Drop the window_guard to unlock the Mutex
    drop(window_guard);
}

But the code is not compilable, due to WINDOW with issue

`*mut winapi::shared::windef::HBRUSH__` cannot be sent between threads safely
within `Option<Window>`, the trait `Send` is not implemented for `*mut winapi::shared::windef::HBRUSH__`
required for `Mutex<Option<Window>>` to implement `Sync`
1 redundant requirement hidden
required for `Arc<Mutex<Option<Window>>>` to implement `Sync`

I don't know if I am doing it a wrong way, or it is just impossible with minifb?

[emoon]

Hi,

It's seems that you are making this code a bit more complex than it has to be. I think doing something like this would be easier.

fn show_image(width: u32, height: u32,  image: Image, window: &Window) {
    let buffer = image.get_buffer_u32();

    window
        .update_with_buffer(&buffer, width as usize, height as usize)
        .unwrap();
}

fn main() {
   let mut window = Window::new(
        "Test - ESC to exit",
        WIDTH,
        HEIGHT,
        WindowOptions::default(),
    )
    .unwrap_or_else(|e| {
        panic!("{}", e);
    });

   while window.is_open() && !window.is_key_down(Key::Escape) {
       // TODO: Setup new image here
       /// 
       show_image(image, width, height, &window);
    }
}

[dbsxdbsx]

@emoon, thanks for your feedback. But still with your code(I refined it a little), it would stuck at while window.is_open() && !window.is_key_down(Key::Escape) since it is a loop. But luckily, I worked it out using thread_local!:

mod image_tool;
use std::cell::RefCell;

use crate::image_multi::image_tool::Image;
use minifb::{Key, Window, WindowOptions};

#[derive(PartialEq, Debug, Clone)]
pub enum ImageType {
    Screenshot { x: i32, y: i32, w: usize, h: usize },
    ImagePath(String),
}

thread_local! {
    static WINDOW: RefCell<Option<Window>> = RefCell::new(None);
}

fn show_image(image: &Image, width: u32, height: u32, window: &mut Window) {
    let buffer = image.get_buffer_u32();
    window
        .update_with_buffer(&buffer, width as usize, height as usize)
        .unwrap();
}

pub fn test_show_image_3(title: Option<String>, image_type: ImageType) {
    let image = match image_type {
        ImageType::Screenshot { x, y, w, h } => Image::from_screen(x, y, w, h),
        ImageType::ImagePath(path) => Image::from_path(path),
    };
    let (width, height) = image.get_dims();

    WINDOW.with(|window_cell| {
        let mut window_opt = window_cell.borrow_mut();
        let window = match &mut *window_opt {
            None => {
                let new_window = Window::new(
                    &title.unwrap_or_default(),
                    width as usize,
                    height as usize,
                    WindowOptions {
                        resize: true, // resizable by mouse
                        ..WindowOptions::default()
                    },
                )
                .unwrap_or_else(|e| {
                    panic!("{}", e);
                });
                *window_opt = Some(new_window);
                window_opt.as_mut().unwrap()
            }
            Some(window) => {
                // Update window info
                window.set_title(&title.unwrap_or_default());
                // TODO: window.set_size(width as usize, height as usize);
                window
            }
        };

        // Directly use the window variable
        if window.is_open() && !window.is_key_down(Key::Escape) {
            show_image(&image, width, height, window);
        } else {
            // // close the window safely
            drop_window();

        }
    });
}

fn drop_window() {
    WINDOW.with(|window_cell| {
        let mut window_opt = window_cell.borrow_mut();
        if let Some(window) = window_opt.take() {
            drop(window);
        }
    });
}

With this code, now user can use it directly like this:

use rand::Rng;

fn main() {
    // let ten_millis = std::time::Duration::from_millis(1000);
    loop {
        let a = rand::thread_rng().gen_range(0..=8);
        test_show_image_3(
            Some(format!("title{}", a)),
            ImageType::Screenshot {
                x: a,
                y: a,
                w: 500,
                h: 500,
                // panic with different sizes
                // w: (a * 100) as usize,
                // h: (a * 100) as usize,
            },
        );
        // std::thread::sleep(ten_millis);
    }
}

But there are 2 drawbacks, 1. the existing window can't be resized; 2. when drop_window() is called, it would panic at https://github.com/emoon/rust_minifb/blob/master/src/os/windows/mod.rs, in line of winuser::ReleaseDC(self.window.unwrap(), self.dc.unwrap()); with error msg like "exception has occurred: W32/0xC0000005 Unhandled exception at 0x00007FFB898F31E1 (gdi32.dll) in test_rust.exe: 0xC0000005: Access violation reading location 0x00000000000291FD." on windows 10.

[emoon]

I wouldn't recommend doing it this way.

On some OSes the APIs used for windowing is only supported on the main thread. If you need to use several threads it's better to do the decoding on separate threads and then have the main thread read the result and have all the window handling there.

[dbsxdbsx]

On some OSes the APIs used for windowing is only supported on the main thread.

I am using thread_local!, which implies that all stuff is on the same thread.

Besides, sometimes users would need to do live screenshot videos. Thus, no idea how many images would be shown, so I think handling all images together inside a certain function is not feasible.

[emoon]

I am using thread_local!, which implies that all stuff is on the same thread.

Yes, but not local to the main thread. So if you call test_show_image_3 from another thread it will not be the case.

[dbsxdbsx]

I am using thread_local!, which implies that all stuff is on the same thread.

Yes, but not local to the main thread. So if you call test_show_image_3 from another thread it will not be the case.

So within my solution, I guess I have to make sure everything is running in the same thread.