hyousefGopher commented 4 years ago

I'm trying to run Python code calling OpenCV using your crate, and it worked perfectly as below, along with your crate I used web-view crate, and Windows displaying the OpenCV output had been displayed.

What I plat to do, is to take this output as image or as video and return it back to the Rust code, so that I can display it in the web-view, can you help in this? If this can be done using #![context = &c] appreciate to show me how to do it for the image/frame.

Thanks

#![feature(proc_macro_hygiene)]
use inline_python::python;

use web_view::*;

fn main() {
    web_view::builder()
    .title("Call open CV")
    .content(Content::Html(HTML))
    .size(800, 600)
    .user_data(())
    .invoke_handler(|_, arg| {
        match arg {
            "python" => {
                    python!  {
                            print("Hi from Python in line")
                            import cv2

                            cap = cv2.VideoCapture(0)
                            while cap.isOpened():
                                ret, frame = cap.read()
                                if ret:
                                    cv2.imshow("Frame", frame)
                                    if cv2.waitKey(25) & 0xFF == ord('q'):
                                       break
                                else:
                                    break

                            cap.release()
                            cv2.destroyAllWindows()
                    }
            },
            _ => (),
        }
        Ok(())
    }).run()
    .unwrap();
}

const HTML: &str  = r#"
<!doctype html>
<html>
    <body>
        <button onclick="external.invoke('python')">Run</button>
    </body>
</html>
"#;

de-vri-es commented 4 years ago

You can use the context to retrieve a python object. Assuming it is a numpy array (which I believe OpenCV images in Python are), you can then use the numpy crate to interact with it.

Note that the Context API will change somewhat in the upcoming 0.5.0 release.

de-vri-es commented 4 years ago

0.5.0 was just released :tada:

In particular, you can use Context::get to retrieve a global variable from your python script, and then use the numpy crate to inspect the data.

de-vri-es commented 4 years ago

Btw, there is also a Rust wrapper for OpenCV, which may be more ergonomic in use: https://docs.rs/opencv/

I have never worked with it myself though, so I can't be sure.

hyousefGopher commented 4 years ago

Thanks @de-vri-es

I tried simple context::get example as below it worked, will see how can I apply it for my concept
```
#![feature(proc_macro_hygiene)]
use inline_python::*;
```

fn main() { let c = Context::new(); c.set("x", 3); c.run(python!{ x += 2 print("x = {}".format(x)) });

let z: i8 = c.get("x");
println!("z = {}", z);

}



2. I saw that lib, but trying different approach, that is reducing the binding channels, so instead of wrapping Rust with Python, which is already a wrapper for CPP, better to call CPP directly, or single binding layer, that is calling CPP through Python

hyousefGopher commented 4 years ago

In Context::get it looks I've issues getting the returned type as image, tried:

use image::{ImageFormat, GenericImageView, DynamicImage};

let img: DynamicImage = c.get("frame")

And

use imgref::*;

let img: Img<Vec<i32>> = c.get("frame");

But got the errors:

45 |                 let img: DynamicImage = c.get("frame");
   |                                            ^^^ the trait `pyo3::pyclass::PyClass` is not implemented for `imgref::Img<std::vec::Vec<i32>>`

And

44 |                 let img: Img<Vec<i32>> = c.get("frame");
   |                                            ^^^ the trait `pyo3::pyclass::PyClass` is not implemented for `imgref::Img<std::vec::Vec<i32>>`

Looking how to fix any of them, any thought/idea?

de-vri-es commented 4 years ago

As you can see in the docs, Context::get requires the return type to implement pyo3::FromPyObject. You can't just get any arbitrary type from python and expect a magic conversion to happen.

hyousefGopher commented 4 years ago

mmm, is there a way to make the Context::get return an image?

hyousefGopher commented 4 years ago

I was able to make it using your conetext as in version 0.5.1 as below:

With inline-python I run OpenCV, display the image, which is 3D array as of ndarray,
Using Context I return this 3D array as Vec<Vec<Vec<i32>>> to Rust
Using image I create Rust image from this Vec
```
#![feature(proc_macro_hygiene)]
```

use inline_python::pyo3::{self, prelude::, wrap_pyfunction}; use inline_python::{python, Context}; use image::{ImageBuffer, RgbImage}; use web_view::;

[pyfunction]

fn rust_image(vec: Vec<Vec<Vec>>) { let mut imgbuf: RgbImage = ImageBuffer::new(vec[0].len() as u32, vec.len() as u32); for (x, y, pixel) in imgbuf.enumerate_pixels_mut() { let r = vec[y as usize][x as usize][2] as u8; let g = vec[y as usize][x as usize][1] as u8; let b = vec[y as usize][x as usize][0] as u8; *pixel = image::Rgb([r, g, b]); } imgbuf.save("fractal.png").unwrap(); }

fn main() { let c = Context::new(); c.add_wrapped(wrap_pyfunction!(rust_image));

web_view::builder()
.title("Call open CV")
.content(Content::Html(HTML))
.size(800, 600)
.user_data(())
.invoke_handler(|_, arg| {
    match arg {
        "open" => {
            c.run(python!  {
                        import cv2
                        x = 2
                        cap = cv2.VideoCapture(0)
                        ret, frame = cap.read()
                        cv2.imshow("frame", frame)
                        cv2.waitKey(0)
                        rust_image(frame)
                })
        },
        "close" => {
            c.run(python!  {
                # execfile("/home/el/foo2/mylib.py")
                # from fox import load
                # load()
            })
        },
        _ => (),
    }
    Ok(())
}).run()
.unwrap();

}

const HTML: &str = r#" <!doctype html>

"#;

fusion-engineering / inline-python

Can I return image or frame from python_inline to Rust? #21

[pyfunction]