ValveSoftware / openvr

OpenVR SDK
http://steamvr.com
BSD 3-Clause "New" or "Revised" License
6.07k stars 1.28k forks source link

How to use IVRNotifications? #1133

Open AndersMalmgren opened 5 years ago

AndersMalmgren commented 5 years ago

I cant seem to get Notifications working, can you guys provide docs please?

I talk about the SteamVR notifications, IVRNotifications, CreateNotification etc

AndersMalmgren commented 5 years ago

I invested some time in this, Just a few days ago this was pushed to the Openvr repo

https://github.com/ValveSoftware/openvr/commit/823135df1783009cb468d0fc4190816254f7687d#diff-a7e4e65799ecc1976f2eca8869a9c7a7

It adds support to the IVRNotifications API. I tried it out and it works.

I suggest a wrapper that takes care of creating the overlay. But more important act as a wrapper for a Unity sprite so we can use a Unity sprite object as image for the overlay.

AndersMalmgren commented 5 years ago

Also I cant get the icon image to work, tried,

            image.m_nBytesPerPixel = ?;
            image.m_pImageData = MyImage.texture.GetNativeTexturePtr();
            image.m_nWidth = MyImage.texture.width;
            image.m_nHeight = MyImage.texture.height;

Unity crashes hard

AndersMalmgren commented 5 years ago

Also tried

            var bytes = MyImage.texture.GetRawTextureData();
            IntPtr unmanagedPointer = Marshal.AllocHGlobal(bytes.Length);
            Marshal.Copy(bytes, 0, unmanagedPointer, bytes.Length);

            image.m_nBytesPerPixel = 4;
            image.m_pImageData = unmanagedPointer;
            image.m_nWidth = MyImage.texture.width;
            image.m_nHeight = MyImage.texture.height;

Doesnt crash, but no image in notification

AndersMalmgren commented 5 years ago

I created a minimal example in Unity without the SteamVR plugin just raw OpenVR,

        var e = EVRInitError.None;
        OpenVR.Init(ref e, EVRApplicationType.VRApplication_Overlay);

        ulong handle = 0;
        var error = OpenVR.Overlay.CreateOverlay(Guid.NewGuid().ToString(), "My app", ref handle);

        var image = new NotificationBitmap_t();
        var bitmap = new Bitmap(File.OpenRead(@"myimage.png"));

        System.Drawing.Imaging.BitmapData TextureData =
            bitmap.LockBits(
                new Rectangle(0, 0, bitmap.Width, bitmap.Height),
                System.Drawing.Imaging.ImageLockMode.ReadOnly,
                System.Drawing.Imaging.PixelFormat.Format32bppArgb
            );

        var foo = BitConverter.IsLittleEndian;

        image.m_pImageData = TextureData.Scan0;
        image.m_nWidth = TextureData.Width;
        image.m_nHeight = TextureData.Height;
        image.m_nBytesPerPixel = 4;

        uint id = 0;
        OpenVR.Notifications.CreateNotification(handle, 0, Valve.VR.EVRNotificationType.Transient, "foobar", Valve.VR.EVRNotificationStyle.Application, ref image, ref id);

        //Unlocks Image Data
        bitmap.UnlockBits(TextureData);

This works in vanilla .net but not in Unity, the images does not show, the notification does show but without image. Image work in vanilla. It seems the pointer data types differ in .net and mono/unity.

In mono: image

.net image

bddckr commented 5 years ago

Related: https://github.com/ValveSoftware/openvr/issues/1038

AndersMalmgren commented 5 years ago

@bddckr Yep, its related to x64 dll