Webtexture with Canvas Renderer & AwesomiumUnity.DLL

[TheMajorO]

Hey @bkeiren , thank you so much for this awesome refactored wrapper and for the decent features you implemented. Your instructions in README.MD led to a successful attempt of integrating Awesomium with Unity using your awesome wrapper on my 64-bit windows 7 and 32-bit Unity 5.1.

Though I would like to ask you the following things: -1. Would kindly allow integration of AwesomiumUnityWebTexture.CS component with Canvas Renderer? I can only use either two methods: (Plane or any object with mesh renderer, material and collider OR deprecated GUITexture) I tried changing AwesomiumUnityWebTexture.CS, namely "void Initalize()", by creating a sprite using Texture2D attribute "m_texture" and assigning that sprite to Image component in a Canvas-child-object but that didn't work properly; image was uninteractive(it did view the page though). Perhaps I should also change OnMouseDown()?

EDIT: webpage is indeed loadable after making changes to both "Initialize()" and "OnMouseDown()", but uninteractive. Here's the code for AwesomiumUnityWebTexture.Initialize() & AwesomiumUnityWebTexture.OnMouseDown()

    public void Initialize()
    {
    m_HasBeenInitialized = true;

    AwesomiumUnityWebCore.EnsureInitialized();

    // Call resize which will create a texture and a webview for us if they do not exist yet at this point.
    Resize(m_Width, m_Height);
    var h = GetComponent<Image>();

    if (h)
    {
        var piv = GetComponent<RectTransform>().pivot;
        var myRect = new Rect();//GetComponent<RectTransform>().rect; produces a an error (current bug in unity 5.1.1f1)
        myRect.x = myRect.y = 0;
        myRect.width = m_Texture.width;
        myRect.height = m_Texture.height;
        var img = GetComponent<Image>();
        img.sprite = Sprite.Create(m_Texture, myRect, piv);
    }
    }
    //------------------------------------------OnMouseOver()------------------------------------------
    void OnMouseOver()
    {
    if (!m_Interactive) return;

    if (m_WebView != null /*&& !m_WebView.IsLoading*/)
    {
        //!--Maj
        var img = GetComponent<Image>();
        if (img)
        {
            float rectWidth =  img.rectTransform.rect.width;
            float rectHeight =  img.rectTransform.rect.height;
            float rectX =  img.rectTransform.rect.x;
            float rectY =  img.rectTransform.rect.y;

            int MouseX = (int)Input.mousePosition.x;
            int MouseY = Screen.height - (int)Input.mousePosition.y;

            MouseX = (int)(((MouseX - rectX) / rectWidth) * m_Width);
            MouseY = (int)(((MouseY - rectY) / rectHeight) * m_Height);

            m_WebView.InjectMouseMove(MouseX, MouseY);

        }
    }
    }

-2. Is it possible we view AwesomiumUnity.DLL? I couldn't decompile it since it's unmanaged C++ code. If it's a no then that is totally fine.

Again thank you so much for the hard work! I really hope you see this. I know you are really preoccupied but I believe in the potential of Awesomium and delighted with the easily understandable wrapper you wrote.

[bkeiren]

Hi!

Thanks for your interest and kind words!

To start off with your question about being able to view AwesomiumUnity.DLL: The pre-compiled binary DLL that comes with AwesomiumUnity is compiled from the code in CppDLL/AwesomiumUnity in this repository. You already have full source code access :)

It's good to hear that you managed to get the page to load and display it! You can relay mouse and keyboard input to the webview by calling the following functions on it:

• InjectMouseMove
• InjectMouseUp
• InjectMouseDown
• InjectMouseWheel
• InjectKeyboardEvent

You can decide for yourself when to call these, but typically you would do so in your scripts's OnGUI function (see https://github.com/bkeiren/AwesomiumUnity/blob/master/AwesomiumUnityScripts/AwesomiumUnityWebTexture.cs#L160 for an example).

If you have any further questions I'm all ears :)

Cheers,

[TheMajorO]

Thank you so much for your kind and quick informative reply! Apparently AwesomiumUnity.DLL's source code is indeed visible, it was stupid of me not to look really good enough :P

I will do what you suggested in order to have WebTexture work on canvas renderer, let's hope things go well! Though I would rather implement it with Unity's new UI system using RawImage since it takes Texture2D; I will try to avoid using OnGUI and see how things work out! (I know this might sound not so exciting to somebody who works on relatively low-level programming - I do too sometimes :P - but this is more convenient and general)

When I'm talking about 'The new Unity UI" this is what I'm talking about: forum.unity3d.com/threads/from-ongui-to-new-ui.309751/#post-2015419

[bkeiren]

I think it should be easy enough to integrate AwesomiumUnity with the new UI system. As long as you get the callbacks required to know of those events I mentioned and can pass them through, you're set. (Of course you'd also need to have some sort of image buffer to render to but I think you got that down already, right?)

And don't worry, I enjoy and do lots of higher-level programming all the time! All types of programming have their uses and advantages, just like they all have their disadvantages.

Cheers

On 2 July 2015 at 19:42, TheMajorO notifications@github.com wrote:

Thank you so much for your kind and quick informative reply! Apparently AwesomiumUnity.DLL's source code is indeed visible, it was stupid of me not to look really good enough :P

I will do what you suggested in order to have WebTexture work on canvas renderer, let's hope things go well! Though I would rather implement it with Unity's new UI system using RawImage since it takes Texture2D; I will try to avoid using OnGUI and see how things work out! (I know this might sound not so exciting to somebody who works on relatively low-level programming - I do too sometimes :P - but this is more convenient and general)

When I'm talking about 'The new Unity UI" this is what I'm talking about: forum.unity3d.com/threads/from-ongui-to-new-ui.309751/#post-2015419

— Reply to this email directly or view it on GitHub https://github.com/bkeiren/AwesomiumUnity/issues/6#issuecomment-118106580 .

[TheMajorO]

Hey @bkeiren , I got it to work! Keyboard input, mouse input etc... Took lots of time and effort but it did work eventually, thanks to your awesomely refactored and self-explanatory code! Though it was a big paradigm shift; since I'm using new Unity UI which heavily depends on events like OnPointerEnter(PointerEventData) and OnPointerDown(PointerEventData) etc.. for Focus() as well as InjectMouseUp(...) and InjectMouseDown(...) rather than depreciated OnGUI() and whatnot. Also I had to use Update() for: InjectKeyboardEvent(...) and InjectMouseMove(...).

Liked that MapModifiers() (to detect multiple key modifiers) where you used bitwise ORing :) took me back to those days in ASM class lol

I would like to ask ya about two issues though:

1 I wonder if it's possible to send "held keys" to the engine? right now your wrapper has

public enum AwesomiumUnityWebKeyType
{
     KeyDown,
     KeyUp,
     Char
}

which is used in AwesomiumUnityWebKeyboardEvent (No KeyHold or Key enumeration exists). For key input queries I'm using Input.UnityEngine.GetKeyDown, Input.GetKeyUp, and Input.GetKey(when holding buttons) and the latter doesn't work in this case. For example when holding x key I get x once, but not repeatedly(only GetKeyDown is registered).

2 Is it possible to get Ctrl-A (Select all) and Copy-Paste to work? Should I import external functions from Awesomium.dll and call them directly like what you did in AwesomiumUnityWebView? or is it possible to do that using your wrapper?

I know you're busy, so I understand if you wouldn't elaborately answer my questions but rather direct me with the right keywords :)

Appreciate your collaborative spirit man; I would love to share this new method I got to work with new Unity UI system as soon as it's complete, and add it to your awesome work!

[bkeiren]

Hey!

Great to hear you got it to work, and your excitement is wonderful to see!

As far as key input is concerned, I believe that Awesomium does not assume responsibility for these types of actions (copy, paste, select-all, etc.). Rather, it is up to the user (in this case you or I) to implement this. The reasoning behind this would be that such behaviour, while standard in most regular webbrowsers, might not be desired for other purposes (for example, an in-game UI) for which Awesomium would typically be used, or the programmer might want to bind different keys to the actions. Basically, by not implementing this behaviour natively it is possible to customize it more easily. Some may call it lazy on behalf of Awesomium, some may call it staying flexible.

AwesomiumUnity doesn't attempt to provide this behaviour either, but it does expose the normal input injection functionality that Awesomium also provides (the Inject* functions you've seen).

If you want to interact with the page to copy selected content you could catch the Ctrl+C event in your own code and then use any of the JavaScript callback functions to copy the content and obtain the result (see this page I found, while the actual code to execute the JavaScript will be slightly different because you're using AwesomiumUnity, the JS code should be the same.

As far as repeating key input is concerned, this is also something you can implement yourself. The reason why its not there is again customizability, because think for example of the different factors that any implementation of key-repeat deals with: How quickly does the key repeat? How much time is there between the initial key press and the start of the repeating sequence? By putting it to the programmer to implement this behaviour Awesomium rids itself of responsibility and allows you to do it however you want to. If I were you I would get the key down event and store it for the next frame. You keep track of how much time has passed since that initial key press and once enough time has passed (300 milliseconds perhaps?) you start to inject that key every, say, 200 or 300 milliseconds into the web view. This would give you a key-repeat that starts 0.3 seconds after the initial key press, and repeats every 0.2 to 0.3 seconds.

Lastly, I appreciate the effort too, man! By all means once you feel it's ready we could take a look at integrating your work into the project and let other people use your code too!

Cheers,

[TheMajorO]

Many thanks! for the speedy reply, elaborate answer, fine reasoning, and the link to the JS code! I actually was going to ask you how I can execute JS queries with a return value since it confused me. But guess you beat me to it :) I really needed that for hiding default scroll bars and other stuff. Also thank you for the tip on achieving Key Input Repeat/Character Repeat! I am going to try achieving that using a bool flag and a constant interval (in ms) as you suggested to produce Character Repeat effect. Let's hope things go well!

Appreciate directing me in the right direction! Now I can do almost all kind of stuff with JS execution and perhaps even call Awesomium.DLL functions if needed(and if the functionality those functions serve were not already present in your wrapper nor possible to do with JS queries).

All the best,

[TheMajorO]

Hey @bkeiren , almost everything is working perfectly! Thanks for your advice :) Executing JavaScript functions is smooth, except for things like:

        //web.WebView.ExecuteJavaScript("(function(){ document.selection.createRange().pasteHTML(" + text + ");})();", null);
        //web.WebView.ExecuteJavaScript("document.selection.createRange().pasteHTML(" + text + ");", null);
        //web.WebView.ExecuteJavaScript("if (window.getSelection) {var newNode = document.createElement(\"span\");newNode.innerHTML = "+text+";window.getSelection().getRangeAt(0).insertNode(newNode);} else {document.selection.createRange().pasteHTML("+text+");}", null);
        //web.WebView.ExecuteJavaScript("document.selection.empty();", null);
        //web.WebView.ExecuteJavaScript("window.Selection = \"\";",null);
        //web.WebView.ExecuteJavaScript("window.getSelection().removeRange;", null);
        //web.WebView.ExecuteJavaScript("window.getSelection().deleteFromDocument;", null);

Which I intended to use for pasting purposes... Alas none of those attempts worked... maybe the error is in the JS syntax itself? I really have no idea though other functions/executions regarding other purposes worked. Also, this execution you provided:

m_WebTexture.WebView.ExecuteJavaScript("alert(\"test\");", null);

in your Example0.cs did not work, unfortunately.

The link you referred me to in your last answer worked like charm though. And I was able to do pseudo-copy via Unity after retrieving selection! I tried calling Awesomium.DLL methods for pasting but could not figure out how to do it. Namely, I tried calling WebView::Copy. Here's its info in the docs: http://docs.awesomium.net/?tc=M_Awesomium_Core_WebView_Copy

Here's my attempt which I added locally to your AwesomiumUnityWebView.cs(After putting the SDK dlls in "PLUGIN" folder & adding environment var -PATH- pointing to Unity's Editor folder):

    internal const string CustomDLLName1 = "Awesomium.dll";
    /// DLL Imported functions.
    //EXTRA
    [DllImport(CustomDLLName1, CallingConvention = CallingConvention.Cdecl)]
    extern static private void Copy(System.IntPtr _Instance);

    [DllImport(CustomDLLName1, CallingConvention = CallingConvention.Cdecl)]
    extern static private void Paste(System.IntPtr _Instance);
    //end of EXTRA

Though unfortunately, that did not work. Just wanted to show you how I approached this. I have no idea how to drill down in Awesomium.DLL in order to reach Awesomium.Core namespace and access WebView class to finally call Paste() or Copy()!

Since I could not paste text using 1. JaveScript, nor 2. using Inject_KeyboardEvent (characters being injected are always of a constant size.. there are 20B and 4B char arrays your wrapper is providing). Then I started thinking about the possibility of interoperating with Awesomium.dll directly just like your wrapper and use Copy and Paste functions and other useful ones like CopyImage... I would be so glad if you could point me in the direct direction regarding that matter since you successfully made the Unity Wrapper.

Appreciate your help in advance! and the speedy replies :) Again I understand you're busy so doing your best is more than enough. ^^

All the best,

[bkeiren]

Hey!

First of all let me apologize because I missed something and it actually does seem to be possible to get Awesomium to handle most of the logic behind copying, pasting, etc. I found this page which to me explains exactly how to implement this. I can look into this later because it is actually fairly easy to implement this (it is similar to how the LoadLis tener, ViewListener, and JSMethodHandler are implemented right now, in case you want to give it a go). Basically, you can register an instance of a (custom) class that implements the InputMethodListener with each WebView, and that will provide you with callbacks for certain events (such as copy/pasting).

Lastly, I can explain why the code you wrote to access Awesomium.DLL doesn't work. The reason is that you cannot directly access that DLL from within your Unity C# scripts. This is why AwesomiumUnity exists in the first place: because it wraps the Awesomium C++ DLL in a layer that C# can talk to.
In addition to this even if you could access the Awesomium DLL directly, you wouldn't be able to access the Copy and Paste methods because those are actually only present in that location in the Awesomium. NET code, which we are not using. Also, the WebView class that you find in the C# code is not the same class as the one that Awesomium provides, it is again just a simple wrapper class that does some bookkeeping to enable us to talk to C++ and access the actual WebView.

I applaud your proactiveness to find out how things work though, very good!

Cheers

[TheMajorO]

Wow! That was amazingly fast :P

Thank you for the link! Ima look into this today and perhaps beat ya to implementing copy and paste. haha.

I'm going to follow the instructions in your answer and the link, and then build AwesomeiumUnity.DLL again(targeting x86 architecture). Then from C# call the new two functions which the modified AwesomiumUnity.DLL will be providing.

I am really having so much fun discussing this with you, solving the issues I am having on my own or using your advice, and using your wrapper. Can't express how delighted I am especially for your quick answers and feedback! You're giving all of this for free so people would build on it, I truly appreciate this.

Will reply with the results tonight, hopefully positive results!

Cheers, all the best!

[TheMajorO]

HALLELUJAH! New DLL is ready :D after much headache lol (from Linker)

New Features: Copy, Paste, Undo, Redo

Was fairly easy to implement them as you said. I just looked how you implemented a void function (namely LoadURL) and made changes accordingly.

Gonna be testing the features today, but DLL is finally successfully built!

I will be looking into doing more stuff later today, like perhaps downloading files :)

Thank you so much man! Can't express enough how happy I am. Will update ya after testing the new exported methods :)

[TheMajorO]

Everything is working perfectly, Cut() has some problem that I will investigate. I will also implement SelectAll().

So far so good! Thanks a lot man!

EDIT: Cut() is working too! 2nd EDIT: Zoom In, Zoom Out, Zoom Reset are all implemented now!

[bkeiren]

Hey!

My apologies for the late reply, but great to hear that you've managed to make such amazing progress! I'm also happy to hear you're having so much fun with it! :)

If you feel like you'd like to share your code (and be credit for it of course) and have it integrated with the rest of the published codebase here on GitHub, submit a pull request with your changes so that I can review it and provide feedback or outright accept it. Once it's integrated all users of AwesomiumUnity will be able to make use of your awesome new features!

I am certain many people would be very thankful for it!

Cheers,

[TheMajorO]

No problem, I understand one can be preoccupied with other stuff they are committed to.

I would be more than glad to add the code, that I wrote to implement the features I needed, to your source so others would make good use of it.

The features are: Copy, Paste, Cut (to system clipboard). Undo, Redo. Zoom-in, Zoom-out (+-20%). Zoom-reset. Bool parameter and default argument in WebCore Initialize(...) to hide webkit default scrollbars.

I have been fixing to do a pull request sometime within this week. Thanks for reminding me and and for the kind words, really appreciate it.

Cheers man!

[bkeiren]

No problem, I'll be anxiously awaiting your pull request :)

[TheMajorO]

Hey @bkeiren , I wish you would take a look at this when you have the time: http://answers.awesomium.com/questions/6708/fetching-favicon-using-unity-and-unity-c-wrapper.html

It's about getting the FAVICON(site icon) to Unity. Maybe I should replicate what's done in Surface.h to do that? I have no idea.

And oh I am not forgetting about the pull request, gonna do it sometime this week :P

[StephenHodgson]

Hey guys, I was doing some work on the new uGUI in Unity 5.1.x with Awesomium and stumbled across your thread here. I've gotten stuck on rendering out new pages once I've clicked on a link. @bkeiren , how are you displaying the AwesomiumUnityWebTexture? Did you use an Image, or raw image component? Whenever I click on a link, the web texture just gets dull gray and doesn't update. Thanks.

[bkeiren]

Hey @HodgsonSDAS,

Initially (pre Unity 5.x) AwesomiumUnity used a Texture2D as the render target. This is still the case with the code that is currently in the repository, however @TheMajorO here has been so kind to implement canvas rendering for Unity 5.x as well (and many other features).

These features have not yet been merged into the regular code base because a pull request has not yet been posted.

However, you mention that you can click on a link and only then does the web texture get dull gray and stops updating? This implies that it does actually render initially. Are you sure you are calling WebView.CopyBufferToTexture whenever there could be new data (preferably every render frame)? Have a look here for an example of how to update the image buffer every frame (uses the old OnGUI system though, but the same principle applies).

[StephenHodgson]

Yes, I'm very much looking forward to @TheMajorO 's pull. I've also been working on this same problem, but I'm guessing we're the only two with this need at the moment. I'd like to also contribute, where I can. Another interesting thing is I've implemented a custom Input Module for the Oculus, and I can get the interactions from it using Unity's EventSystem, but can't get the updated web page to render! haha.

I did utilize that line of code you referenced, and it calls the method on each frame the webView.isDirty flag is changed, but it doesn't seem to update the Texture2D.

[bkeiren]

Is the Web view perhaps crashing? There should be some accessor functions to check this on the AwesomiumUnityWebTexture.

On Tue, 14 Jul 2015 17:40 HodgsonSDAS notifications@github.com wrote:

Yes, I'm very much looking forward to @TheMajorO https://github.com/TheMajorO 's pull. I've also been working on this same problem, but I'm guessing we're the only two with this need at the moment. I'd like to also contribute, where I can. Another interesting thing is I've implemented a custom Input Module for the Oculus, and I can get the interactions from it using Unity's EventSystem, but can't get the updated web page to render! haha.

I did utilize that line of code you referenced, and it calls the method on each frame the webView.isDirty flag is changed, but it doesn't seem to update the Texture2D.

— Reply to this email directly or view it on GitHub https://github.com/bkeiren/AwesomiumUnity/issues/6#issuecomment-121285003 .

[StephenHodgson]

I don't believe so, but I could be mistaken. There isn't any error messages in the console.

[TheMajorO]

Hey @HodgsonSDAS !

First of all, yeah you can use either an image or a raw image. I prefer RawImage since it takes Texture2D, unlike Image which requires creating a sprite form the texture. Here's the difference:

    //This method is in AwesomiumUnityWebTexture.CS
    public void Initialize()
    {
        m_HasBeenInitialized = true;

        AwesomiumUnityWebCore.EnsureInitialized();

        // Call resize which will create a texture and a webview for us if they do not exist yet at this point.
        Resize(m_Width, m_Height);

        var h = GetComponent<Image>();
        var raw = GetComponent<RawImage>();
        if (raw)
        {
            //simple and easy!
            raw.texture = m_Texture;
        }
        else if (h)
        {
            //overhead :)
            var piv = GetComponent<RectTransform>().pivot;
            var myRect = new Rect();//GetComponent<RectTransform>().rect; produces a an error (current bug in unity 5.1.1f1)
            myRect.x = myRect.y = 0;
            myRect.width = m_Texture.width;
            myRect.height = m_Texture.height;
            var img = GetComponent<Image>();
            img.sprite = Sprite.Create(m_Texture, myRect, piv);
        }

        //old code for GUITexture and Plane, by the awesome author of this wrapper :D
        else if (GetComponent<GUITexture>())
        {
            GetComponent<GUITexture>().texture = m_Texture;
        }
        else if (GetComponent<Renderer>() && GetComponent<Renderer>().material)
        {
            GetComponent<Renderer>().material.mainTexture = m_Texture;
            GetComponent<Renderer>().material.mainTextureScale = new Vector2(Mathf.Abs(GetComponent<Renderer>().material.mainTextureScale.x) * (m_FlipX ? -1.0f : 1.0f),
                                                             Mathf.Abs(GetComponent<Renderer>().material.mainTextureScale.y) * (m_FlipY ? -1.0f : 1.0f));
        }
        else
        {
            Debug.LogWarning("There is no Renderer or guiTexture attached to this GameObject! AwesomiumUnityWebTexture will render to a texture but it will not be visible.");
        }

        m_WebView.SetTransparent(m_Transparent);
    }

Secondly, even if you loaded the webpage, you can't click or do anything! You gonna have to replicate the functionality in OnGUI and OnMouseDown and OnMouseUp etc.. in AwesomiumUnityWebTexture.cs using the new event systerm. i.e. UnityEngine.EventSystems and implement the interfaces there. For example, the interfaces I used are the following:

    IPointerEnterHandler,
    IPointerExitHandler,
    IPointerDownHandler,
    IPointerUpHandler,
    IPointerClickHandler,
    IScrollHandler,
    IBeginDragHandler,
    IDragHandler,
    IEndDragHandler

You have to do this in order to inject important events into Awesomium engine, see Bkeiren answer here: https://github.com/bkeiren/AwesomiumUnity/issues/6#issuecomment-118021892

Also, you have to delve into AwesomiumUnityWebTexture.cs and AwesomiumUnityWebView.cs. Heck you might even modify the C++ files and rebuild AwesomiumUnity.DLL for extra functionality! But don't worry it's all possible and not that hard to perform.

If you are having problems with the interfaces UnityEngine.EventSystems, lemme know so I give ya a useful script for handling multiple delegates.

[StephenHodgson]

@TheMajorO , Yes sir, I did implement the interfaces! I have those working fine, but can't seem to get the webView to render properly. Let me compare your code snippet above and see if there's something I've done wrong.

[TheMajorO]

Also, you gonna have to do this in some Update method:

var web = GetComponent<AwesomiumUnityWebTexture>();
web.WebView.CopyBufferToTexture(web.WebTexture.GetNativeTexturePtr(), web.WebTexture.width, web.WebTexture.height);

[StephenHodgson]

Yup, I did call something similar in the update method:

public void Initialize()
{
    m_HasBeenInitialized = true;

    AwesomiumUnityWebCore.EnsureInitialized();

    // Call resize which will create a texture and a webview for us if they do not exist yet at this point.
    Resize( m_Width, m_Height );

    var _image = GetComponent<RawImage>();

    if( _image )
    {
        _image.texture = m_Texture;
    }

    m_WebView.SetTransparent( m_Transparent );
}

public void OnPointerUp( PointerEventData eventData )
{
    if( !m_Interactive )
        return;
    if( m_MouseIsOver )
    {
        m_WebView.InjectMouseDown( 0 );//Left Mouse click only
        Debug.Log( "WebTexture: OnPointerUp()" );
    }
}
void Update()
{
    if( m_WebView != null )
    {
        if( m_MouseIsOver )
        {
            Vector2 _mosPos;// Event.current.mousePosition;
            Vector2 lookPosition;
            lookPosition.x = Screen.width / 2;
            lookPosition.y = Screen.height / 2;
            RectTransform _rectTrans = GetComponent<RectTransform>();
            RectTransformUtility.ScreenPointToLocalPointInRectangle( _rectTrans, lookPosition, Camera.main, out _mosPos );
            m_WebView.InjectMouseMove( ( int )_mosPos.x, ( int )_mosPos.y );
            //Debug.Log( "WebTexture: Mouse position: " + _mosPos.x + ", " + _mosPos.y );
        }

        if( m_WebView.IsDirty )
        {

            //Debug.Log( "WebView is Dirty... Redrawing" );
            if( m_Texture == null )
                Debug.LogError( "The WebTexture does not have a texture assigned and will not paint." );
            else
                m_WebView.CopyBufferToTexture( m_Texture.GetNativeTexturePtr(), m_Texture.width, m_Texture.height );
        }

        if( m_WebView.IsLoading )
            Debug.Log( "...Loading new page..." );
    }
}

I can't help but think it's just selecting and highlighting the whole page, but I don't know how to be sure, lol

[bkeiren]

Your log appears to indicate that the native pointer for the WebView is being cleared. If this happens you no longer have any way to access the underlying data of the WebView, which could easily explain this issue.

Could you try to find out why the native pointer is being cleared?

[bkeiren]

Oh and I agree that it looks like it's simply selecting some stuff on the page. Are you sure that isn't just what's happening? If you're not passing all of your input to the WebView it will not be able to properly react to it. As a result the input it thinks it is being given may result in this kind of behaviour. To verify, if you call WebView.Focus, do the gray areas turn blue?

[StephenHodgson]

@bkeiren the webView native pointer is only cleared when I stop the application.

[bkeiren]

Oh I see, good. In that case could you please try the focus thing I mentioned in my previous post and let us know how that worked out?

[StephenHodgson]

Yes I will. Btw, I've updated the above code snippet with my OnPointerUp that's driving the mouse click.

When should I be calling Focus & UnFocus?

Edit: Also, yes it did turn blue when I gave it focus.

Hmm, maybe I should just wait for @TheMajorO to fork XD It seems I have some work to do on my Input Handlers. Thanks to both of your for your help.

[TheMajorO]

@HodgsonSDAS @bkeiren I am terribly sorry that I haven't done a pull request already. I am mainly focusing on my work that is sucking the life outta me xD (and at the same time getting terribly underpaid compared to other countries, unfortunately, even though everybody here just works on Web Dev and you rarely find somebody who devoted his life to Unity lol).

However, since another person is going through the same I went through, then I will make sure to do the pull request as soon as possible; that means in the next 6 hours! promise.

@HodgsonSDAS you gonna have to set a private flag in OnPointerEnter and reset that flag in OnPointerExit; consequently indicating if you are hovering over the web window or not. Also, in OnPointerEnter you have to call Focus(). Finally, in Update() you gonna have to do the following:

    void Update()
    {
        #region Unfocus?
        if (Input.GetMouseButtonDown(0) || Input.GetMouseButtonDown(1) || Input.GetMouseButtonDown(2))
        {
            if (!IsMouseOver)
            {
                Unfocus();
            }
        }
        #endregion

        #region repaint
        if (!web.WebTexture)
            Debug.LogError("The WebTexture does not have a texture assigned and will not paint.");
        else
            web.WebView.CopyBufferToTexture(web.WebTexture.GetNativeTexturePtr(), web.WebTexture.width, web.WebTexture.height);
        #endregion

        #region mouseHover_Awesomium
        if (IsMouseOver)
        {
            if (web.m_Interactive && web && raw || web.m_Interactive && web && img)
            {
                Vector2 localCursor;
                var rect1 = GetComponent<RectTransform>();
                var pos1 = Input.mousePosition;
                if (!RectTransformUtility.ScreenPointToLocalPointInRectangle(rect1, pos1,
                    null, out localCursor))
                    Debug.LogException(new System.Exception("can't do ScreenPointToLocalPointInRectangle!"));

                xMousePos = localCursor.x;
                yMousePos = localCursor.y;

                //some math that magically worked after trial and error!
                if (xMousePos < 0)
                    xMousePos = xMousePos + rect1.rect.width / 2;
                else xMousePos += rect1.rect.width / 2;

                if (yMousePos > 0)
                    yMousePos = yMousePos + rect1.rect.height / 2;
                else yMousePos += rect1.rect.height / 2;

                //flipping Y axis to make Awesomium happy
                yMousePos = rect1.rect.height - yMousePos;
                //Debug.Log(ypos);

                //Debug.Log("Correct Cursor Pos: " + xpos + " " + ypos);
                web.WebView.InjectMouseMove((int)xMousePos, (int)yMousePos);
            }
            else Debug.LogError("WebTexture or RawImage missing! (Perhaps browser is non-interactive?");
        }
        #endregion
}

[StephenHodgson]

@TheMajorO I really, really appreciate it. I'm not too pressed on time, so please, do at your own convenience. On the bright side, this is work for me, but down side is I'm a one man team. It's nice to be able to work with others when facing a problem like this.

[TheMajorO]

@HodgsonSDAS I will assemble a working mini example today and attach it to the pull request, with all the fancy stuff that I took from awesome people like @bkeiren or stuff I figured out on my own. You can then compare your code with the example and resolve any present issues. You don't have to go figure out you way through the nonsense I have been going through in the last 2 weeks for 6 hours every day xD Just like how @bkeiren helped with his great wrapper that cut many corners for us :)

All the best man!

[TheMajorO]

And yeah I can feel ya man. I'm a one man team in this work I am doing as well... I am almost done with functionality and now doing animation. Also I'm the UI editing guy lol It's sad the guy they hired for design from overseas is getting x10 more than me, and I am even doing some design work he's supposed to do lol But hey it's all experience, and a chance to interact with good people like you guys :) Heck I am amused how much work I achieved in the last two weeks! That wouldn't happen without guiding from @bkeiren :D

[StephenHodgson]

@TheMajorO , what country do you live in?

[StephenHodgson]

Also, I figured out another thing, The RectTransform needs to have it's pivot at 0,1 in the upper left hand corner. That was my problem the whole time XD. I've got everything working pretty well now.

[TheMajorO]

You might be surprised, but I live in Syria xD I'm not in desperate need for the money, that's not the reason why I accepted the job. It's just that I want to do something ya know, and gain some experience. Nobody here understands or appreciates what Game Dev is... So it's merely me and my designer friend head to head as an Indie team. But when I saw this job I wanted to have it on my CV since it somehow related to my experience in Unity and C# lol Though at the same time, if it's not something collaborative then I want something in return that lives up to the quality of the hard work I am putting! lol

Concerning Pivots, the uGUI is terribly annoying in regards to that, but I got used to it. In fact what I did is resetting the RectTransform to the middle and keeping the pivots to 0.5,0.5

Hold ALT key and choose the middle element.

I suggest you check out official Live Training Archive videos to get used to this new UGUI that is causing us to go through pleasure and pain! Go to this link https://www.youtube.com/playlist?list=PLX2vGYjWbI0QvLHla7C_Z_s3q1Oi461o4 and hit Ctrl-F and type UI lol There are 4 videos on the new UI.

All the best!

[TheMajorO]

Oh and what about you? where do you live? if you don't mind sharing that :P

@bkeiren sorry for turning this into a chat room xD

[StephenHodgson]

@TheMajorO , Florida, USA. Yeah I'm familiar with uGUIs Pivot and Anchor functions. I've had to go under the hood of that thing, and it's scary. lol. Because I'm working with the Oculus I needed to get the rects local point from the center of my screen where I was raycasting my "clicks". I ended up having to invert my mouse Y position. This was my final code:

public void Initialize()
{
    m_HasBeenInitialized = true;

    AwesomiumUnityWebCore.EnsureInitialized();

    var _image = GetComponent<RawImage>();
    Rect _rect;

    if( _image != null )
    {
        _rect = _image.rectTransform.rect;
        m_Height = (int)_rect.height;
        m_Width = (int)_rect.width;

        // Call resize which will create a texture and a webview for us if they do not exist yet at this point.
        Resize( m_Width, m_Height );
    }
    else
    {
        Resize( m_Width, m_Height );
    }

    if( _image )
    {
        _image.texture = m_Texture;
    }

    m_WebView.SetTransparent( m_Transparent );
}
void Update()
{
    if( m_WebView != null )
    {
        if( m_WebView.IsDirty )
        {
            if( m_Texture == null )
                Debug.LogError( "The WebTexture does not have a texture assigned and will not paint." );
            else
                m_WebView.CopyBufferToTexture( m_Texture.GetNativeTexturePtr(), m_Texture.width, m_Texture.height );
        }

        if( m_WebView.IsLoading )
            Debug.Log( "...Loading new page..." );

        if( m_MouseIsOver )
        {
            Focus();
        }
        else
        {
            Unfocus();
        }
    }
}

public void UpdateMousePos( PointerEventData eventData )
{
    Vector2 _mosPos;// Event.current.mousePosition;
    Vector2 lookPosition;
    lookPosition.x = Screen.width / 2;
    lookPosition.y = Screen.height / 2;
    RectTransform _rectTrans = GetComponent<RectTransform>();
    RectTransformUtility.ScreenPointToLocalPointInRectangle( _rectTrans, lookPosition, eventData.pressEventCamera, out _mosPos );
    m_WebView.InjectMouseMove( ( int )_mosPos.x, -( int )_mosPos.y );
    Debug.Log( "WebTexture: Mouse position3: " + _mosPos.x + ", " + -_mosPos.y );
}

public void OnPointerEnter( PointerEventData eventData )
{       
    m_MouseIsOver = true;
    if( !m_Interactive )
        return;
}

public void OnPointerExit( PointerEventData eventData )
{
    m_MouseIsOver = false;
    if( !m_Interactive )
        return;
}

public void OnPointerDown( PointerEventData eventData )
{
    if( !m_Interactive )
        return;
    if( m_MouseIsOver )
    {
        UpdateMousePos( eventData );
        m_WebView.InjectMouseDown( MapMouseButtons( 0 ) );//Left Mouse click only
    }
}

public void OnPointerUp( PointerEventData eventData )
{
    if( !m_Interactive )
        return;
    if( m_MouseIsOver )
    {
        UpdateMousePos( eventData );
        m_WebView.InjectMouseUp( MapMouseButtons( 0 ) );//Left Mouse click only
    }
}

[TheMajorO]

That's the exact same thing I did in _#region mouseHoverAwesomium; ScreenPointToLocalPointInRectangle, some math, and finally inverted Y position xD

[TheMajorO]

@bkeiren , @HodgsonSDAS hey guys. Sorry for the delay! when @HodgsonSDAS said he wasn't pressed on time I took my time working on the code lol

Just wanna give you an update. Yesterday I stayed up all night and finished generifying and refactoring the new AwesomiumUnityWebTextureUGUI, I will upload it today when I get back home along with the other changes in other scripts and the modified wrapper C++ files (but not the DLL, in case @bkeiren wanna revise the code and rebuild the DLL himself).

@bkeiren I was able to register held keys finally! all other features I mentioned before are successfully implemented as well.

Cheers :D :coffee:

[bkeiren]

Hey @TheMajorO, do you perhaps have an update on your work? I'd love to hear from you again!

[TheMajorO]

Hey @bkeiren! Sorry for the delay man, I figured submitting a pull request would be easy. To be honest this is the first time I actually use GitHub and I am feeling ashamed of myself!

I didn't even have a repo to begin with. So I decided to do it the right way and started learning Git using a Lynda tutorial and so far I can do simple stuff from Git Bash command line (using my custom Vim editor :D ) and God it feels good!

I really have no idea how I was living without a SVC.. I used to just export everything and upload it to my cloud lol But now I am ready to depart for GitHub!

I did finish the code when I posted my last comment as I promised, but when I got back home later that day I discovered how complex git is to a beginner like me, and had really hard time uploading my work to yours the right way like how everybody else is doing it.

[bkeiren]

If you're a beginning Git user I would advise you to download GitHub's own Windows client and use that in combination with the buttons on the GitHub website to clone, fork, and branch repos, and to commit changes to your repositories. Made it much easier for me at least.

Also, once you have your own clone of the AwesomiumUnity repo and your changes comitted to it, you can very easily submit a pull request for it through the GitHub website using this button:

Looking forward to your pull request, and let me know if you need any help!

[TheMajorO]

Thanks for the explanation @bkeiren! Hopefully by following your instructions it will be a trivial matter to submit a pull request. I will be working on that today.

[TheMajorO]

Done :) It's actually pretty easy when using the GUI client, but I'm still gonna give Git Bash a try and continue learning it :v

Please feel free to edit anything you want, after all you're the repo master lol

Also, I'm downloading FAVICON from each website I am visiting using a JS method to fetch the link and having troubles converting some FAVICONs that are not in .PNG format (like Google's) from .ICO to .PNG using .NET API. The solution I found is not portable (windows-specific) so I was wondering if there is another way to do it in the wrapper? Here's the question I asked regarding this matter: StackOverflow Unity3D Answers

Thanks for everything man! And for writing the wrapper and giving us the chance to use it and contribute to it :)

[TheMajorO]

Oh one more thing, some changes merely originate from Auto Format Document (my bad lol). Also I haven't changed anything significant with AwesomiumUnityWebTexture.CS (references are substituted with GetCompnent<> cuz I was experimenting with stuff) so please don't accept those changes.

Things could have been more professional; I try but this is my first commit & pull request. xP

[TheMajorO]

Hey @bkeiren How have ya been doing?

Apparently detecting key presses using update (Input.GetKeyDown etc..) or deprecated GUI system input events (such as OnButtonDown) are not a good idea since uGUI was shipped with a new Event System (that I partly used in my Pull Request code for advanced custom mouse input).

So please keep my current pull request on hold while I implement keyboard input events using the new Event System, which will probably solve the problems we have now in Key Repeat.

All the best! I see you're working on a linux-mac-os Branch :) would love to provide anything I can do!

[bkeiren]

Hey @TheMajorO ,

I'll hold off on reviewing your pull request until you're done. If you don't mind could you also please fix the code that you mentioned you wanted me to ignore in the pull request? I don't think I can selectively accept or ignore individual changes from a pull request, rather I have to accept it entirely or not at all (otherwise I'd have to get your changes and make my own modifications, but then the repo history won't show it as your pull request being merged but rather as my own custom changes being comitted).

And yes, I am working on a Linux and Mac port. I currently have Mac working but there are certain requirements to get to that point that make it unsuited for actual release yet. It needs more work. If you want to help out with it I'll see what we can arrange :)

Cheers,

[StephenHodgson]

Any luck on the keyboard input events @TheMajorO ?

[TheMajorO]

Hey @HodgsonSDAS ! Hope things are going well with OR :P

I'm still poking around here and there... Unity Answers, digging in uGUI's source here on Github, etc... But currently I'm working on other stuff and fixing to getting back to this later. For now keyboard input in my pull request works perfectly unless you are in need for "key repeat" and "alt keys" (i.e. Shift-/ will produce ?). I mean it's working for testing but I'm not satisfied about it so definitely not leaving it as is forever.

Heck why not Ima give it another try tonight (it's nighttime here now lol) and report back with hopefully a positive result!