search

hotwired / turbo-android

Android framework for making Turbo native apps
MIT License
423 stars 51 forks source link

Custom User Agent is only set upon initial load #191

Closed queenannalisek closed 6 months ago

queenannalisek commented 2 years ago

Hello!

I've found that the following code in MainSessionNavHostFragment.kt only sets the user agent upon the initial opening/loading of the app. On subsequent page visits, it falls back to the default WebView user agent string:

override fun onSessionCreated() {
        super.onSessionCreated()
        session.webView.settings.userAgentString = customUserAgent(session.webView)
    }

    private fun customUserAgent(webView: WebView): String {
        return "Turbo Native Android ${webView.settings.userAgentString}"
    }

I need to have Turbo Native Android be present in every request done within the Android app.

Is there somewhere else that I can set it?

Alternatively, I've been trying (and failing) to access the main session object within the MainActivity class. That way I can attempt to set the user agent in the onCreate method, which is where StackOverflow tells me it should be set.

Any tips on how I can do this?

The custom user agent is always set in my turbo-ios app, which is great!

Thank you very much for your time!

jayohms commented 2 years ago

I'm not able to replicate this issue. In the demo app, the custom user agent is set on every WebView request. Please provide a code sample to demonstrate the issue in your app. Are you able to replicate this using only the demo app code?

You should not set the user agent through the MainActivity. onSessionCreated() is the proper place to update the user agent, because it's called immediately after the TurboSession and its WebView object have been created. If the session is recreated due to the app lifecycle (or the WebView render process is gone), onSessionCreated() will be called again.

queenannalisek commented 2 years ago

Hi @jayohms! Thank you very much for the quick response.

While I investigate a little further, I'll give you more detail on my setup and what I'm seeing.

I'm using turbo-android v7.0.0-rc1 and my Android app is connected to a Rails 6.1 app.

The Android app was set up by following the 6-step Documentation section of this repo.

The code in MainSessionNavHostFragment.kt is as follows:

import android.webkit.WebView
import androidx.appcompat.app.AppCompatActivity
import androidx.fragment.app.Fragment
import dev.hotwire.turbo.config.TurboPathConfiguration
import dev.hotwire.turbo.session.TurboSessionNavHostFragment
import kotlin.reflect.KClass

class MainSessionNavHostFragment : TurboSessionNavHostFragment() {
    override val sessionName = "main"

    override val startLocation = <app_url>

    override val registeredActivities: List<KClass<out AppCompatActivity>>
        get() = listOf(
            // Leave empty unless you have more
            // than one TurboActivity in your app
        )

    override val registeredFragments: List<KClass<out Fragment>>
        get() = listOf(
            WebFragment::class
            // And any other TurboFragments in your app
        )

    override val pathConfigurationLocation: TurboPathConfiguration.Location
        get() = TurboPathConfiguration.Location(
            assetFilePath = "json/configuration.json",
//            remoteFileUrl = "https://turbo.hotwired.dev/demo/configurations/android-v1.json"
        )

    override fun onSessionCreated() {
        super.onSessionCreated()
        session.webView.settings.userAgentString = customUserAgent(session.webView)
        session.webView.settings.javaScriptEnabled = true
        session.webView.addJavascriptInterface(WebAppInterface(session.webView.context), "Android")
    }

    private fun customUserAgent(webView: WebView): String {
        return "Turbo Native Android ${webView.settings.userAgentString}"
    }
}

In my Rails app, I have two before_action methods in the application_controller that check whether the user is in one of our Turbo apps and then whether they are in the Android app specifically (as we need to adapt our views conditionally based on this).

To check if the user is in a Turbo app, I have a wrapper around the turbo-rails built-in turbo_native_app? method.

To check if the user is in our Turbo Android app, I have an additional check for Android appearing alongside Turbo Native in request.user_agent.to_s.

Right now, in these methods, for debugging purposes, I print out request.user_agent.to_s.

When I first launch the app in the Android Studio emulator, the user agent string is as follows:

Turbo Native Android Mozilla/5.0 (Linux; Android 10; Android SDK built for x86 Build/QSR1.190920.001; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/93.0.4577.82 Mobile Safari/537.36

For any subsequent visit within the app (via the WebView that is presenting my Rails web app), the user agent string becomes:

Mozilla/5.0 (Linux; Android 10; Android SDK built for x86 Build/QSR1.190920.001; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/93.0.4577.82 Mobile Safari/537.36

The 'Turbo Native Android' prefix has disappeared!

Happy to share any other relevant parts of the codebase.

Happy to try this with the demo app code, though what's the best way for me to test that the user agent string is indeed set each time?

Thank you so much for your time!

queenannalisek commented 2 years ago

And one further test!

When I change:

session.webView.settings.userAgentString = customUserAgent(session.webView)

to:

session.webView.settings.userAgentString = "Turbo Native Android"

The first time the app loads, the user agent string, logged by Rails is:

Turbo Native Android

Subsequent visits and the user agent is logged as:

Mozilla/5.0 (Linux; Android 10; Android SDK built for x86 Build/QSR1.190920.001; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/93.0.4577.82 Mobile Safari/537.36
queenannalisek commented 2 years ago

Another observation:

While the custom user agent doesn't stick when accessing it via request.user_agent.to_s, it seems Turbo Native Android is maintained when accessing it via Javascript with navigator.userAgent.

Ideally, I'd be able to access the custom user agent reliably outside of JS, within the controller.

Thank you for any time you spend on looking into this!

jayohms commented 2 years ago

This is very odd. I still don't know what is happening. It sounds like your app is not properly navigating within the TurboWebView instance 🤔.

I tested this out in the demo app with the demo server and you can see the user-agent looks fine for all requests:

Server is listening on port 45678
Fri Oct 01 2021 15:57:52 GMT-0400 (Eastern Daylight Time) -- GET /
Fri Oct 01 2021 15:57:52 GMT-0400 (Eastern Daylight Time) -- Turbo Native Android Mozilla/5.0 (Linux; Android 12; Pixel 3 Build/SPB5.210812.002; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/94.0.4606.61 Mobile Safari/537.36
Fri Oct 01 2021 15:57:52 GMT-0400 (Eastern Daylight Time) -- GET /favicon.ico
Fri Oct 01 2021 15:57:52 GMT-0400 (Eastern Daylight Time) -- Turbo Native Android Mozilla/5.0 (Linux; Android 12; Pixel 3 Build/SPB5.210812.002; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/94.0.4606.61 Mobile Safari/537.36
Fri Oct 01 2021 15:57:58 GMT-0400 (Eastern Daylight Time) -- GET /slow
Fri Oct 01 2021 15:57:58 GMT-0400 (Eastern Daylight Time) -- Turbo Native Android Mozilla/5.0 (Linux; Android 12; Pixel 3 Build/SPB5.210812.002; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/94.0.4606.61 Mobile Safari/537.36
Fri Oct 01 2021 15:58:04 GMT-0400 (Eastern Daylight Time) -- GET /one
Fri Oct 01 2021 15:58:04 GMT-0400 (Eastern Daylight Time) -- Turbo Native Android Mozilla/5.0 (Linux; Android 12; Pixel 3 Build/SPB5.210812.002; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/94.0.4606.61 Mobile Safari/537.36
Fri Oct 01 2021 15:58:08 GMT-0400 (Eastern Daylight Time) -- GET /long
Fri Oct 01 2021 15:58:08 GMT-0400 (Eastern Daylight Time) -- Turbo Native Android Mozilla/5.0 (Linux; Android 12; Pixel 3 Build/SPB5.210812.002; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/94.0.4606.61 Mobile Safari/537.36
Fri Oct 01 2021 15:58:11 GMT-0400 (Eastern Daylight Time) -- GET /nonexistent
Fri Oct 01 2021 15:58:11 GMT-0400 (Eastern Daylight Time) -- Turbo Native Android Mozilla/5.0 (Linux; Android 12; Pixel 3 Build/SPB5.210812.002; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/94.0.4606.61 Mobile Safari/537.36
Fri Oct 01 2021 15:58:18 GMT-0400 (Eastern Daylight Time) -- GET /protected
Fri Oct 01 2021 15:58:18 GMT-0400 (Eastern Daylight Time) -- Turbo Native Android Mozilla/5.0 (Linux; Android 12; Pixel 3 Build/SPB5.210812.002; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/94.0.4606.61 Mobile Safari/537.36
Fri Oct 01 2021 15:58:18 GMT-0400 (Eastern Daylight Time) -- GET /signin
Fri Oct 01 2021 15:58:18 GMT-0400 (Eastern Daylight Time) -- Turbo Native Android Mozilla/5.0 (Linux; Android 12; Pixel 3 Build/SPB5.210812.002; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/94.0.4606.61 Mobile Safari/537.36
Fri Oct 01 2021 15:58:22 GMT-0400 (Eastern Daylight Time) -- POST /signin
Fri Oct 01 2021 15:58:22 GMT-0400 (Eastern Daylight Time) -- Turbo Native Android Mozilla/5.0 (Linux; Android 12; Pixel 3 Build/SPB5.210812.002; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/94.0.4606.61 Mobile Safari/537.36
Fri Oct 01 2021 15:58:22 GMT-0400 (Eastern Daylight Time) -- GET /
Fri Oct 01 2021 15:58:22 GMT-0400 (Eastern Daylight Time) -- Turbo Native Android Mozilla/5.0 (Linux; Android 12; Pixel 3 Build/SPB5.210812.002; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/94.0.4606.61 Mobile Safari/537.36
Fri Oct 01 2021 15:58:22 GMT-0400 (Eastern Daylight Time) -- GET /
Fri Oct 01 2021 15:58:22 GMT-0400 (Eastern Daylight Time) -- Turbo Native Android Mozilla/5.0 (Linux; Android 12; Pixel 3 Build/SPB5.210812.002; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/94.0.4606.61 Mobile Safari/537.36
Fri Oct 01 2021 15:58:29 GMT-0400 (Eastern Daylight Time) -- GET /new
Fri Oct 01 2021 15:58:29 GMT-0400 (Eastern Daylight Time) -- Turbo Native Android Mozilla/5.0 (Linux; Android 12; Pixel 3 Build/SPB5.210812.002; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/94.0.4606.61 Mobile Safari/537.36
Fri Oct 01 2021 15:58:30 GMT-0400 (Eastern Daylight Time) -- POST /new
Fri Oct 01 2021 15:58:30 GMT-0400 (Eastern Daylight Time) -- Turbo Native Android Mozilla/5.0 (Linux; Android 12; Pixel 3 Build/SPB5.210812.002; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/94.0.4606.61 Mobile Safari/537.36
Fri Oct 01 2021 15:58:30 GMT-0400 (Eastern Daylight Time) -- GET /success
Fri Oct 01 2021 15:58:30 GMT-0400 (Eastern Daylight Time) -- Turbo Native Android Mozilla/5.0 (Linux; Android 12; Pixel 3 Build/SPB5.210812.002; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/94.0.4606.61 Mobile Safari/537.36
Fri Oct 01 2021 15:58:32 GMT-0400 (Eastern Daylight Time) -- GET /
Fri Oct 01 2021 15:58:32 GMT-0400 (Eastern Daylight Time) -- Turbo Native Android Mozilla/5.0 (Linux; Android 12; Pixel 3 Build/SPB5.210812.002; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/94.0.4606.61 Mobile Safari/537.36
queenannalisek commented 2 years ago

Okay, I'll dig into my setup some more! Thank you so much for your time, Jay! :-)

jayohms commented 2 years ago

Sure thing. I'd recommend pointing your Android app code to the local demo server to isolate the issue: https://github.com/hotwired/turbo-native-demo

You can modify the server.js file to log the user-agent on each request, like i did above:

--- a/server.js
+++ b/server.js
@@ -46,6 +46,7 @@ app.use((request, response, next) => {

 app.use((request, response, next) => {
   console.log(`${Date()} -- ${request.method} ${request.path}`)
+  console.log(`${Date()} -- ${request.get('user-agent')}`)
   next()
 })
queenannalisek commented 2 years ago

For whatever reason, I think I'm running into what's detailed over here:

Screenshot 2021-10-03 at 16 40 41

https://developer.chrome.com/docs/multidevice/webview/

I guess the app launching is a request sent by the Webview itself and subsequent ones are XMLHttpRequests via Turbo?

This would explain why the turbo_native_app? method works for turbo-ios but not Android, but why we can still access the customised user agent via JS.

Going forward, I'll access it via navigator.userAgent or by looking for a combo of Android and wv in the user agent string:

Screenshot 2021-10-03 at 16 44 05

https://developer.chrome.com/docs/multidevice/user-agent/

Thank you for the time you spent looking into this, Jay!

ghiculescu commented 2 years ago

@queenannalisek What version of Android are you testing in? For what it's worth, on a very newly built turbo-android app I'm not able to replicate your issue.

I'm setting the user agent like this

    session.webView.settings.userAgentString = customUserAgent(session.webView)

    // ...

    private fun customUserAgent(webView: WebView): String {
        return listOf(
            "our app name",
            "version: ${BuildConfig.VERSION_NAME}",
            "baseUA: ${webView.settings.userAgentString}"
        ).joinToString(separator = " | ")
    }

And on the server side the UA is logged as

our app name | version: 2.0 | baseUA: Mozilla/5.0 (Linux; Android 11; sdk_gphone_x86 Build/RSR1.201013.001; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/83.0.4103.106 Mobile Safari/537.36

It shows like that on multiple requests (GET and POST, including XMLHttpRequests sent by Turbo and by Stimulus).

The code I've shared seems similar to the code you've shared, so that's probably not the issue.

Just as a wild guess, have you overridden shouldNavigateTo by any chance?

henrik commented 1 year ago

I have not read the full discussion yet, but we're seeing an issue that seems related.

When we pull-to-refresh on a certain page of our app that contains an iframe embedding an external service, Turbo somehow loses our custom user agent.

We have indeed overridden shouldNavigateTo in order to handle external URLs and non-HTTP protocols ("bankid://").

henrik commented 1 year ago

A bit more detail of what we know so far.

We used to only override the user agent in OurSubclassOfTurboSessionNavHostFragment:

override fun onSessionCreated() {
        super.onSessionCreated()
        session.webView.settings.userAgentString = UserAgent(session.webView).string()
}

But then we noticed an issue. When we pull-to-refresh on a certain page containing an iframe for a payment provider (but not if we pull-to-refresh on other pages), our customisation is lost.

So then we tried also setting the user agent in OurSubclassOfTurboWebFragment:

override fun onVisitStarted(location: String) {
        super.onVisitStarted(location)
        session.webView.settings.userAgentString = UserAgent(session.webView).string()
}

This fixed that issue, but it over-applies our customisation since it's added on every visit start. So we can end up with "Original user agent | Customisation | Customisation | Customisation" and so on.

Then, we tried making it so that it only adds the customisation if not already present:

    fun string(): String {
        val oldAgent = webView.settings.userAgentString

        return if (oldAgent.contains("Customisation")) {
             oldAgent
        } else {
             "$oldAgent | Customisation"
        }
    }

This makes our customisation no longer be over-applied. But it also reintroduces the pull-to-refresh bug.

When we look in a Web Inspector, we can see that the initial load of the page-with-iframe happens over XHR and contains the customisation. When we pull-to-refresh, it's a non-XHR request and it no longer has our customisation.

But again, on other pages, it keeps the customisation when we pull-to-refresh.

It is as though the contains check looks in one place to see if it's present, preventing it from adding it in some other place. I suppose these places could be the XHR user agent vs the WebView user agent.

So perhaps our logic ends up being:

FWIW we use Turbolinks on the web site (and Turbo in the app), but I'm not sure the web has much bearing on this issue.

queenannalisek commented 1 year ago

@ghiculescu Sorry it took me a ridiculous amount of time to get back to this!

I'm now testing on Android 13 and the problem still persists, whether I set the custom user agent like this:

 private fun customUserAgent(webView: WebView): String {
        return "Turbo Native Android ${webView.settings.userAgentString}"
    }

Or how you suggest:

  private fun customUserAgent(webView: WebView): String {
        return listOf(
            "our app name",
            "version: ${BuildConfig.VERSION_NAME}",
            "baseUA: ${webView.settings.userAgentString}"
        ).joinToString(separator = " | ")
    }

In the meantime, looking for Android combined with wv has served us well!

Thank you for your extra notes on this issue, @henrik!

jean-francois-labbe commented 1 year ago

@queenannalisek Do you have a serviceWorker in your application ? If it is configured to serve from cache and fallback to fetch from the application server. Then you'll observe this behavior. First request will have your custom User Agent (cache miss), then following requests will have your browser User Agent.

As I've understood. This is because the service worker is running for all your browser tabs, not just your current tab. Then the serviceWorker User Agent cannot be customized and will always be your browser default User Agent. Except if you start your browser with a custom User Agent from the CLI (I don't know if it's possible). Here is a bug report that will provide more insights. https://bugs.chromium.org/p/chromium/issues/detail?id=595993

queenannalisek commented 1 year ago

Hi @jean-francois-labbe! I think that's it! We do have a service worker and it looks like it's set up to behave that way! Thank you!

henrik commented 1 year ago

I'm afraid we don't seem to have any service workers – Chrome's web inspector doesn't show any when loading the page we're seeing this issue on, and nothing in our Android code. So it doesn't seem to explain our issue.