In this article, we will learn about autofill methods for input fields/forms in WebView used inside of an android app. I’d like to stress that the main topic is the autofill in WebView, because, when filling standard EditText views in the app - no issues arise. But when we display content owned by other parties, we can’t fill the form with our data in a simple way.
Okay, in our case we have a native Android app that stores some user data, such as name/surname/date of birth, etc. One of the app screens is WebView opening a website with a domain of a different company. When a user taps on an input field, we want to insert (or suggest inserting) the data already stored in the app. An example would be an app for a discount and coupon provider that allows users to buy directly from the app (opens WebView with the service’s/seller’s page like Amazon, Nike, etc).
Autofill framework based approach
We can use a standard Autofill framework used by almost all password managers. It’s flexible and will work both within our app and with other apps, browsers and webviews to which you want to provide your data (for example, promo codes). However, it works since Android 8.0 only (API level 26).
Let’s try to configure our app so that it works with the Autofill framework. Create an empty class DiscountsAutofillService and declare it as a service in AndroidManifest.xml:
<service
android:name=".DiscountsAutofillService"
android:label="Discounts Autofill Service"
android:permission="android.permission.BIND_AUTOFILL_SERVICE">
<intent-filter>
<action android:name="android.service.autofill.AutofillService" />
</intent-filter>
</service>Permission android:permission="android.permission.BIND_AUTOFILL_SERVICE" and intent-filter ensure that our app will be displayed in the Autofill service system list. Add a button in the app (by simply placing it on the main screen) that will be displayed if the app is not selected as the main Autofill service. When the button is clicked, the settings screen will open.
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
val afm = getSystemService(AutofillManager::class.java)
binding.enableAutofillButton.isVisible =
!afm.hasEnabledAutofillServices() && afm.isAutofillSupported
binding.enableAutofillButton.setOnClickListener {
startActivity(
Intent(
Settings.ACTION_REQUEST_SET_AUTOFILL_SERVICE,
Uri.parse("package:$packageName")
)
)
}
}Now, click enableAutofillButton, and the settings screen will open. Here is our Discounts Autofill Service.
Do not activate it right now because it’s just a dummy.
Fill DiscountsAutofillService with code so it executes what it is intended to do, namely insert promo codes (or any other data) into the WebView of our app.
class DiscountsAutofillService : AutofillService() {
override fun onFillRequest(
request: FillRequest,
cancellationSignal: CancellationSignal,
callback: FillCallback
) {}
override fun onSaveRequest(request: SaveRequest, callback: SaveCallback) {}
}There, we need to implement two methods – onFillRequest and onSaveRequest – for the autofill to work. The first one is called when the system informs the app that the user clicked some text field and we need to suggest autofill options if there is something to suggest for this website/field. Take note of the first and the last arguments of this method: FillRequest contains information about the hierarchy of the views and their attributes, so we can bypass the whole hierarchy, find the views where we can insert something and return the result (Dataset) with the data that we want to fill these views with. FillCallback is the method used to return this result.
Another method onSaveRequest is called when we switch to the next screen or leave the current screen. It allows us to save the data entered by the user. The method works with many data types, such as logins, passwords, addresses, and bank card data. Though when we, for example, pre-fill promo code data, we don’t need to save promo codes because we already store them in the app. This is why right now, we are interested in onFillRequest only.
override fun onFillRequest(
request: FillRequest,
cancellationSignal: CancellationSignal,
callback: FillCallback
) {
val context: List<FillContext> = request.fillContexts
val structure: AssistStructure = context[context.size - 1].structure
val autofillMap = mutableMapOf<Field, AutofillId>()
parseStructure(structure, autofillMap)
if (autofillMap.containsKey(Field.PROMOCODE)) {
val promocodes = getPromocodes()
val fillResponse: FillResponse = FillResponse.Builder()
.apply {
promocodes.forEach { promocode ->
val promocodePresentation = RemoteViews(packageName, android.R.layout.simple_list_item_1).apply {
setTextViewText(android.R.id.text1, "Promocode $promocode")
}
addDataset(
Dataset.Builder()
.setValue(
autofillMap[Field.PROMOCODE]!!,
AutofillValue.forText(promocode),
promocodePresentation
)
.build()
)
}
}
.build()
callback.onSuccess(fillResponse)
} else {
callback.onSuccess(null)
}
}
enum class Field {
PROMOCODE
}In this piece of code, we extract AssistStructure from FillRequest (it contains all the data about ViewNode), try parsing this structure, and extract the ViewNodes containing the promo code data. If such ViewNodes are found, we retrieve the promo codes stored in the app and fill FillResponse with the Dataset structures. Then we pass this response to onSuccess. If the promo code input field is not found in the hierarchy, null is returned.
Since the whole hierarchy of the views is a tree, we need to traverse this structure.
private fun parseStructure(
structure: AssistStructure,
autofillMap: MutableMap<Field, AutofillId>
) {
val windowNodes: List<AssistStructure.WindowNode> =
structure.run {
(0 until windowNodeCount).map { getWindowNodeAt(it) }
}
windowNodes.forEach { windowNode: AssistStructure.WindowNode ->
val viewNode: ViewNode? = windowNode.rootViewNode
viewNode?.let { traverseNode(it, autofillMap) }
}
}
private fun traverseNode(viewNode: ViewNode, autofillMap: MutableMap<Field, AutofillId>) {
viewNode.htmlInfo?.attributes
?.map { pair -> pair.first to pair.second }
?.forEach { (attrName, attrValue) ->
if (attrName.equals("name", ignoreCase = true) && attrValue.equals("promocode", ignoreCase = true) ||
attrName.equals("label", ignoreCase = true) && attrValue.equals("promocode", ignoreCase = true)
) {
autofillMap[Field.PROMOCODE] = viewNode.autofillId!!
}
}
val children: List<ViewNode> = viewNode.run {
(0 until childCount).map { getChildAt(it) }
}
children.forEach { childNode: ViewNode ->
traverseNode(childNode, autofillMap)
}
}Please note that ViewNode contains many useful attributes that can help us determine the current field and how to fill it, but the majority of these attributes characterize Android EditText, while we’d like to fill the inputs that are within WebView. This is why it is helpful to use the property viewNode.htmlInfo?.attributes that contains all standard HTML attributes assigned to the elements of the current page. Next, you can make up as fancy logic as you like to determine the elements you are interested in. For example, the code above selects elements where name or label attributes have the value “promocode”.
After we have found the elements on the page suitable for the autofill, we can retrieve the promo codes stored in the app.
val promocodes = getPromocodes()The logic for the upload is up to you: the promo codes can be stored in a database or in cache and updated asynchronously.
As a matter of fact, it’s not entirely correct to upload all the promo codes since we need only those that suit the current website. That’s why we can identify the URL of the current page in WebView and, using it, select only matching promo codes.
private fun getPromocodes(url: String?): List<String> {
return if (url?.contains("starfish") == true) { // here could be your apps logic
listOf("NEWYEAR23", "VALENTINE14", "1PURCHASE")
} else {
emptyList()
}
}Update traverseNode method so that it returned the URL currently viewed by the user:
private fun traverseNode(viewNode: ViewNode, resultStructure: ResultStructure) {
if (viewNode.className?.contains("android.webkit.WebView") == true) {
resultStructure.url = viewNode.webDomain
}
...
}
data class ResultStructure(
var url: String? = null,
val autofillMap: MutableMap<Field, AutofillId> = mutableMapOf()
)Now, if we launch our app and switch over to the desired website that has a field with a promo code that satisfies the logic defined in traverseNode, we will see our suggestions when we click on the input field.
You can find the full code of the solution on GitHub.
As a matter of fact, the Autofill framework has many more features than are described in this article. For example, we can add a screen with settings for DiscountsAutofillService and configure various parameters for our service from the system settings screen. Also, for Android 11, we can embed autofill suggestions into the virtual keyboard. As for other features, you can learn about them in the official documentation.
Here are the advantages of Autofill framework approach to the WebView autofill:
Provides the data stored in the entire system, and not only in your app
Configured with a lot of flexibility
And disadvantages:
Works for Android 8 only
The user must open the settings and install our app as Autofill Service manually. This is an extra step that cuts down users’ conversion.
Only one Autofill Service can be selected at any point in time. It means that, if a user has already chosen a password manager, he can’t add another autofill service for them to work in parallel—only one service can be selected.
As you can see, despite all the flexibility, this approach has significant drawbacks. That’s why I will describe two other approaches below.
JavaScript injection based approach
Android WebView allows us to write various workarounds related to JavaScript injection and execution on a loaded page. However, this entails some risks. Consider an example: same as in the approach above, we have an Activity with WebView but we will display autofill suggestions not in the popups linked to the current input field but in a scrollable list at the bottom of the screen (as shown in the screenshot), for simplicity.
Our first challenge is to determine the moment when we need to display this bottom bar, i.e. the moment when some editable field is in focus. After we have displayed this bar with the needed suggestions, the user will click on a certain field. So, another challenge will be to insert the selected text into the input field in focus.
First, configure our WebView by executing JavaScript code on it.
webivew.settings.javaScriptEnabled = trueThe next step is to set up JavascriptInterface of our WebView to ensure interaction between its contents and our app’s code.
webview.addJavascriptInterface(JSInterface(), "Android")
class JSInterface {
@JavascriptInterface
fun handleInputAttrs(attrsMapJson: String) {
// processFocusedInput(attrsMapJson)
}
}Right now, this code doesn’t return any results but we will add the functionality later.
Now we need to add EventListener for all input fields of the website to determine when and which input field is in focus. For this, write a small piece of JavaScript code and inject it into a loaded page in WebView. You can create the autofill.jsfile and add the following into it:
(function () {
var inputs = document.getElementsByTagName('input');
for (var index = 0; index < inputs.length; ++index) {
inputs[index].addEventListener('focus', (event) => {
var attrsMap = event.target.attributes
const attrs = Object.fromEntries(Array.from(attrsMap).map(item => [item.name, item.value]))
Android.handleInputAttrs(JSON.stringify(attrs));
}, true);
}
})();In this piece of code, we are searching for all the elements with the “input” tag and assign the event listener with type “focus” to them. This event listener will activate when we click on one of the elements. At this point, we take the list of the attributes of the selected element event.target.attributes, convert it to JSON, and pass it to the function Android.handleInputAttrs declared in the android client code.
This JS code can be executed when the page was loaded. So, add the corresponding event handler to our WebView:
val js = InputStreamReader(context.assets.open("autofill.js")).buffered().use { it.readText() }
webView.webViewClient = object : WebViewClient() {
override fun onPageFinished(view: WebView, url: String?) {
super.onPageFinished(view, url)
view.evaluateJavascript(js, null)
}
}In the first line, we read the contents of the JS script from the file. Then, install WebViewClient with the handler onPageFinished. And when this happens, the script is uploaded to WebView: view.evaluateJavascript(js, null).
Now, when the user taps on a certain input field on the page, handleInputAttrs(attrsMapJson: String) method, where the JSON line with the attributes of the current element is the argument, will be called.
In the next step, using the attributes, find out whether this field is suitable for autofill. Since our app deals with coupons, it can be checked as follows:
private fun processFocusedInput(attrsMapJson: String) {
val attrsMap = Gson().fromJson<Map<String, String>>(attrsMapJson, object : TypeToken<Map<String, String>>() {}.type)
when {
attrsMap.containsKey("name") && attrsMap["name"]!!.equals("coupon", ignoreCase = true) ||
attrsMap.containsKey("label") && attrsMap["label"]!!.equals("coupon", ignoreCase = true) -> {
showPromocodesSuggestions()
}
else -> {
hidePromocodesSuggestions()
}
}
}I use Gson library to map JSON into Map<String, String>, but you can use any other json parser. In the next step, check the attributes for presence of “coupon” in them. Here, the logic is entirely up to you; in theory, it should be developed by your teams’ analysts who have studied numerous websites dealing with promo codes. That’s why the above code is given as an example only.
If the field with the attributes satisfies our logic, display the bar with promo codes showPromocodesSuggestions(). Otherwise, hide it with hidePromocodesSuggestions().
Now it’s time for the second challenge: when the user clicks the selected suggestion, its text must be inserted into the active input field. For this, use a simpler script:
(function () {
document.activeElement.value = "<AUTOFILL_TEXT>";
})();In this piece of code, <AUTOFILL_TEXT> is the line that should be dynamically replaced with the text you need. This script is so short that it can be written in one line without being saved in a separate file. So lets just execute this script in our WebView:
private fun sendSuggestionToFocusedField(suggestion: Suggestion) {
val js = """
(function () {
document.activeElement.value = "${suggestion.value}";
})();
""".trimIndent()
binding.inAppBrowserViewWebView.evaluateJavascript(js, null)
}You can find the full code of this solution here.
This is it—the autofill works! But, as I’ve noted before, the logic for identifying the fields with promo codes varies from website to website. That’s why you (or your company’s analysts) will have to spend a lot of time to make your code work with the maximum number of websites.
Also, I’ve encountered an issue here: such an approach doesn’t work for input fields within iframe, since the Same-Origin Policy is used almost on all websites. The policy only allows JavaScript to operate with the contents of your own website. So your script won’t see any input fields located inside of the iframe. There’s no simple solution to this problem, that’s why this is a very important drawback of this approach to autofill.
Thus, it has the following advantages:
No unnecessary actions for users (entering the system settings and installing the autofill service)
Identifies types of fields (email, address, bank card data)
Though there are some problems:
Needs custom logic for different websites and input fields and doesn’t work with the elements within an iframe
Needs additional UI to display autofill suggestions
Approach based on sending KeyEvent to the input field
The previous two approaches have a number of limitations, so let’s consider another one. It’s based on sending KeyEventdirectly to the input field, i.e. we try imitating input of a text as if the user was printing it themselves.
So, we have a screen with WebView and a bottom bar for selecting suggestions. We need to be able to determine when the input of the field is in focus, and, for Android, it’s usually done by determining whether the virtual keyboard is displayed or hidden.
I assume we have some custom WebView where we can attach OnApplyWindowListener to find changes in the insets in the current window.
override fun onAttachedToWindow() {
super.onAttachedToWindow()
ViewCompat.setOnApplyWindowInsetsListener((context as Activity).window.decorView) { view, insets ->
val isKeyboardVisible = insets.isVisible(WindowInsetsCompat.Type.ime())
toggleSuggestions(isKeyboardVisible)
return ViewCompat.onApplyWindowInsets(view, insets)
}
}When the keyboard is displayed on the screen, we know that the user has tapped on some input field but we don’t know which field exactly. In this case, WebView is a black box for us. But we still can identify the URL of the page where the user currently is and pull coupons for the relevant website only. However, within this page, we don’t know whether the user is at the stage of entering a name, an address, or something else. Some apps simply display everything that’s possible to autofill. Klarna, for example, works this way.
After the suggestions bar is displayed to the user, they can click on some of them, and, at this moment, we can send a series from KeyEvent with relevant letters:
private fun fillCurrentInput(suggestion: String) {
val charMap = KeyCharacterMap.load(-1)
val events = charMap.getEvents(suggestion.toCharArray())
events.forEach {
dispatchKeyEvent(it)
}
}This way the input field currently in focus will be filled with the coupon text.
You can find all the code of this solution on GitHub.
To summarize, here are the advantages of this approach:
No unnecessary actions from users (going to the system settings and installing the autofill service)
Works for all input fields (including those within iframe)
No need to learn the intricacies of JS and specific websites
And the disadvantages:
Doesn’t identify types of fields (email, address, band card data) – everything is shown to the user at once
Needs additional UI to display autofill suggestions
Conclusion
We have described three methods of displaying autofill suggestions to a user within WebView of our app. They all have certain advantages and disadvantages, and deciding which one suits your app best is up to you. You can go through the code of all three on my Github.
The method based on sending KeyEvent with a text is probably the most universal since it doesn’t have any limitations at the level of a website and the Android system.
At the same time, Autofill framework has more features and, if Google engineers develop it further, maybe in future we won’t have the existing limitations. Also, there is no reason why you shouldn’t combine several approaches to achieve the best result.
What about you? Have you ever added autofill into a WebView of your app? Which approach did you use?
