Skip to content

AndroidNode

Interface

Accessibility tree nodes for UI element interaction.

AndroidNode represents a node in the Android accessibility tree. Each node corresponds to a UI element on screen and contains properties like text, bounds, and state. Use methods like screenContent() to get the root node and traverse the tree.

Getting Screen Content
// Get the accessibility tree
const screen = await agent.actions.screenContent();

// Find elements
const button = screen.findTextOne("Submit");
const allInputs = screen.filterAdvanced(f => f.isEditText());

// Interact with elements
if (button) {
  const { left, top, right, bottom } = button.boundsInScreen;
  await agent.actions.tap((left + right) / 2, (top + bottom) / 2);
}
AndroidNodeFilter
Learn about the builder pattern for complex node queries

Identification

NameTypeDescription
falseIdstringUUID unique per request (changes each time screen content is fetched)
viewId?stringAndroid view ID (e.g., 'com.example:id/button')
classNamestring | nullAndroid class name (e.g., 'android.widget.Button')
packageNamestringPackage name of the app owning this node

Layout

NameTypeDescription
boundsInScreen{left, top, right, bottom}Screen coordinates of the node's bounding box
indexnumberIndex within parent's children

Text Content

NameTypeDescription
text?stringText content of the node
description?stringContent description (accessibility label)
hintTextstring | nullHint text for input fields
maxTextLengthnumberMaximum text length for input fields
inputTypestringInput type for text fields
isShowingHintTextbooleanWhether hint text is currently shown
tooltipText?string | nullTooltip textsince Android 9+
paneTitle?string | nullPane title for accessibilitysince Android 9+
containerTitle?string | nullContainer titlesince Android 14+

State

NameTypeDescription
isEnabledbooleanWhether the node is enabled
clickablebooleanWhether the node is clickable
isLongClickablebooleanWhether the node responds to long click
isContextClickablebooleanWhether the node responds to context click
isFocusablebooleanWhether the node can receive focus
isFocusedboolean | nullWhether the node currently has focus
isScrollablebooleanWhether the node is scrollable
isSelectedbooleanWhether the node is selected
isCheckedboolean | nullChecked state (for checkboxes, switches)
isEditablebooleanWhether text can be edited
isDismissablebooleanWhether the node can be dismissed
isPasswordbooleanWhether this is a password field
isMultiLinebooleanWhether this is a multi-line text field
isVisibleToUserbooleanWhether the node is visible on screen
isImportantForAccessibilitybooleanWhether important for accessibility
isHeading?booleanWhether this node is a headingsince Android 9+
isTextSelectable?booleanWhether text can be selectedsince Android 13+
isTextEntryKey?booleanWhether this is a keyboard keysince Android 10+

Collection Info

NameTypeDescription
collectionInfo?objectInfo when node is a collection (ListView, RecyclerView). Contains rowCount, columnCount, itemCount, hierarchical, selectionMode
collectionItemInfo?objectInfo when node is a collection item. Contains rowIndex, columnIndex, rowSpan, columnSpan, isSelected

Hierarchy

NameTypeDescription
childrenAndroidNode[]Child nodes
parentAndroidNode | nullParent node (null for root)

Actions

NameTypeDescription
actionsnumber[]Array of supported action IDs (use with agent.constants.ACTION_*)
actionLabels{id: number, label?: string}[]Action labels with optional custom text

Methods

allNodes

Signature
allNodes(): AndroidNode[]

Returns all nodes in the subtree as a flat array, including this node and all descendants.

Returns AndroidNode[]Flat array of all nodes

Example
const screen = await agent.actions.screenContent();
const allNodes = screen.allNodes();
console.log("Total nodes:", allNodes.length);

// Find all visible text
const textNodes = allNodes.filter(n => n.text && n.isVisibleToUser);

findById

Signature
findById(id: string): AndroidNode[]

Finds all nodes with the matching viewId.

ParameterTypeDescription
idstringView ID to search for (e.g., 'com.example:id/button')

Returns AndroidNode[]Array of matching nodes

Example
const buttons = screen.findById("com.example:id/submit_button");

findByIdOne

Signature
findByIdOne(id: string): AndroidNode | null

Finds the first node with the matching viewId.

ParameterTypeDescription
idstringView ID to search for

Returns AndroidNode | nullFirst matching node or null

Example
const button = screen.findByIdOne("com.example:id/submit");
if (button) {
  await button.performAction(agent.constants.ACTION_CLICK);
}

findText

Signature
findText(textOrRegex: string | RegExp): AndroidNode[]

Finds all nodes containing the text in description, text, or hintText (case-insensitive). Supports strings and regular expressions.

ParameterTypeDescription
textOrRegexstring | RegExpText to search for or regex pattern

Returns AndroidNode[]Array of matching nodes

Find by string
const nodes = screen.findText("Submit");
Find by regex
const priceNodes = screen.findText(/\$\d+\.\d{2}/);

findTextOne

Signature
findTextOne(textOrRegex: string | RegExp): AndroidNode | null

Finds the first node containing the text (case-insensitive).

ParameterTypeDescription
textOrRegexstring | RegExpText to search for or regex pattern

Returns AndroidNode | nullFirst matching node or null

Example
const submit = screen.findTextOne("Submit");
if (submit) {
  const { left, top, right, bottom } = submit.boundsInScreen;
  await agent.actions.tap((left + right) / 2, (top + bottom) / 2);
}

find

Signature
find(predicate: (node: AndroidNode) => boolean): AndroidNode | null

Finds the first node matching a custom predicate function.

ParameterTypeDescription
predicate(node: AndroidNode) => booleanFunction to test each node

Returns AndroidNode | nullFirst matching node or null

Example
// Find first clickable button with specific text
const btn = screen.find(node =>
  node.className === "android.widget.Button" &&
  node.clickable &&
  node.text === "Submit"
);

// Find first visible input field
const input = screen.find(node =>
  node.isEditable && node.isVisibleToUser
);

filter

Signature
filter(predicate: (node: AndroidNode) => boolean): AndroidNode[]

Filters all nodes matching a custom predicate function.

ParameterTypeDescription
predicate(node: AndroidNode) => booleanFunction to test each node

Returns AndroidNode[]Array of matching nodes

Example
// Find all buttons
const buttons = screen.filter(node =>
  node.className === "android.widget.Button"
);

// Find all clickable, visible nodes
const clickables = screen.filter(node =>
  node.clickable && node.isVisibleToUser
);

// Find nodes with text containing dollar amounts
const prices = screen.filter(node =>
  node.text?.includes("$")
);

findAdvanced

Signature
findAdvanced(filterBuilder: (f: AndroidNodeFilter) => AndroidNodeFilter): AndroidNode | null

Finds the first node using the AndroidNodeFilter builder pattern.

ParameterTypeDescription
filterBuilder(f: AndroidNodeFilter) => AndroidNodeFilterFunction to build the filter

Returns AndroidNode | nullFirst matching node or null

Example
// Find the submit button
const submit = screen.findAdvanced(f => f.isButton().hasText("Submit"));

// Find first editable text field
const input = screen.findAdvanced(f => f.isEditText().isEditable());

// Find by view ID
const header = screen.findAdvanced(f => f.hasId("com.example:id/header"));

filterAdvanced

Signature
filterAdvanced(filterBuilder: (f: AndroidNodeFilter) => AndroidNodeFilter): AndroidNode[]

Filters all nodes using the AndroidNodeFilter builder pattern.

ParameterTypeDescription
filterBuilder(f: AndroidNodeFilter) => AndroidNodeFilterFunction to build the filter

Returns AndroidNode[]Array of matching nodes

Example
// Find all clickable buttons
const buttons = screen.filterAdvanced(f => f.isButton().isClickable());

// Find text nodes with specific text
const labels = screen.filterAdvanced(f => f.isText().hasText("Welcome"));

// Find nodes in a specific list
const listItems = screen.filterAdvanced(f =>
  f.anyParent(p => p.hasId("com.example:id/list"))
);

matches

Signature
matches(filterBuilder: (f: AndroidNodeFilter) => AndroidNodeFilter): boolean

Checks if this node matches the given filter.

ParameterTypeDescription
filterBuilder(f: AndroidNodeFilter) => AndroidNodeFilterFunction to build the filter

Returns booleantrue if node matches the filter

Example
// Check if a node is a clickable button
if (node.matches(f => f.isButton().isClickable())) {
  await node.performAction(agent.constants.ACTION_CLICK);
}

// Filter children manually
const clickableChildren = node.children.filter(child =>
  child.matches(f => f.isClickable())
);

toJSON

Signature
toJSON(): object

Converts the node to a plain JSON object. Useful for debugging or serialization.

Returns objectPlain object representation

Example
console.log(JSON.stringify(node.toJSON(), null, 2));

Action Methods

These methods allow you to perform actions directly on nodes. Available since app version 2.119.

performAction

since 2.119
Signature
performAction(actionInt: number, data?: object, fieldsToIgnore?: string[]): Promise<{ actionPerformed: boolean }>

Performs an accessibility action on this node. This is the preferred way to interact with UI elements as it uses the accessibility system.

ParameterTypeDescription
actionIntnumberAction constant (use agent.constants.ACTION_*)
data?objectAdditional action data (used for ACTION_SET_TEXT etc.)
fieldsToIgnore?string[]Fields to ignore when matching node

Returns Promise<{ actionPerformed: boolean }>Whether the action was performed successfully

Click a button
const button = screen.findTextOne("Submit");
if (button) {
  await button.performAction(agent.constants.ACTION_CLICK);
}
Set text on an input field
const input = screen.findAdvanced(f => f.isEditText());
if (input) {
  await input.performAction(agent.constants.ACTION_SET_TEXT, {
    [agent.constants.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE]: "Hello World"
  });
}
Scroll a list
const scrollView = screen.findAdvanced(f => f.isScrollable());
if (scrollView) {
  await scrollView.performAction(agent.constants.ACTION_SCROLL_FORWARD);
}

adbPerformAction

since 2.119
Signature
adbPerformAction(actionInt: number, data?: object, fieldsToIgnore?: string[]): Promise<{ actionPerformed: boolean }>

Performs an accessibility action on this node via ADB. The ADB equivalent of performAction(). Uses UiAutomation through the ADB touch server instead of the accessibility service.

ParameterTypeDescription
actionIntnumberAction constant (use agent.constants.ACTION_*)
data?objectAdditional action data (used for ACTION_SET_TEXT etc.)
fieldsToIgnore?string[]Fields to ignore when matching node

Returns Promise<{ actionPerformed: boolean }>Whether the action was successfully performed

Click a button via ADB
const button = screen.findTextOne("Submit");
if (button) {
  await button.adbPerformAction(agent.constants.ACTION_CLICK);
}
Set text via ADB
const input = screen.findAdvanced(f => f.isEditText());
if (input) {
  await input.adbPerformAction(agent.constants.ACTION_SET_TEXT, {
    [agent.constants.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE]: "Hello World"
  });
}

randomClick

since 2.119
Signature
randomClick(): void

Performs a tap at a random position within this node's bounds. Uses boundsInScreen to determine the click area. Useful for more natural-looking automation.

Example
const button = screen.findTextOne("Submit");
if (button) {
  button.randomClick();
}
Click multiple items
const items = screen.filterAdvanced(f => f.isClickable().hasText(/Item \d+/));
for (const item of items) {
  item.randomClick();
  await sleep(500);
}

randomSwipe

since 2.119
Signature
randomSwipe(direction: "up" | "down" | "left" | "right"): void

Performs a swipe gesture within this node's bounds. The swipe starts and ends at random positions within the node for more natural-looking automation.

ParameterTypeDescription
direction"up" | "down" | "left" | "right"Direction to swipe
Scroll a list down
const listView = screen.findAdvanced(f => f.isScrollable());
if (listView) {
  listView.randomSwipe("up"); // Swipe up to scroll down
}
Swipe through a carousel
const carousel = screen.findAdvanced(f => f.hasId("carousel"));
if (carousel) {
  carousel.randomSwipe("left"); // Swipe left to see next item
}

randomClickAdb

since 2.141 (153)
Signature
randomClickAdb(): void

Performs a tap at a random position within this node's bounds using ADB shell input instead of the accessibility service. Fire-and-forget like randomClick().

Example
const button = screen.findTextOne("Submit");
if (button) {
  button.randomClickAdb();
}

randomSwipeAdb

since 2.141 (153)
Signature
randomSwipeAdb(direction: "up" | "down" | "left" | "right"): void

Performs a swipe gesture within this node's bounds using ADB shell input instead of the accessibility service. Fire-and-forget like randomSwipe().

ParameterTypeDescription
direction"up" | "down" | "left" | "right"Direction to swipe
Scroll a list down via ADB
const listView = screen.findAdvanced(f => f.isScrollable());
if (listView) {
  listView.randomSwipeAdb("up"); // Swipe up to scroll down
}
Note: randomClickAdb() and randomSwipeAdb() require app version 2.141 (153)+. They use ADB shell commands instead of the accessibility service, which can be more reliable in some scenarios.

Common Patterns

Click a button (random position)
const button = screen.findTextOne("Submit");
if (button) {
  button.randomClick();
}
Click using accessibility action
const button = screen.findAdvanced(f => f.isButton().hasText("OK"));
if (button) {
  await button.performAction(agent.constants.ACTION_CLICK);
}
Type in an input field
const input = screen.findAdvanced(f => f.isEditText().isEditable());
if (input) {
  await input.performAction(agent.constants.ACTION_CLICK);
  await agent.actions.writeText("Hello World");
}
Scroll a list
const scrollView = screen.findAdvanced(f => f.isScrollable());
if (scrollView) {
  // Using accessibility action
  await scrollView.performAction(agent.constants.ACTION_SCROLL_FORWARD);

  // Or using random swipe for more natural scrolling
  scrollView.randomSwipe("up");
}