# Xgodo Documentation The complete `/docs` surface as plain text: the v2 Client API reference followed by the automation agent guide and reference. The internal v1 REST reference is a separate document at /docs/v1/llm.txt. -------------------------------------------------------------------------------- # Client API (v2) ## Authentication Bearer Token Authentication All API endpoints require authentication using a Bearer token. Include the following header in your requests: ``` Authorization: Bearer ``` -------------------------------------------------------------------------------- ## Jobs Manage job postings, retrieve applicants, and handle job-related operations. ### POST /api/v2/jobs/applicants — Retrieve job applicants Retrieves a list of job applicants for a specified job. Allows for sorting, pagination, and searching within job applicants. Note: per-job task count aggregates (`total_task`, `job_done`, `running_tasks`, `pending_tasks`, `satisfied_tasks`, `declined_tasks`, `failed_tasks`) are intentionally NOT included in the v2 response — they were rarely consumed and computing them under concurrent load is the cluster's hottest read. If you need them, derive client-side by paging with `status` filters. Request body: - `job_id` (string, required): The ID of the job for which to retrieve applicants - `task_id` (string): The ID of the job task or planned task. Accepts either a job_task_id or planned_task_id - if a planned_task_id is provided, it will be resolved to its associated job_task_id. Used when only one task details is needed. If present, the task result list array will contain only one task or empty array if task not found. - `sortby` (string): Field by which to sort job applicants - `order` (string): Sort order. Can be 'asc' for ascending or 'desc' for descending Example: asc - `page` (integer): Page number for pagination Example: 1 - `limit` (integer): Number of applicants per page Example: 10 - `hours_ago` (integer): The day interval, specified as the number of days starting from the given hours_ago up to today Example: 10 - `status` (string): The status of the task ('processing', 'confirmed', 'notcomplete', 'declined', 'pending') Example: confirmed - `search` (string): Search term to filter applicants based on worker name, job proof, job title, comment, or worker IP Example: developer Responses: - **200** — Successfully retrieved job applicants Returns job details with applicant information including planned_task, job tasks (each carrying its own `job_price` in USD — the actual amount paid for that task; defaults to the parent job's `job_price` at task creation and may diverge thereafter), total tasks, job done count, pending tasks, satisfied tasks, declined tasks, and failed tasks - **400** — Invalid request data or planned task exists but has no associated job task {"error": "Invalid task_id format"} or {"error": "Planned task {id} exists but has no associated job task (job_task_id is null)"} - **404** — Job not found, task_id is neither a valid job task ID nor a planned task ID {"error": "Job with this Id not found"} or {"error": "task_id {id} is neither a valid job task ID nor a planned task ID"} ### PUT /api/v2/jobs/applicants — Update job applicants' task statuses Updates the status of specified job tasks. It supports changing the status, adding comments, and handles referral bonuses if the task is confirmed. Request body: - `JobTasks_Ids` (array, required): An array of job task IDs or planned task IDs to be updated. Accepts a mix of job_task_ids and planned_task_ids - planned task IDs will be resolved to their associated job_task_ids. Example: ["60d21b4667d0d8992e610c85", "60d21b4667d0d8992e610c86"] - `status` (string, required): The new status to set for the job tasks Example: confirmed - `job_id` (string, required): The ID of the job to which the tasks belong Example: 60d21b4667d0d8992e610c85 - `comment` (string): Optional comment to add regarding the task update Example: Task successfully completed. Responses: - **200** — Successfully updated job task statuses. Returns the updated job task details - **400** — Bad request. Invalid input data, invalid status provided, or planned task exists but has no associated job task. {"error": "Invalid status..."} or {"error": "Planned task {id} exists but has no associated job task (job_task_id is null)"} - **404** — Job or job tasks not found, or ID is neither a valid job task ID nor a planned task ID. {"error": "Job Task with this Id not found"} or {"error": "ID {id} is neither a valid job task ID nor a planned task ID"} ### GET /api/v2/jobs/details — Get job details by ID Returns job details for a specific job ID. When the job is linked to an automation (`automation_id` is set), the response also includes the automation's `automation_parameters_schema`, `job_variables_schema`, and `bucket_schema` so callers can validate / build UIs around the `automationParameters`, `jobVariables`, and bucket payloads accepted by the automation. Each schema follows the `IInputSchema` shape (`{ fields: IFieldSchema[] }`) and is `null` when the automation has no schema configured or the job has no linked automation. Note: per-job task count fields (`totla_tasks_count`, `tasks_done_count`, `tasks_confirmed_count`, `tasks_running_count`, `tasks_pending_count`) are intentionally NOT included in the v2 response — they were rarely consumed and computing them under concurrent load is the cluster's hottest read. Query parameters: - `job_id` (string): Unique identifier for the job Example: 68910c4a5e1c7092a1fdc03a Example response: ```json { "_id": "68910c4a5e1c7092a1fdc03a", "job_id": "68910c4a5e1c7092a1fdc03a", "job_type": "devices_automation", "payment_type": "action", "title": "Create a Google account", "category": ["Google"], "description": "Create a Google account. ...", "proof": "Google account email. ...", "Target_Workers": ["worldwide"], "few_times": true, "premium": false, "featured": false, "duration": "3", "positions": "3", "job_price": 0.06, "price": "0.30", "status": "active", "files": [...], "added": "2025-08-04T19:38:50.204Z", "user": {...}, "automation_id": "83ba7359-d8eb-4374-8ecb-055af01fddfe", "automation_parameters_schema": { "fields": [ { "name": "maxRetries", "type": "number", "required": false, "min": 1, "max": 10 } ] }, "job_variables_schema": { "fields": [ { "name": "email", "type": "string", "required": true }, { "name": "password", "type": "string", "required": true } ] }, "bucket_schema": { "fields": [ { "name": "sessionToken", "type": "string", "required": false } ] } } ``` ### POST /api/v2/jobs/check-uniquenes — Check for uniqueness of job proofs This endpoint checks if a given search term exists in the job proofs. You can specify the search option to look for exact words or substrings. Request body: - `search` (string, required): The term to search for in job proofs Example: developer - `option` (string): The search option to use. Can be "word" for exact word matching or "charactor" for substring matching Example: word Responses: - **200** — Successfully checked for uniqueness Returns a boolean indicating if the search term is unique - **400** — Bad request. Either the search term is missing, or an invalid search option is provided Error message describing the validation error ### POST /api/v2/jobs/myjobs — Retrieve a list of jobs that you have posted Returns a paginated list of jobs the caller has posted under `data` (with `count`, paginated by `page`), plus a separate list of jobs shared with the caller under `shared_data` (with `shared_count`, paginated independently by `shared_page` — the current shared page is echoed back as `shared_page`). Each entry in `shared_data` carries `permission: "view" | "edit"` and `owner_username`. The response is a thin projection of the Job documents — `job_price` and `price` are stringified. Results are sorted by `added` descending. Note: per-job task counters (`job_done`, `running_task`, `pending_task`) and planned-task counters (`used_planned_tasks`, `added_planned_tasks`) are not included. To get per-task data for a specific job, use the `/api/v2/jobs/applicants` endpoint. Request body: - `page` (integer): Page number for the jobs you posted (the `data` list) Example: 1 - `limit` (integer): Number of items per page (applies to both lists; default 10, max 200) Example: 10 - `shared_page` (integer): Page number for the jobs shared with you (the `shared_data` list). Paginates independently of `page`. Defaults to 1. Example: 1 - `job_id` (string): Filter jobs by a specific job ID - `status` (string): Filter jobs by status (e.g., 'active', 'completed') - `orientation` (string): The orientation for the data sorting (e.g., 'asc' or 'desc') Example: asc - `Query` (string): Search query for job title or description - `SortBY` (string): Field to sort the results by ### POST /api/v2/jobs/search — Search and filter jobs Returns a paginated list of jobs matching the provided filters. Supports search, category, price, location, and other filters. Request body: - `page` (string): Page number for pagination Example: 1 - `limit` (string): Number of jobs per page Example: 10 - `searchTerm` (string): Search term for job title or description - `job_category` (string): Filter by job category - `min_price` (string): Minimum job price - `location` (string): Filter by worker location - `Featured` (boolean): Filter for featured jobs - `Premium` (boolean): Filter for premium jobs - `orientation` (string): Sort orientation (asc, desc) Example: desc - `SortBY` (string): Field to sort by Example: added ### POST /api/v2/jobs/submit — Create a new job posting This endpoint allows users to create a new job posting. It requires specific details about the job, including title, description, category, number of positions, job price, and duration. Request body: - `title` (string, required): The title of the job posting Example: Software Developer - `category` (string, required): Category of the job Example: Engineering - `description` (string, required): Detailed description of the job Example: We are looking for a skilled software developer... - `proof` (string, required): Proof or evidence related to the job posting Example: Sample proof text. - `is_proof_file` (boolean): Indicates if the proof is a file or text Example: false - `auto_rate` (boolean): Whether the job should be automatically rated Example: true - `vcode` (string): Verification code for the job posting Example: cdcdc - `Target_Workers` (array): List of target workers or countries Example: ["worldwide"] - `few_times` (boolean): Indicates if the job can be posted a few times Example: false - `premium` (boolean): Indicates if the job is premium Example: false - `featured` (boolean): Whether the job is featured or not Example: false - `duration` (integer, required): Duration for which the job is posted (in days) Example: 60 - `positions` (integer, required): Number of available positions for the job Example: 3 - `daily_limit` (integer): Daily limit for the job posting Example: 0 - `job_price` (number, required): Price for posting the job Example: 100 Responses: - **201** — Job created successfully Success message indicating job creation - **400** — Bad request. Invalid or missing required fields List of validation error messages ### PUT /api/v2/jobs/update-status — Update job status Updates the status of a specific job. Request body: - `job_id` (string, required): The ID of the job Example: 60c72b2f9b1e8c001c8e4d8a - `status` (string, required): New status for the job (active, inactive) Example: active Responses: - **204** — Job status updated successfully - **400** — Invalid request body or status - **401** — Unauthorized action - **404** — Job not found or invalid job_id -------------------------------------------------------------------------------- ## Tasks Manage job tasks, apply for tasks, and submit task completions. ### GET /api/v2/tasks/apply — Apply for task submission to a job This API must be called before tasks/submit API for jobs that have job variables. This endpoint allows workers to apply to an active job. Job variables (if any) will be returned. Query parameters: - `job_id` (string, required): Unique identifier for the job Example: 68868530c189957861cd698a Example response: ```json { "_id": "68910c4a5e1c7092a1fdc03c", "var1": "Custom variable value 1", "var2": "Custom variable value 2" } ``` ### POST /api/v2/tasks/details — Get recent tasks or details of a specific job task Returns the list of recent tasks of the user. If task_id query param is provided, it will return details of a single task. Query parameters: - `task_id` (string): The ID of the job task to fetch details for Example: 686f668db9c27eea026e60c7 Request body: - `page` (integer): Page number for pagination (starts from 1) Example: 1 - `limit` (integer): Number of tasks to return per page Example: 10 Example response: ```json { "_id": "686f668db9c27eea026e60c7", "job_task_id": "686f668db9c27eea026e60c7", "worker_id": "676d3525e618d4ad99dae7ff", "job_id": "686f44cf8abf68ace53ebc8f", "job_title": "Create a Google account", "job_price": 0.06, "job_proof": "", "comment": "", "country_code": "IN", "worker_ip": "192.168.249.223", "status": "confirmed", "failureReason": null, "reviewed": null, "device_owner_share": null, "added": "2025-07-10T07:06:53.631Z", "updated": "2025-07-10T07:06:53.631Z", "proof_files_info": [] } ``` ### POST /api/v2/tasks/submit — Submit a task to a job Submits a task to an active job. The endpoint operates in one of two modes selected by the target job: 1. Manual mode (default) — for regular jobs. Honors job-level gates (worker rating, country, allowed_workers, premium, few_times, custom_vars, predeclared file fields, positions, task_limit, Worker_Invites) and triggers task/job completion emails. 2. Automation data-dump mode — selected automatically when the target job is of type devices_automation, has automation_id set, and has no planned tasks. Used by an automation agent running another job to write structured data into a separate sink job. In this mode the endpoint: • requires the caller's token to carry job_task_id and remote_device_id (i.e. a running agent token); • requires job_proof to be valid JSON; • accepts files in the {name, extension, base64Data} shape, embedded into the JSON proof under proof[name]; • accepts an explicit status of "pending" (default), "failed", or "declined"; • derives device_id and the device-owner share from the agent's remote_device_id; • authorizes by requiring the running agent's parent job and the target data-dump job to share the same automation_id, AND the running job's owner to be the target job's owner or to appear in the target's shared_view / shared_edit; • skips manual-mode gates (rating/country/few_times/custom_vars/proof_files/positions/task_limit/Worker_Invites) and skips completion emails; • does not update automation iteration / success-rate / blacklist analytics — those are reserved for "real" agent-driven tasks created by the automation engine. Request body: - `job_id` (string, required): ID of the job. The endpoint runs in one of two modes depending on the target job: "manual" mode for regular jobs, or "automation data-dump" mode when the target job is of type devices_automation with automation_id set and no planned tasks. Example: 68868530c189957861cd698a - `job_proof` (string, required): Proof of the task done. In automation data-dump mode this string MUST be valid JSON. Example: irjfoirf@gmail.com:iejowiedj:ehiuehwd@outlook.com:wuehuidhewd - `custom_vars` (object): Manual mode only. Custom vars obtained from tasks/apply. Required for jobs with variables. Example: {"_id": "68910c4a5e1c7092a1fdc03c", "var1": "value1"} - `proof_files_base64` (array): Manual mode only. Array of required and optional proof files in base64 format, mapped to the job's predeclared file fields. Example: [{"fieldName": "screenshot", "fileName": "image.jpg", "base64": "..."}] - `status` (string): Automation data-dump mode only. One of "pending", "failed", or "declined". Defaults to "pending". Example: pending - `files` (array): Automation data-dump mode only. Files to attach to the submission. Each file is written to disk and its public URL is embedded into the JSON job_proof under the key matching `name`. Each base64Data is capped at 5 MB. Example: [{"name": "screenshot", "extension": ".png", "base64Data": "..."}] Responses: - **201** — Job task submitted successfully Success message indicating task submission, job task id and job id - **400** — Bad request — e.g. job_proof is not a string, or in automation data-dump mode job_proof is not valid JSON / files validation failed. Error message describing the validation failure - **403** — Forbidden. Returned for automation data-dump submissions when the caller is not a running agent (token must carry remote_device_id and job_task_id), the agent has no currently-running task, the running job and target job do not share the same automation_id, or the running job's owner has no access (owner / shared_view / shared_edit) to the target data-dump job. Error message describing the authorization failure - **404** — Job not found, running task's job not found, or device not found Error message indicating the missing resource ### GET /api/v2/planned_tasks — List unassigned planned tasks for a job Returns planned tasks for a job that have not yet been assigned to a job task (i.e. job_task_id is null). Supports search, pagination, and sort order. Only the job owner (or a user the job is shared with) can list its planned tasks. Query parameters: - `job_id` (string, required): The ID of the job whose planned tasks should be listed Example: 60c72b2f9b1e8c001c8e4d8a - `search` (string): Case-insensitive substring match against the planned task input (max 10,000 chars) Example: john@example.com - `page` (integer): Page number for pagination (starts from 1). Defaults to 1 Example: 1 - `limit` (integer): Number of planned tasks to return per page (1-100). Defaults to 10 Example: 10 - `sortOrder` (string): Sort order by creation time. One of "asc" or "desc". Defaults to "desc" Example: desc Responses: - **200** — Planned tasks (unassigned) fetched successfully. Only tasks whose job_task_id is null are returned. {"success": true, "data": {"job": {...}, "plannedTasks": [{"planned_task_id": "...", "job_id": "...", "input": "...", "added": "..."}], "total": 42}} - **400** — Bad request - Invalid query parameters {"success": false, "error": "Invalid job id"} - **404** — Job not found or user doesn't own the job {"success": false, "error": "Job not found"} - **500** — Internal server error {"success": false, "message": "Internal Server Error"} ### POST /api/v2/planned_tasks/submit — Submit planned tasks for a job This endpoint allows users to submit multiple planned tasks for a specific job. Planned tasks are pre-defined inputs that will be used when workers apply for the job. Request body: - `job_id` (string, required): The ID of the job to submit planned tasks for Example: 60c72b2f9b1e8c001c8e4d8a - `inputs` (array, required): Array of input strings for planned tasks (usually JSON strings, minimum 1, maximum 10,000 characters per string) Example: ["{\"fname\":\"John\",\"lname\":\"Doe\",\"email\":\"john@example.com\"}", "{\"fname\":\"Jane\",\"lname\":\"Smith\",\"email\":\"jane@example.com\"}"] - `remote_device_id` (string (UUID v4)): Optional device ID to pin all submitted tasks to a specific device. When set, the system will only assign these tasks to the specified device. If the device is not available, the task will be retried later. Example: a1b2c3d4-e5f6-7890-abcd-ef1234567890 - `device_name` (string): Optional device name (system-generated, two-word labels like "Atomic Mammal" or "Enormous Elephant") to pin all submitted tasks to a specific device. Resolved to a remote_device_id server-side. Ignored when remote_device_id is also provided. If the name matches more than one device, the request fails with 400 — use remote_device_id to disambiguate. Example: Atomic Mammal - `run_immediately` (boolean): If true, the system will attempt to assign the planned tasks immediately after creation (up to 3 attempts, 5 seconds apart). The response will include assignment_results indicating whether each task was assigned. Defaults to false. Example: true Responses: - **200** — Planned tasks submitted successfully. When run_immediately is true, includes assignment_results. {"success": true, "inserted_ids": [{"planned_task_id": "60c72b2f9b1e8c001c8e4d8a", "input": "{\"fname\":\"John\",\"email\":\"john@example.com\"}"}], "assignment_results": [{"planned_task_id": "60c72b2f9b1e8c001c8e4d8a", "assigned": true}]} - **400** — Bad request - Invalid input data, exceeds available positions, or validation errors {"success": false, "error": "Invalid job id"} or {"success": false, "message": "No enough positions left. Received X tasks in excess."} - **404** — Job not found or user doesn't own the job {"success": false, "error": "Job not found"} - **500** — Internal server error {"success": false, "message": "Internal Server Error"} ### DELETE /api/v2/tasks/delete — Delete a job task by ID This endpoint allows employers to delete a job task. Accepts either a job_task_id or planned_task_id (a planned_task_id is resolved to its associated job task). Only tasks with certain statuses can be deleted to maintain data integrity (failed, declined, confirmed). The associated planned task is deleted along with the job task by default; pass keep_planned_task: true to instead preserve it (its job_task_id is reset to null) so it can be reused. Request body: - `task_id` (string, required): The ID of the job task or planned task to delete. Accepts either a job_task_id or planned_task_id - if a planned_task_id is provided, it is resolved to its associated job task, which is then deleted. Example: 60c72b2f9b1e8c001c8e4d8a - `keep_planned_task` (boolean): If true, the planned task associated with the deleted job task is preserved (its job_task_id is reset to null) so it can be reused; if false (default), that planned task is deleted along with the job task. Applies whether task_id is a job_task_id or a planned_task_id. Example: false Responses: - **200** — Job task deleted successfully. The associated planned task is deleted by default, or preserved with job_task_id set to null when keep_planned_task is true. {"success": true, "message": "Job task deleted successfully"} - **400** — Bad request - Invalid task ID, task cannot be deleted due to status restrictions, or planned task exists but has no associated job task {"success": false, "error": "Invalid task id"} or {"success": false, "message": "Cannot delete job task with status 'confirmed'. Only tasks with status: failed, declined can be deleted."} or {"success": false, "error": "Planned task {id} exists but has no associated job task (job_task_id is null)"} - **403** — Forbidden - User doesn't have permission to delete this job task {"success": false, "message": "You don't have permission to delete this job task"} - **404** — Job task not found, or task_id is neither a valid job task ID nor a planned task ID {"success": false, "message": "Job task not found"} or {"success": false, "error": "task_id {id} is neither a valid job task ID nor a planned task ID"} - **500** — Internal server error {"success": false, "message": "Internal Server Error"} ### DELETE /api/v2/planned_tasks — Delete one or more planned tasks Deletes unassigned planned tasks by their IDs. Planned tasks already assigned to a pending or successful job task cannot be deleted — reinitiate or delete the job task first. All IDs must belong to jobs owned by (or shared with edit permission to) the caller. Request body: - `planned_task_ids` (string[], required): Array of planned task IDs to delete (minimum 1). A planned task that is already assigned to a pending or successful job task cannot be deleted. Example: ["60c72b2f9b1e8c001c8e4d8a", "60c72b2f9b1e8c001c8e4d8b"] Responses: - **200** — Planned tasks deleted successfully {"success": true} - **400** — Bad request - Invalid planned task id, or one of the planned tasks is already assigned to a pending/successful job task {"success": false, "message": "Cannot delete a planned task that is assigned to a pending / successful job task"} - **404** — One or more planned tasks not found, or the job is not owned by the caller {"success": false, "message": "Planned task not found"} - **500** — Internal server error {"success": false, "message": "Internal Server Error"} ### POST /api/v2/planned_tasks/reinitiate — Re-initiate failed/declined tasks This endpoint allows employers to re-initiate one or more failed or declined tasks. Accepts either a single task_id or an array of task_ids. This makes the planned tasks available again for workers to apply. Request body: - `task_id` (string): The ID of a single job task to re-initiate (for backward compatibility) Example: 60c72b2f9b1e8c001c8e4d8a - `task_ids` (string[]): An array of job task IDs to re-initiate. Either task_id or task_ids must be provided. Example: ["60c72b2f9b1e8c001c8e4d8a", "60c72b2f9b1e8c001c8e4d8b"] Responses: - **200** — Task re-initiated successfully {"success": true, "message": "Task successfully reinitiated"} - **400** — Bad request - Invalid task ID or task cannot be reinitiated due to status restrictions {"success": false, "error": "Invalid task id"} or {"success": false, "message": "Cannot reinitiate task with status 'confirmed'. Only failed or declined tasks can be reinitiated."} - **403** — Forbidden - User doesn't have permission to reinitiate this task {"success": false, "message": "You don't have permission to reinitiate this task"} - **404** — Job task not found or no planned task associated with this job task {"success": false, "message": "Job task not found"} - **500** — Internal server error {"success": false, "message": "Internal Server Error"} -------------------------------------------------------------------------------- ## Payments Handle payment operations including withdrawals. ### POST /api/v2/payments/withdraw — Withdraw user balance Initiate a withdrawal to a wallet address. Request body: - `amount` (number, required): Amount to withdraw Example: 10 - `wallet_address` (string, required): Wallet address to receive the withdrawal Example: 0x1234567890abcdef Responses: - **200** — Withdrawal successful {"msg": "withdraw done"} - **400** — Insufficient balance, below minimum payout, or invalid request Error message with specific reason - **401** — Unauthorized or missing token - **500** — Server error ### GET /api/v2/payments/withdrawals — Get recent withdrawal details Returns the list & details of recent withdrawals made by the user. Query parameters: - `page` (integer): Page number for pagination (starts from 1) Example: 1 - `limit` (integer): Number of withdrawals to return per page Example: 10 Responses: - **200** — Details of the withdrawals made by the user - **401** — Unauthorized or missing token - **404** — No transactions found - **500** — Server error Example response: ```json { "total_count": 3, "transactions": [ { "_id": "67d9f0b57131be1625d301c4", "reference_id": "26795145", "user_id": "676d3619e618d4ad99dae899", "recipient_id": null, "crypto_uuid": null, "payment_type": "Withdrawal", "payment_method": "cryptomus", "payment_details": "", "transaction_fee": "0", "currency": "USD", "amount": "-10", "wallet_address": "2w", "status": "paid", "reason": null, "created": "2025-03-18T22:16:21.895Z", "updated": "2025-03-18T22:16:21.895Z", "process_date": "2025-03-18T22:16:21.895Z", "transaction_id": "67d9f0b57131be1625d301c4", "transaction_proof": "2ca560b36dcdd7c8", "__v": 0 } ], "page": 1, "balance": 3.54, "total_withdraw": -30, "total_deposit": 0 } ``` -------------------------------------------------------------------------------- ## User User account management and balance information. ### GET /api/v2/user/balance — Get user balance Returns the user's current balance, active balance and pending earnings. Responses: - **200** — User balance information retrieved successfully - **401** — Unauthorized or missing token - **500** — Server error Example response: ```json { "balance": 4.434, "active_balance": 4.434, "pending_earnings": 1.25 } ``` -------------------------------------------------------------------------------- ## Devices Device management and information retrieval endpoints. ### GET /api/v2/devices — Get debuggable devices Retrieves a list of devices that are either owned by the user or rented with live payment type. Responses: - **200** — Successfully retrieved devices list - **401** — Unauthorized or missing token - **500** — Server error Example response: ```json [ { "remote_device_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "added": "2025-08-19T10:00:00Z", "automations": [], "brand": "Samsung", "country": "US", "isEmulator": false, "model": "Galaxy S21", "name": "Atomic Mammal", "networkType": "wifi", "numberOfCores": 8, "online": true, "processor": "Snapdragon 888", "ramMb": 8192, "sdkVersion": "31", "version": "12" } ] ``` ### GET /api/v2/devices/market — List market devices (action / online) Returns devices that are listed on the market with payment_type 'action' and currently online. Each item includes the device name, country, and an isAvailable flag (false when the device has at least one job task in 'running' status, true otherwise). Results can be filtered by country and are sorted by country. Query parameters: - `country` (string): Filter devices by country. Example: US - `sortDirection` (string): Sort direction for the country field. "asc" or "desc". Defaults to "asc". Example: asc Responses: - **200** — Successfully retrieved the list of online, action-payment market devices. - **401** — Unauthorized or missing token - **500** — Server error Example response: ```json [ { "name": "Atomic Mammal", "country": "BR", "isAvailable": true }, { "name": "Enormous Elephant", "country": "US", "isAvailable": false } ] ``` ### GET /api/v2/devices/verified-phone-number — Get the device's verified phone number Agent-scoped endpoint. Identifies the device from the automation agent token and returns the phone number stored on the device record only when phoneNumberVerified is true. Returns null otherwise. Backs `agent.info.getVerifiedPhoneNumber` in the automation bootstrap. Responses: - **200** — Returns the verified phone number stored on the device record, or null when the device has no verified phone number. - **400** — Token is not bound to a device (e.g. user-scoped token rather than an automation agent token). - **401** — Unauthorized or missing token - **404** — Device not found - **500** — Server error Example response: ```json { "phoneNumber": "+14155550123" } ``` -------------------------------------------------------------------------------- ## Proxy Manage SOCKS5/HTTP proxy credentials. A connection is one credential pair (`connectionId` + `password`) routed through one of your devices. Re-pointing the association to a different device is the IP-rotation primitive — new sessions begin exiting through the new device within ~30 s (orchestrator auth-cache TTL); in-flight sessions stay on the old device until they close. Endpoints are scoped to devices the caller owns; the admin endpoint requires the admin token. ### POST /api/v2/devices/connections/list — List my connections All connections owned by, or currently routed through, devices the caller owns. The `password` is plaintext — RMS owns the credential. Responses: - **200** — All connections owned by, or routed through, devices the caller owns. - **401** — Missing or invalid API token. Example response: ```json { "success": true, "connections": [ { "connectionId": "wXyZ04827193", "password": "p9F7bX3cVqLt", "ownerUserId": "add02036-6754-4cf7-8432-cc28be4468f0", "name": "client-A", "createdAt": "2026-04-25T13:22:11Z", "deviceId": "add02036-6754-4cf7-8432-cc28be4468f0" } ] } ``` ### POST /api/v2/devices/connections/create — Create a connection Generates a fresh `connectionId` (4 letters + 8 digits) and 12-char password, associates it with the chosen device, and syncs to the orchestrator + legacy mirror. The plaintext password is in the response — save it now. Request body: - `device_id` (string (UUID — `remote_device_id`), required): Owned device id. - `name` (string): Optional human label (≤ 100 chars). Cosmetic only — the credential is the auto-generated `connectionId`. Responses: - **200** — Created — RMS generates a `connectionId` (4 letters + 8 digits) + 12-char password and pushes to the orchestrator. Save the plaintext password from the response: the orchestrator stores only a hash. - **400** — `device_id` missing. - **404** — Device not owned by caller. Example response: ```json { "success": true, "connection": { "connectionId": "wXyZ04827193", "password": "p9F7bX3cVqLt", "ownerUserId": "add02036-6754-4cf7-8432-cc28be4468f0", "name": "client-A", "createdAt": "2026-04-25T13:22:11Z", "deviceId": "add02036-6754-4cf7-8432-cc28be4468f0" } } ``` ### POST /api/v2/devices/connections/delete — Delete a connection Cascades on RMS, orchestrator, and the legacy mirror. Active sessions are terminated. Request body: - `connection_id` (string, required): The connection's id — a 12-char token (4 letters + 8 digits), also the SOCKS5 username. Responses: - **200** — Deleted on RMS, the orchestrator, and the legacy mirror. Active proxy sessions using this credential are terminated. - **403** — Connection not owned/routed through any device the caller owns. - **404** — Connection not found. ### POST /api/v2/devices/connections/password/rotate — Rotate password Replaces the connection's password (same `connectionId`). Active sessions continue on the old credential until they close. Request body: - `connection_id` (string, required): The connection's id — a 12-char token (4 letters + 8 digits), also the SOCKS5 username. Responses: - **200** — Generates a new 12-char password (same `connectionId`). - **403** — Forbidden. Example response: ```json { "success": true, "password": "qK4LmPzWxR7a" } ``` ### POST /api/v2/devices/connections/association/set — Move connection to another device (IP rotation) Hot-swap the exit device. Owner must own both the source connection and the target device. Request body: - `connection_id` (string, required): The connection's id — a 12-char token (4 letters + 8 digits), also the SOCKS5 username. - `device_id` (string (UUID), required): Target device — **must also be owned by the caller**. Responses: - **200** — Hot-swap successful. New SOCKS5/HTTP CONNECT sessions begin routing to the new device within ~30 s (orchestrator auth-cache TTL). In-flight sessions stay on the old device until they close. - **403** — Source connection or target device not owned by caller. ### POST /api/v2/devices/connections/association/delete — Remove association Connection stays alive but cannot authenticate until re-associated. Request body: - `connection_id` (string, required): The connection's id — a 12-char token (4 letters + 8 digits), also the SOCKS5 username. Responses: - **200** — Connection stays but cannot route until a new association is set. Active sessions are terminated. - **403** — Forbidden. ### POST /api/v2/devices/connections/blacklist/list — List per-connection blacklist Per-connection block list (in addition to global + per-device). Request body: - `connection_id` (string, required): The connection's id — a 12-char token (4 letters + 8 digits), also the SOCKS5 username. Responses: - **200** — Per-connection blacklist domains (in addition to the global blacklist). Suffix matching applies. - **403** — Forbidden. Example response: ```json { "success": true, "domains": ["example.com", "tracker.io"] } ``` ### POST /api/v2/devices/connections/blacklist/add — Add domains to blacklist Adds one or more domains to the connection's blacklist. Suffix matching applies on the proxy side — blocking `example.com` also blocks `sub.example.com`. Request body: - `connection_id` (string, required): The connection's id — a 12-char token (4 letters + 8 digits), also the SOCKS5 username. - `domains` (string[], required): Non-empty array of domains to add or remove. Responses: - **200** — Updated. - **403** — Forbidden. ### POST /api/v2/devices/connections/blacklist/remove — Remove domains from blacklist Removes one or more domains from the connection's blacklist. Request body: - `connection_id` (string, required): The connection's id — a 12-char token (4 letters + 8 digits), also the SOCKS5 username. - `domains` (string[], required): Non-empty array of domains to add or remove. Responses: - **200** — Updated. - **403** — Forbidden. ### POST /api/v2/devices/connections/whitelist/list — List whitelisted IPs Returns the IP/CIDR patterns currently on the connection's whitelist. When a customer's source IP matches any pattern, they can open proxy streams without a password — just the `connectionId` in the SOCKS5/HTTP username field. Request body: - `connection_id` (string, required): The connection's id — a 12-char token (4 letters + 8 digits), also the SOCKS5 username. Responses: - **200** — Array of whitelisted IP patterns. - **401** — Missing or invalid API token. Example response: ```json { "success": true, "ips": ["203.0.113.5", "10.0.0.0/8"] } ``` ### POST /api/v2/devices/connections/whitelist/add — Add IP to whitelist Adds an IP or CIDR to the connection's whitelist. Idempotent — re-posting the same pattern updates its label. Domain patterns are rejected (use blacklist for domain-level control). Request body: - `connection_id` (string, required): The connection's id — a 12-char token (4 letters + 8 digits), also the SOCKS5 username. - `pattern` (string, required): IP address or CIDR to whitelist (e.g. `1.2.3.4` or `10.0.0.0/8`). Domain patterns are rejected. - `label` (string): Optional human label (e.g. `office`, `CI runner`). Responses: - **200** — Whitelist entry added/removed. - **400** — Invalid pattern (must be IP or CIDR). - **401** — Missing or invalid API token. - **502** — Gost orchestrator unreachable. ### POST /api/v2/devices/connections/whitelist/remove — Remove IP from whitelist Removes an IP/CIDR from the connection's whitelist. Request body: - `connection_id` (string, required): The connection's id — a 12-char token (4 letters + 8 digits), also the SOCKS5 username. - `pattern` (string, required): IP or CIDR to remove from the whitelist. Responses: - **200** — Whitelist entry added/removed. - **400** — Invalid pattern (must be IP or CIDR). - **401** — Missing or invalid API token. - **502** — Gost orchestrator unreachable. ### POST /api/v2/devices/transport-mode/get — Get device transport mode Returns the current egress transport mode for the device. Determines which network interface outbound traffic exits through. Request body: - `device_id` (string (UUID — `remote_device_id`), required): Owned device id. Responses: - **200** — Mode returned or updated. - **400** — Invalid mode value. - **401** — Missing or invalid API token. - **403** — Device not owned by caller. - **502** — Gost orchestrator unreachable. ### POST /api/v2/devices/transport-mode/set — Set device transport mode Changes the egress transport mode. The phone picks up the change within one heartbeat interval (~30 s). `cellular` uses the carrier IP (default), `wifi` uses the WiFi router's public IP, `auto` prefers cellular with WiFi fallback. Note: `wifi` mode is partial — the downstream transport binding (cellular + WiFi tunnel for cost savings) requires additional phone-side work. Request body: - `device_id` (string (UUID — `remote_device_id`), required): Owned device id. - `mode` (string, required): `cellular` (default — carrier IP), `wifi` (WiFi router's IP), or `auto` (prefer cellular, fallback to WiFi). Responses: - **200** — Mode returned or updated. - **400** — Invalid mode value. - **401** — Missing or invalid API token. - **403** — Device not owned by caller. - **502** — Gost orchestrator unreachable. ### POST /api/v2/devices/proxy-logs/list — Query request logs One log entry per SOCKS5 / HTTP CONNECT attempt. Backed by ClickHouse; results are post-filtered to devices owned by the caller. Request body: - `from` (string (ISO 8601), required): Inclusive lower bound on `timestamp`. - `to` (string (ISO 8601), required): Inclusive upper bound on `timestamp`. - `connectionId` (string): Filter to one connection. - `deviceId` (string (UUID)): Filter to one device — must be owned by the caller. - `target` (string): Substring match on the request target (`host:port`). - `action` ("allowed" | "blocked"): Filter by outcome. - `limit` (integer): Page size (default 100, max 10000). - `offset` (integer): 0-indexed offset for pagination. - `backend` ("zarif" | "gost"): Which proxy orchestrator to query. Default `zarif`. Pass in the JSON request body for POST endpoints; URL query params are ignored. Responses: - **200** — Request log rows from ClickHouse. Post-filtered to devices owned by the caller. - **400** — `from`/`to` missing or invalid; `limit` out of range. Example response: ```json { "success": true, "total": 12345, "entries": [ { "timestamp": "2026-04-25T13:22:11Z", "connectionId": "wXyZ04827193", "deviceId": "add02036-6754-4cf7-8432-cc28be4468f0", "serverNode": "server-us-aws", "target": "example.com:443", "targetType": "domain", "port": 443, "resolvedIp": "", "action": "allowed", "blockReason": null, "bytesIn": 184213, "bytesOut": 4218 } ] } ``` ### POST /api/v2/devices/proxy-bandwidth — Query bandwidth aggregates Per-(device, connection) byte counts, optionally grouped by hour or day. When `deviceId` is omitted the backend issues one query per owned device and merges. Request body: - `deviceId` (string (UUID)): Filter to one device — must be owned by the caller. Omit to fetch per-device rows for every device the caller owns. - `connectionId` (string): Filter to one connection. - `from` (string (ISO 8601)): Inclusive lower bound. - `to` (string (ISO 8601)): Inclusive upper bound. - `groupBy` ("hour" | "day"): Time-bucket granularity. - `backend` ("zarif" | "gost"): Which proxy orchestrator to query. Default `zarif`. Pass in the JSON request body for POST endpoints; URL query params are ignored. Responses: - **200** — Aggregated bandwidth rows from the orchestrator. Example response: ```json { "success": true, "rows": [ { "deviceId": "add02036-6754-4cf7-8432-cc28be4468f0", "connectionId": "wXyZ04827193", "bytesIn": 152841234, "bytesOut": 9183748, "bucket": "2026-04-25T13:00:00Z" } ] } ``` ### GET /api/v2/admin/devices/proxies — List all proxy connections (admin) Returns every proxy connection across all users, joined with the associated device's metadata and the orchestrator's bandwidth aggregates (24h + lifetime). Requires the admin API token (`X-Admin-Token`). `device` is `null` when the connection has no current association. Query parameters: - `backend` ("zarif" | "gost"): Which proxy orchestrator to read aggregates from (URL query string, since this endpoint is GET). Default `zarif`. Responses: - **200** — Connection list with associated device + bandwidth. - **400** — Unknown `backend` value. - **401** — Missing or invalid admin API token. - **502** — Could not reach the proxy orchestrator. - **503** — Proxy orchestrator / RMS not configured (env vars missing). Example response: ```json { "total": 2, "proxies": [ { "connection_id": "wXyZ04827193", "name": "client-A", "password": "p9F7bX3cVqLt", "created_at": "2026-04-25T13:22:11Z", "bytes_last_24h": 152841234, "bytes_lifetime": 9183748192, "device": { "remote_device_id": "add02036-6754-4cf7-8432-cc28be4468f0", "user_id": "65f3a1c0d4e2f10012abcd34", "name": "Atomic Mammal", "brand": "Samsung", "model": "SM-G998B", "country": "US", "online": true, "is_rented": false, "is_onMarket": false } }, { "connection_id": "aBcD19384756", "name": null, "password": "qK4LmPzWxR7a", "created_at": "2026-04-25T13:22:12Z", "bytes_last_24h": 0, "bytes_lifetime": 24576, "device": null } ] } ``` -------------------------------------------------------------------------------- ## Bucket Per-(job, device) JSON bucket — the persistent state automations attach to a single hire on a single device. The endpoints below are the **employer-facing** surface, gated by job ownership / share permission. They mirror the same model as the **Buckets** tab on the Job details page. (The agent surface used from inside automations is `agent.utils.bucket` — see the automation docs.) ### GET /api/v2/bucket/:job_id — List buckets for a job Returns a paginated list of every bucket attached to this job, with the device display name resolved for each. Path parameters: - `job_id` (string (ObjectId), required): Path param. The job's `job_id`. Caller must own the job, or have it shared to them (`view` for reads, `edit` for writes/deletes). Query parameters: - `page` (integer): 1-indexed page. Defaults to `1`. Example: 1 - `limit` (integer): Page size. Defaults to `20`. Capped at `100`. Example: 20 Responses: - **200** — Paginated list of buckets for this job (newest first by `updated_at`). Each entry is enriched with the device's display name. - **404** — Job not found, or caller does not own / does not have the required share permission (`view` for reads, `edit` for writes / deletes). { "error": "Job not found" } - **500** — Unexpected DB error. { "error": "Internal server error" } Example response: ```json { "buckets": [ { "_id": "65f1...", "job_id": "65aa...", "remote_device_id": "8b2f-uuid", "data": { "sessionToken": "abc123", "loggedIn": true }, "updated_at": "2026-04-15T08:14:00.000Z", "device_name": "Pixel 5 — Lab" } ], "total": 1, "page": 1, "pages": 1 } ``` ### DELETE /api/v2/bucket/:job_id — Delete all buckets for a job Wipes every bucket associated with this job. Caller must have `edit` access (owner or shared-edit). Path parameters: - `job_id` (string (ObjectId), required): Path param. The job's `job_id`. Caller must own the job, or have it shared to them (`view` for reads, `edit` for writes/deletes). Responses: - **200** — All buckets for this job removed. Returns the count deleted. { "success": true, "deleted": 12 } - **404** — Job not found, or caller does not own / does not have the required share permission (`view` for reads, `edit` for writes / deletes). { "error": "Job not found" } - **500** — Unexpected DB error. { "error": "Internal server error" } ### GET /api/v2/bucket/:job_id/:device_id — Read a single bucket Returns the bucket for this `(job_id, device_id)` pair. 404 when none exists or the job is inaccessible. Path parameters: - `job_id` (string (ObjectId), required): Path param. The job's `job_id`. Caller must own the job, or have it shared to them (`view` for reads, `edit` for writes/deletes). - `device_id` (string (UUID), required): Path param. Device's `remote_device_id`. Responses: - **200** — Returns the bucket document for this `(job_id, device_id)` pair. - **404** — Either the job is inaccessible to the caller, or no bucket exists for this device. { "error": "Bucket not found" } - **500** — Unexpected DB error. { "error": "Internal server error" } Example response: ```json { "bucket": { "_id": "65f1...", "job_id": "65aa...", "remote_device_id": "8b2f-uuid", "data": { "sessionToken": "abc123", "loggedIn": true }, "updated_at": "2026-04-15T08:14:00.000Z" } } ``` ### PUT /api/v2/bucket/:job_id/:device_id — Write a bucket Replaces the bucket payload for this `(job_id, device_id)` pair (no merge — `data` becomes the new bucket). Validated against the automation's `bucket_schema` when one is defined. Caller must have `edit` access. Path parameters: - `job_id` (string (ObjectId), required): Path param. The job's `job_id`. Caller must own the job, or have it shared to them (`view` for reads, `edit` for writes/deletes). - `device_id` (string (UUID), required): Path param. Device's `remote_device_id`. Request body: - `data` (object, required): New bucket payload. Replaces the existing object (no merge). If the linked automation defines a `bucket_schema`, `data` is validated against it. Responses: - **200** — Bucket created or replaced for this `(job_id, device_id)` pair. Returns the saved document. - **400** — `data` missing / not an object, or schema validation against the automation's `bucket_schema` failed. { "error": "Invalid bucket data" } - **404** — Job not found, or caller does not own / does not have the required share permission (`view` for reads, `edit` for writes / deletes). { "error": "Job not found" } - **500** — Unexpected DB error. { "error": "Internal server error" } ### DELETE /api/v2/bucket/:job_id/:device_id — Delete a bucket Removes the bucket for this `(job_id, device_id)` pair. Caller must have `edit` access. Path parameters: - `job_id` (string (ObjectId), required): Path param. The job's `job_id`. Caller must own the job, or have it shared to them (`view` for reads, `edit` for writes/deletes). - `device_id` (string (UUID), required): Path param. Device's `remote_device_id`. Responses: - **200** — Bucket for this `(job_id, device_id)` pair removed. { "success": true } - **404** — Job not found, or caller does not own / does not have the required share permission (`view` for reads, `edit` for writes / deletes). { "error": "Job not found" } - **500** — Unexpected DB error. { "error": "Internal server error" } -------------------------------------------------------------------------------- ## Device Bucket Per-device JSON bucket, shared across every job and automation running on the device. Useful for cross-job state — login cookies, account tokens, app config. Distinct from the per-job bucket: there is no `bucket_schema` validation, and no job context is required, so direct-run automations can use it too. The agent calls these via `agent.utils.deviceBucket`. ### POST /api/v2/device-bucket — Read device bucket Reads the device-scoped bucket. The `remote_device_id` is taken from the token when it carries a device context; non-agent callers must supply it in the body. Request body: - `remote_device_id` (string (UUID)): The device's `remote_device_id`. Optional when the token already carries a device context (agent tokens). Required for plain Client API tokens. Must point to a device the caller owns or is currently running on. Responses: - **200** — Returns the device-scoped bucket. `deviceBucket` is `{}` on first read. - **400** — Token has no device context and `remote_device_id` was not supplied. { "error": "Device bucket requires a device context (remote_device_id)" } Example response: ```json { "deviceBucket": { "accountToken": "xyz", "username": "user@example.com" } } ``` ### POST /api/v2/device-bucket/set — Merge into device bucket Merges the supplied object into the existing device bucket. Existing keys not in the payload are preserved. Request body: - `remote_device_id` (string (UUID)): The device's `remote_device_id`. Optional when the token already carries a device context (agent tokens). Required for plain Client API tokens. Must point to a device the caller owns or is currently running on. - `data` (object, required): Non-null, non-array JSON object. Top-level keys are merged with the stored bucket — existing keys not in the payload are preserved. Responses: - **200** — Bucket merged and persisted. Returns the full merged object. - **400** — Missing device context or `data` is missing / null / an array / not an object. { "error": "Invalid data. Must be a non-null object." } Example response: ```json { "success": true, "deviceBucket": { "accountToken": "xyz", "username": "user@example.com" } } ``` -------------------------------------------------------------------------------- ## Other File upload and temporary file access endpoints. ### POST /api/v2/files/upload — Upload temporary file Uploads a file to temporary storage. Files are automatically deleted after 15 minutes. Maximum file size is 50 MB. Use multipart/form-data with 'file' field name. Request body: - `file` (file, required): File to upload (multipart/form-data) Example: Binary file data Responses: - **200** — File uploaded successfully { "success": true, "message": "File uploaded successfully", "data": { "filename": "1234567890_example.pdf", "originalName": "example.pdf", "size": 1024000, "url": "https://api.example.com/temp/1234567890_example.pdf", "expiresAt": "2025-10-08T12:15:00.000Z" } } - **400** — No file uploaded { "success": false, "message": "No file uploaded" } - **401** — Unauthorized or missing token Authentication error message - **500** — Server error during upload { "success": false, "message": "Error uploading file", "error": "Error details" } Example response: ```json { "success": true, "message": "File uploaded successfully", "data": { "filename": "1234567890_example.pdf", "originalName": "example.pdf", "size": 1024000, "url": "https://api.example.com/temp/1234567890_example.pdf", "expiresAt": "2025-10-08T12:15:00.000Z" } } ``` ### GET /api/v2/files/temp/:filename — Get uploaded file Retrieves a previously uploaded temporary file. Files expire after 15 minutes from upload time. Path parameters: - `filename` (string, required): Filename returned from the upload endpoint (path parameter) Example: 1234567890_example.pdf Responses: - **200** — File retrieved successfully Binary file data - **400** — Filename is required { "success": false, "message": "Filename is required" } - **404** — File not found or expired { "success": false, "message": "File not found or expired" } - **401** — Unauthorized or missing token Authentication error message - **500** — Server error during retrieval { "success": false, "message": "Error retrieving file", "error": "Error details" } -------------------------------------------------------------------------------- # Automation API ## Guide / Automation API Documentation Path: /docs/automation Description: Complete reference for automating Android devices with the agent SDK. Content: lucide-react next Automation docs example.ts Automation API Documentation Complete reference for automating Android devices. Build powerful automation scripts with full access to device controls, screen content, file operations, and more. book Guide Learn how to build automations with step-by-step tutorials and best practices. /docs/automation/guide code Reference Complete API reference for all interfaces, methods, types, and constants. /docs/automation/reference Quick Links Agent Actions All automation actions: tap, swipe, screenshot, screenContent, launchApp, and more /docs/automation/reference/agent/actions AndroidNode Working with the accessibility tree - properties and methods for finding and interacting with UI elements /docs/automation/reference/android-node AndroidNodeFilter Builder pattern for complex node queries - isButton(), hasText(), isClickable(), and more /docs/automation/reference/android-node/filter File Operations Read, write, list, and manage files on the device /docs/automation/reference/agent/utils Getting Started The automation API is accessed through the global agent object. Here's a simple example: // Get the current screen content const screen = await agent.actions.screenContent(); // Find a button with text "Submit" const submitBtn = screen.findTextOne("Submit"); // Tap the button if (submitBtn) { const { left, top, right, bottom } = submitBtn.boundsInScreen; await agent.actions.tap( (left + right) / 2, (top + bottom) / 2 ); } -------------------------------------------------------------------------------- ## Guide / Overview Path: /docs/automation/guide Description: Learn how to build automations with tutorials and best practices Content: next next/link Automation guide underline Guide Learn how to build automations with tutorials and best practices This guide walks you through everything you need to know to create powerful Android automations. From setting up your first project to building production-ready scripts with proper error handling. Getting Started Create your first automation project and learn the IDE /docs/automation/guide/getting-started Configuration Set up parameters, job variables, requirements, and sharing /docs/automation/guide/configuration Core Concepts Writing Scripts TypeScript basics, Agent API, and common patterns /docs/automation/guide/writing-scripts Screen States Detect and respond to different UI states /docs/automation/guide/screen-states Stages Organize automation into phases with step limits /docs/automation/guide/stages Task Submission Submit results, collect data, and handle job tasks /docs/automation/guide/tasks Error Handling Handle crashes, dialogs, network issues, and recovery /docs/automation/guide/error-handling Running & Tutorial Running Automations Execute automations on devices and collect results /docs/automation/guide/running Full Tutorial: MySocial Auto-Responder Complete example with stages, screen states, and data collection /docs/automation/guide/tutorial Looking for API details? Check out the API Reference /docs/automation/reference for complete documentation of all available methods, types, and constants. -------------------------------------------------------------------------------- ## Guide / Getting Started Path: /docs/automation/guide/getting-started Description: Create your first automation project and learn the IDE Sections: - Creating a New Project - The IDE - Code Editor - File Explorer - Git Integration - Options Panel - Project Structure - Your First Automation - Keyboard Shortcuts - TypeScript Compilation - ES6 Imports - Next Steps Content: Creating a New Project To create a new automation project: Make sure you are logged in Navigate to in your dashboard Click the New Automation button Enter a project name and optional description Click Create Project main.ts file as the entry point. This is where your automation code will start executing. The IDE The automation IDE provides a full development environment with: Code Editor Full TypeScript support with syntax highlighting, autocomplete, and error checking. The editor automatically compiles TypeScript to JavaScript on save. File Explorer Manage your project files and folders. Create, rename, and delete files. Organize your code across multiple TypeScript files with ES6 imports. Git Integration Built-in version control with commit history, diff viewer, and revert functionality. Track changes and roll back to previous versions when needed. Options Panel Configuration Project Structure A typical automation project structure: Your First Automation Here's a simple automation that launches an app and takes a screenshot: Keyboard Shortcuts Shortcut Action Ctrl/Cmd + S Save current file Ctrl/Cmd + Space Trigger autocomplete Ctrl/Cmd + / Toggle line comment Ctrl/Cmd + F Find in file TypeScript Compilation When you save a .ts .js file is what actually runs on the device. If your TypeScript has errors, they'll be displayed when you save. Fix the errors and save again to update the compiled JavaScript. ES6 Imports Organize your code across multiple files using ES6 imports: Always use extension in imports, even when importing from files. This is because the runtime executes the compiled JavaScript. Next Steps next next/link Getting started · Automation Getting Started Create your first automation project and learn the IDE -> /docs/automation/guide/configuration text my-automation/ ├── main.ts # Entry point (required) ├── stages.ts # Stage enum definitions ├── screenStates.ts # Screen state enum ├── detection.ts # Screen detection logic ├── handlers.ts # Stage handlers └── utils.ts # Shared utilities // Simple automation example const PACKAGE_NAME = "com.example.myapp"; async function main() { try { // Launch the app await agent.actions.launchApp(PACKAGE_NAME); // Wait for app to load await sleep(3000); // Get screen content const screen = await agent.actions.screenContent(); // Find all text on screen const allNodes = getAllNodes(screen); const textNodes = allNodes.filter(node => node.text); console.log("Found text nodes:", textNodes.length); // Take a screenshot const screenshot = await agent.actions.screenshot(1080, 1920, 80); console.log("Screenshot taken!"); // Submit success await agent.utils.job.submitTask("success", { textNodesFound: textNodes.length }); } catch (error) { console.error("Automation failed:", error); await agent.utils.job.submitTask("failed", { error: String(error) }); } finally { // Always stop the automation stopCurrentAutomation(); } } // Helper function for delays function sleep(ms: number) { return new Promise(resolve => setTimeout(resolve, ms)); } // Start the automation main(); warning Compilation Errors stages.ts // Export enum from stages.ts export enum Stage { Initialize = "Initialize", LaunchApp = "LaunchApp", ProcessData = "ProcessData", Complete = "Complete", } // Import in main.ts import { Stage } from "./stages.js"; let currentStage = Stage.Initialize; // Use the imported enum if (currentStage === Stage.Initialize) { // ... } info Import Extension Configuration → Set up parameters, job variables, and sharing Writing Scripts → Learn the Agent API and common patterns /docs/automation/guide/writing-scripts -------------------------------------------------------------------------------- ## Guide / Configuration Path: /docs/automation/guide/configuration Description: Set up parameters, job variables, requirements, and sharing Sections: - Metadata - Requirements - Minimum Android Version - Minimum App Version - Automation Parameters - Supported Field Types - Job Variables - Parameters vs Job Variables - Sharing - Execution Access - Editor Access - Logs Access - Managing Access - Auto-Generated Types - Type Safety - Next Steps Content: Access configuration options through the Options panel on the right side of the IDE. These settings control how your automation behaves and who can access it. Metadata Basic information about your automation: Field Description Limits Name Display name for your automation Max 225 characters Explain what the automation does Max 10,100 characters Icon Visual identifier for the automation 512x512 WebP, max 2MB Requirements Set minimum version requirements to ensure compatibility: Minimum Android Version The lowest Android version your automation supports. Options range from Android 7.0 (API 24) to Android 16 (API 36). Tip: Some features like dpad() inputKey() require Android 13+. Minimum App Version The minimum version of the Xgodo app required on the device. Select from available versions sorted by version code. Automation Parameters Define configuration options that users can set when running your automation. These are static values set once before execution. Supported Field Types Type Validations Use Case string minLength, maxLength, pattern, enum Text input, selections number min, max, integer Counts, limits, delays boolean Feature toggles array minItems, maxItems, item schema Lists of values object Nested properties Complex configs Flexible input Job Variables Define runtime variables provided for each job task. Unlike automation parameters, job variables can change between tasks in the same job. Parameters vs Job Variables Automation Parameters: Set once, same for all tasks (e.g., retry count, feature flags) in a job. Job Variables: Different per task (e.g., account credentials, target data) Sharing Control who can access your automation. Only project owners can manage sharing settings. Execution Access Users who can and execute it. Editor Access edit the automation's code. They have full access to the IDE and can modify files. Logs Access view logs from automation runs. Useful for debugging and monitoring without edit access. Managing Access For each sharing type: Open the section in Options panel Enter a username in the input field Click Add or press Enter To remove access, click the remove button next to a user Auto-Generated Types When you define parameter schemas, TypeScript types are automatically generated. These provide full autocomplete and type checking in the IDE. Type Safety The IDE will show errors if you access non-existent properties or use wrong types. This helps catch bugs before running the automation. Next Steps next Configuration · Automation Configuration Set up parameters, job variables, requirements, and sharing Example: Automation Parameters Schema // Accessible via agent.arguments.automationParameters interface AutomationParameters { targetUsername: string; // Required string maxRetries: number; // Number with validation enableNotifications: boolean; // Toggle option messageTemplate: string; // Text with default value } Accessing Parameters in Code // Access automation parameters const { targetUsername, maxRetries, enableNotifications } = agent.arguments.automationParameters; console.log("Target:", targetUsername); console.log("Max retries:", maxRetries); if (enableNotifications) { await agent.actions.showNotification("Started", "Automation is running"); } Example: Job Variables Schema // Accessible via agent.arguments.jobVariables interface JobVariables { email: string; // Account to process password: string; // Credentials proxyUrl?: string; // Optional proxy } Accessing Job Variables // Get current task data const task = await agent.utils.job.getCurrentTask(); if (task.success) { const { email, password } = agent.arguments.jobVariables; // Use the job variables await processAccount(email, password); // Submit results await agent.utils.job.submitTask("success", { email, processedAt: new Date().toISOString() }); // Stop the automation stopCurrentAutomation(); } info -> Generated Type Definitions // Auto-generated from your schema interface AutomationParameters { /** Target user to process */ targetUsername: string; /** Maximum retry attempts */ maxRetries: number; /** Enable push notifications */ enableNotifications: boolean; } interface JobVariables { /** Account email */ email: string; /** Account password */ password: string; } interface AgentArguments { automationParameters: AutomationParameters; jobVariables: JobVariables; } // Available on agent object interface Agent { arguments: AgentArguments; // ... other properties } success Writing Scripts → Learn the Agent API and common patterns /docs/automation/guide/writing-scripts API Reference → Complete documentation of all available methods /docs/automation/reference -------------------------------------------------------------------------------- ## Guide / Writing Scripts Path: /docs/automation/guide/writing-scripts Description: TypeScript basics, Agent API, and common patterns Sections: - The Agent Object - Common Actions - Touch Gestures - Text Input - Navigation - App Management - Screen Content - Helper Functions - Interacting with Nodes - Screenshots - Common Patterns - Sleep/Wait Function - Wait for Screen State - Retry with Backoff - Logging - File Operations - Network Monitoring - Best Practices - Use Random Delays - Use randomClick for Taps - Prefer performAction over tap - Handle Unknown Screens - Next Steps Content: agent object which provides all the APIs for interacting with the device. The Agent Object The object is globally available and provides: Common Actions Touch Gestures Text Input Navigation App Management Screen Content Get the current screen content as an accessibility tree: Helper Functions These helper functions are globally available: Interacting with Nodes Screenshots Common Patterns Sleep/Wait Function Wait for Screen State boolean, timeout: number = 10000 ): Promise Retry with Backoff Promise , maxAttempts: number = 3, baseDelay: number = 1000 ): Promise Logging File Operations Network Monitoring Best Practices Use Random Delays sleepRandom(1000, 2000) instead of fixed delays. Use randomClick for Taps node.randomClick() to tap at random positions within the element bounds. Prefer performAction over tap Use node.performAction() with accessibility actions for more reliable interactions, especially for buttons and inputs. Handle Unknown Screens Always have fallback logic for screens you don't recognize. Log unknown states and retry after a short delay. Next Steps next Writing scripts · Automation Writing Scripts TypeScript basics, Agent API, and common patterns agent = { actions: { ... }, // Device interactions (tap, swipe, type, etc.) utils: { ... }, // Utilities (random helpers, job tasks, files) info: { ... }, // Device information control: { ... }, // Automation control (pause, delay) display: { ... }, // Display settings email: { ... }, // Email utilities notifications: { ... }, // Notification callbacks constants: { ... }, // Action constants arguments: { ... }, // Parameters and job variables } Tapping and Swiping // Simple tap at coordinates await agent.actions.tap(500, 1000); // Swipe from point A to B over 500ms await agent.actions.swipe(500, 1500, 500, 500, 500); // Long press for 2 seconds await agent.actions.hold(500, 1000, 2000); // Double tap with 100ms interval await agent.actions.doubleTap(500, 1000, 100); // Human-like random tap within node bounds const button = screen.findTextOne("Submit"); if (button) { button.randomClick(); } Typing and Clipboard Navigation Actions // System navigation await agent.actions.goBack(); await agent.actions.goHome(); await agent.actions.recents(); // D-pad navigation (Android 13+ only) await agent.actions.dpad("down"); await agent.actions.dpad("center"); Launching Apps // Launch app by package name await agent.actions.launchApp("com.example.myapp"); // Launch fresh (tries closing the existing app first) await agent.actions.launchApp("com.example.myapp", true); // Open URL on in-app browser await agent.actions.browse("https://example.com"); // List all installed apps const apps = await agent.actions.listApps(); console.log(apps["com.android.chrome"]); // "Chrome" Reading Screen Content // Get current screen content const screen = await agent.actions.screenContent(); // Get all nodes recursively const allNodes = getAllNodes(screen); // Find nodes by text const buttons = allNodes.filter(node => node.text?.toLowerCase().includes("submit") ); // Find nodes by ID const loginBtn = allNodes.find(node => node.viewId === "com.example:id/login_button" ); // Find nodes by class const editTexts = allNodes.filter(node => node.className === "android.widget.EditText" ); Node Helper Functions // Get all descendant nodes recursively const allNodes = getAllNodes(screen); // Find nodes by viewId (resourceId) const nodes = findNodesById(screen, "com.example:id/button"); // Find nodes by exact text const textNodes = findNodesByText(screen, "Submit"); // Check if node has specific text const hasText = nodeHasText(node, "Continue"); Node Interactions // Find a button and tap its center const button = allNodes.find(n => n.text === "Submit" && n.clickable); if (button) { const { left, top, right, bottom } = button.boundsInScreen; const centerX = (left + right) / 2; const centerY = (top + bottom) / 2; await agent.actions.tap(centerX, centerY); } // Or use accessibility action for more reliable clicks await button.performAction(agent.constants.ACTION_CLICK); // Scroll a node await scrollableNode.performAction(agent.constants.ACTION_SCROLL_FORWARD); // Focus an input field await inputField.performAction(agent.constants.ACTION_FOCUS); Taking Screenshots // Take a screenshot (maxWidth, maxHeight, quality) const screenshot = await agent.actions.screenshot(1080, 1920, 80); // Result contains base64 image data const { base64, width, height } = screenshot; // Use for debugging or storing console.log(`Screenshot: ${width}x${height}`); Delay Utilities // Simple sleep function function sleep(ms: number): Promise { return new Promise(resolve => setTimeout(resolve, ms)); } // Sleep with random range (more human-like) function sleepRandom(min: number, max: number): Promise { const ms = Math.floor(Math.random() * (max - min) + min); return new Promise(resolve => setTimeout(resolve, ms)); } // Usage await sleep(2000); // Wait 2 seconds await sleepRandom(1000, 3000); // Wait 1-3 seconds randomly Waiting for Conditions // Wait for a specific element to appear async function waitForElement( condition: (nodes: AndroidNode[]) => boolean, timeout: number = 10000 ): Promise { const startTime = Date.now(); while (Date.now() - startTime < timeout) { const screen = await agent.actions.screenContent(); const allNodes = getAllNodes(screen); if (condition(allNodes)) { return true; } await sleep(500); } return false; } // Usage: Wait for "Home" text to appear const found = await waitForElement(nodes => nodes.some(n => n.text === "Home") ); Retry Pattern async function withRetry( fn: () => Promise, maxAttempts: number = 3, baseDelay: number = 1000 ): Promise { let lastError: Error | undefined; for (let attempt = 1; attempt <= maxAttempts; attempt++) { try { return await fn(); } catch (error) { lastError = error as Error; console.log(`Attempt ${attempt} failed: ${lastError.message}`); if (attempt < maxAttempts) { const delay = baseDelay * Math.pow(2, attempt - 1); await sleep(delay); } } } throw lastError; } Console Logging // Standard console methods are available console.log("Info message"); console.warn("Warning message"); console.error("Error message"); // Log objects console.log("Screen content:", { nodeCount: allNodes.length }); // Debug with context console.log(`[Stage: ${currentStage}] Processing screen...`); Reading and Writing Files // Check if file exists const exists = agent.utils.files.exists("/sdcard/Download/data.json"); // Read file content const content = agent.utils.files.readFullFile("/sdcard/Download/data.json"); const data = JSON.parse(content); // List directory const files = agent.utils.files.list("/sdcard/Download"); for (const file of files) { console.log(file.name, file.isDirectory); } // Save file to device await agent.actions.saveFile("result.json", JSON.stringify(data)); Network Callback // Track network connectivity let isOnline = true; agent.utils.setNetworkCallback((networkAvailable) => { isOnline = networkAvailable; if (!networkAvailable) { console.warn("Network connection lost!"); } }); // Check before network operations if (!isOnline) { console.log("Waiting for network..."); await sleep(5000); } // Refresh mobile IP (airplane mode toggle) await agent.actions.airplane(); success Screen States → Detect and respond to different UI states /docs/automation/guide/screen-states Stages → Organize automation into phases with step limits /docs/automation/guide/stages -------------------------------------------------------------------------------- ## Guide / Screen States Path: /docs/automation/guide/screen-states Description: Detect and respond to different UI states Sections: - The ScreenState Pattern - Detection Strategies - 1. Text-Based Detection - 2. ID-Based Detection - 3. Composite Detection - 4. Package Name Filtering - Complete Detection Function - Handling Unknown States - Stage-Specific Detection - Best Practices - Check System States First - Use Multiple Conditions - Order Matters - Filter by Package Name - Store Unknown Screens - Next Steps Content: Screen state detection is the foundation of robust automation. By identifying what's currently on screen, your automation can respond appropriately to each situation. The ScreenState Pattern Define all possible screen states as an enum: Detection Strategies 1. Text-Based Detection The simplest approach - look for specific text on screen: 2. ID-Based Detection More reliable - use resource IDs which don't change with language: 3. Composite Detection Combine multiple conditions for accuracy: 4. Package Name Filtering Filter by app package to avoid false positives from system UI: Complete Detection Function Handling Unknown States Unknown screens will happen. Handle them gracefully: Stage-Specific Detection Different stages may need different detection logic: Best Practices Check System States First Always check for crashes, dialogs, and system UI before app-specific states. These can appear at any time and need immediate handling. Use Multiple Conditions Don't rely on a single text match. Combine multiple checks (text + ID + class + package) for reliable detection. Order Matters Check more specific states before general ones. For example, check "LoginEnterPassword" before "LoginScreen". Filter by Package Name Use package name filtering to avoid false positives from system UI, notifications, or other apps. Store Unknown Screens Use agent.utils.outOfSteps.storeScreen() for unknown states. This helps debug what screens you missed. Next Steps next Screen states · Automation Screen States Detect and respond to different UI states screenStates.ts export enum ScreenState { // System states Unknown = "Unknown", Crash = "Crash", PhoneDialog = "PhoneDialog", NoInternet = "NoInternet", NotificationShade = "NotificationShade", // App states SplashScreen = "SplashScreen", LoginScreen = "LoginScreen", LoginEnterPassword = "LoginEnterPassword", HomeScreen = "HomeScreen", ProfileScreen = "ProfileScreen", SettingsScreen = "SettingsScreen", // Dialog states PermissionDialog = "PermissionDialog", ConfirmDialog = "ConfirmDialog", ErrorDialog = "ErrorDialog", // Loading states Loading = "Loading", // Success/failure states Success = "Success", RateLimited = "RateLimited", } Text-based detection function detectScreenState(screen: AndroidNode): ScreenState { const allNodes = getAllNodes(screen); // Check for specific text if (allNodes.find(n => n.text === "Sign in")) { return ScreenState.LoginScreen; } if (allNodes.find(n => n.text === "Enter your password")) { return ScreenState.LoginEnterPassword; } if (allNodes.find(n => n.text?.toLowerCase() === "home")) { return ScreenState.HomeScreen; } return ScreenState.Unknown; } ID-based detection function detectScreenState(screen: AndroidNode): ScreenState { const allNodes = getAllNodes(screen); // Check for specific view IDs if (findNodesById(screen, "com.example:id/login_form").length) { return ScreenState.LoginScreen; } if (findNodesById(screen, "com.example:id/home_feed").length) { return ScreenState.HomeScreen; } if (findNodesById(screen, "android:id/aerr_close").find(n => n.clickable)) { return ScreenState.Crash; } return ScreenState.Unknown; } Composite detection function detectScreenState(screen: AndroidNode): ScreenState { const allNodes = getAllNodes(screen); // Login screen: has email field AND sign-in button const hasEmailField = allNodes.find(n => n.className === "android.widget.EditText" && (n.hintText?.toLowerCase()?.includes("email") || n.viewId?.includes("email")) ); const hasSignInButton = allNodes.find(n => n.clickable && n.text?.toLowerCase() === "sign in" ); if (hasEmailField && hasSignInButton) { return ScreenState.LoginScreen; } // Password screen: has password field (isPassword = true) const hasPasswordField = allNodes.find(n => n.className === "android.widget.EditText" && n.isPassword ); if (hasPasswordField && !hasEmailField) { return ScreenState.LoginEnterPassword; } return ScreenState.Unknown; } Package name filtering const APP_PACKAGE = "com.example.myapp"; function detectScreenState(screen: AndroidNode): ScreenState { const allNodes = getAllNodes(screen); // Only consider nodes from our target app const appNodes = allNodes.filter(n => n.packageName === APP_PACKAGE); // Check for phone dialog (different package) if (allNodes.every(n => n.packageName === "com.android.phone")) { return ScreenState.PhoneDialog; } // Check for system crash dialog if (allNodes.find(n => n.viewId === "android:id/aerr_close" && n.packageName === "android" )) { return ScreenState.Crash; } // Now check app-specific screens if (appNodes.find(n => n.viewId?.includes("login"))) { return ScreenState.LoginScreen; } return ScreenState.Unknown; } detection.ts import { ScreenState } from "./screenStates.js"; const APP_PACKAGE = "com.mysocial.app"; export function detectScreenState(screen: AndroidNode): ScreenState { const allNodes = getAllNodes(screen); // === SYSTEM STATES (check first) === // Crash dialog if (findNodesById(screen, "android:id/aerr_close").find(n => n.clickable)) { return ScreenState.Crash; } // Phone dialog if (allNodes.every(n => n.packageName === "com.android.phone")) { return ScreenState.PhoneDialog; } // No internet if (allNodes.find(n => n.text?.toLowerCase()?.includes("no internet") || n.text?.toLowerCase()?.includes("you're offline") )) { return ScreenState.NoInternet; } // Loading (only loading indicator visible) if (allNodes.length <= 3 && allNodes.find(n => n.className?.includes("ProgressBar") )) { return ScreenState.Loading; } // === APP-SPECIFIC STATES === // Login screen if (allNodes.find(n => n.className === "android.widget.EditText" && n.hintText?.toLowerCase()?.includes("email") && n.packageName === APP_PACKAGE )) { return ScreenState.LoginScreen; } // Password entry if (allNodes.find(n => n.isPassword && n.packageName === APP_PACKAGE )) { return ScreenState.LoginEnterPassword; } // Home screen (has bottom navigation with Home selected) const homeTab = allNodes.find(n => n.description === "Home" && n.isSelected && n.packageName === APP_PACKAGE ); if (homeTab) { return ScreenState.HomeScreen; } // Profile screen if (allNodes.find(n => n.description === "Profile" && n.isSelected && n.packageName === APP_PACKAGE )) { return ScreenState.ProfileScreen; } // Permission dialog if (allNodes.find(n => n.packageName === "com.android.permissioncontroller" || (n.text?.toLowerCase()?.includes("allow") && n.clickable) )) { return ScreenState.PermissionDialog; } return ScreenState.Unknown; } Unknown state handling let unknownScreenCount = 0; const MAX_UNKNOWN_SCREENS = 3; async function getCurrentScreenState(): Promise<{ state: ScreenState; screen: AndroidNode; }> { let screen = await agent.actions.screenContent(); let state = detectScreenState(screen); // Retry a few times if unknown if (state === ScreenState.Unknown) { for (let i = 0; i < 3; i++) { await sleep(2000); screen = await agent.actions.screenContent(); state = detectScreenState(screen); if (state !== ScreenState.Unknown) { unknownScreenCount = 0; break; } } } // Track consecutive unknown screens if (state === ScreenState.Unknown) { unknownScreenCount++; console.warn(`Unknown screen #${unknownScreenCount}`); // Store for debugging await agent.utils.outOfSteps.storeScreen( screen, "unknown", "Unknown", MAX_UNKNOWN_SCREENS - unknownScreenCount, ScreenshotRecord.HIGH_QUALITY ); if (unknownScreenCount >= MAX_UNKNOWN_SCREENS) { throw new Error("Too many unknown screens"); } } else { unknownScreenCount = 0; } return { state, screen }; } Stage-aware detection import { Stage } from "./stages.js"; import { ScreenState } from "./screenStates.js"; export function detectScreenState( screen: AndroidNode, currentStage: Stage ): ScreenState { // Always check system states first const systemState = detectSystemStates(screen); if (systemState !== ScreenState.Unknown) { return systemState; } // Stage-specific detection switch (currentStage) { case Stage.Login: return detectLoginStates(screen); case Stage.NavigateToMessages: case Stage.ProcessMessages: return detectMessageStates(screen); case Stage.NavigateToFeed: case Stage.LikePosts: return detectFeedStates(screen); default: return detectCommonStates(screen); } } function detectSystemStates(screen: AndroidNode): ScreenState { // ... crash, phone dialog, no internet } function detectLoginStates(screen: AndroidNode): ScreenState { // ... login-specific screens } function detectMessageStates(screen: AndroidNode): ScreenState { // ... message-specific screens } success Stages → Organize automation into phases with step limits /docs/automation/guide/stages Error Handling → Handle crashes, dialogs, and recovery /docs/automation/guide/error-handling -------------------------------------------------------------------------------- ## Guide / Stages Path: /docs/automation/guide/stages Description: Organize automation into phases with step limits Sections: - The Stage Pattern - Stage State Management - Max Steps Per Stage - Stage Transitions - Out of Steps Handling - Dynamic Step Limits - Same Screen Detection - Complete Example - Best Practices - Reset Step Counter on Stage Change - Report Progress on Stage Change - Store Screens Continuously - Detect Stuck States - Next Steps Content: Stages help organize complex automations into logical phases. Each stage has its own step limit, making it easier to debug issues and track progress. The Stage Pattern Define your automation stages as an enum: Stage State Management Track the current stage and step count: Max Steps Per Stage Step limits prevent infinite loops and help identify stuck automations: Stage Transitions Transition between stages based on progress: Out of Steps Handling When the automation runs out of steps, submit data for analysis: Dynamic Step Limits Some stages may need more steps than others: Same Screen Detection Detect when stuck on the same screen: Complete Example Best Practices Reset Step Counter on Stage Change Always reset maxSteps when entering a new stage. This gives each phase its own budget of steps. Report Progress on Stage Change Call submitTask("running", ...) when changing stages. This lets you track progress in the dashboard. Store Screens Continuously storeScreen() on every iteration. This creates a trail for debugging when things go wrong. Detect Stuck States Track same-screen counts and implement recovery logic. Don't let the automation spin on the same screen forever. Next Steps next Stages · Automation Stages Organize automation into phases with step limits stages.ts export enum Stage { Initialize = "Initialize", LaunchApp = "LaunchApp", HandleLogin = "HandleLogin", NavigateToMessages = "NavigateToMessages", SelectUnreadChat = "SelectUnreadChat", ProcessMessages = "ProcessMessages", NavigateToFeed = "NavigateToFeed", LikePosts = "LikePosts", Complete = "Complete", } Stage tracking import { Stage } from "./stages.js"; const MAX_STEPS_PER_STAGE = 48; let currentStage: Stage = Stage.Initialize; let maxSteps = MAX_STEPS_PER_STAGE; async function setCurrentStage(newStage: Stage) { currentStage = newStage; maxSteps = MAX_STEPS_PER_STAGE; // Reset step counter console.log(`[Stage] Entering: ${newStage}`); // Report progress to server (files ignored when finish=false) await agent.utils.job.submitTask( "running", { stage: newStage, timestamp: Date.now() }, false // Don't finish the task ); } Main loop with step counting async function runAutomation() { await setCurrentStage(Stage.Initialize); // Main automation loop while (maxSteps-- > 0) { const { state, screen } = await getCurrentScreenState(); // Store screen for debugging await agent.utils.outOfSteps.storeScreen( screen, currentStage, state, maxSteps, state === ScreenState.Unknown ? ScreenshotRecord.HIGH_QUALITY : ScreenshotRecord.LOW_QUALITY ); // Handle the current screen state await handleScreenState(state, screen); // Check if we've completed if (currentStage === Stage.Complete) { break; } } // Out of steps - submit for analysis if (maxSteps < 0) { await agent.utils.outOfSteps.submit("outOfSteps"); throw new Error("OUT_OF_STEPS"); } } Stage handler with transitions async function handleScreenState( state: ScreenState, screen: AndroidNode ) { // Handle system states first (any stage) if (await handleSystemStates(state, screen)) { return; } // Stage-specific handling switch (currentStage) { case Stage.Initialize: await handleInitialize(state, screen); break; case Stage.LaunchApp: await handleLaunchApp(state, screen); break; case Stage.HandleLogin: await handleLogin(state, screen); break; case Stage.NavigateToMessages: await handleNavigateToMessages(state, screen); break; // ... other stages } } async function handleInitialize(state: ScreenState, screen: AndroidNode) { // Check if app is installed const apps = await agent.actions.listApps(); if (!apps[APP_PACKAGE]) { throw new Error("App not installed"); } // Set up callbacks agent.utils.setNetworkCallback(handleNetworkChange); // Move to next stage await setCurrentStage(Stage.LaunchApp); } async function handleLaunchApp(state: ScreenState, screen: AndroidNode) { if (state === ScreenState.HomeScreen) { // Already on home screen, skip to messages await setCurrentStage(Stage.NavigateToMessages); return; } if (state === ScreenState.SplashScreen || state === ScreenState.Loading) { // Wait for app to load await sleep(2000); return; } if (state === ScreenState.LoginScreen) { // Need to login first await setCurrentStage(Stage.HandleLogin); return; } // Launch the app await agent.actions.launchApp(APP_PACKAGE); await sleep(3000); } Out of steps submission // Store screens during automation await agent.utils.outOfSteps.storeScreen( screen, currentStage, // Which stage screenState, // What screen state maxSteps, // Remaining steps ScreenshotRecord.LOW_QUALITY // Screenshot quality ); // Submit when out of steps if (maxSteps < 0) { const result = await agent.utils.outOfSteps.submit("outOfSteps"); if (result.success) { console.log("Out of steps report ID:", result.id); } // Fail the task await agent.utils.job.submitTask("failed", { reason: "OUT_OF_STEPS", stage: currentStage, outOfStepsId: result.success ? result.id : null }); // Always stop the automation stopCurrentAutomation(); } Dynamic step limits function getMaxStepsForStage(stage: Stage): number { switch (stage) { // Login might need more steps (captchas, 2FA, etc.) case Stage.HandleLogin: return 100; // Message processing depends on conversation length case Stage.ProcessMessages: return 60; // Most stages need fewer steps default: return 48; } } async function setCurrentStage(newStage: Stage) { currentStage = newStage; maxSteps = getMaxStepsForStage(newStage); console.log(`[Stage] ${newStage} (max steps: ${maxSteps})`); await agent.utils.job.submitTask( "running", { stage: newStage }, false ); } Same screen detection let lastScreenState: ScreenState | null = null; let sameScreenCount = 0; const MAX_SAME_SCREEN = 5; async function checkSameScreen(state: ScreenState) { if (state === lastScreenState) { sameScreenCount++; if (sameScreenCount >= MAX_SAME_SCREEN) { console.warn(`Stuck on ${state} for ${sameScreenCount} iterations`); // Try recovery actions if (state === ScreenState.Loading) { // Wait longer for loading await sleep(5000); } else { // Try going back await agent.actions.goBack(); await sleep(1000); } // Reset counter after recovery attempt sameScreenCount = 0; } } else { lastScreenState = state; sameScreenCount = 0; } } Full stage management import { Stage } from "./stages.js"; import { ScreenState } from "./screenStates.js"; import { detectScreenState } from "./detection.js"; const MAX_STEPS_PER_STAGE = 48; let currentStage = Stage.Initialize; let maxSteps = MAX_STEPS_PER_STAGE; let lastScreenState: ScreenState | null = null; let sameScreenCount = 0; async function setCurrentStage(newStage: Stage) { currentStage = newStage; maxSteps = MAX_STEPS_PER_STAGE; lastScreenState = null; sameScreenCount = 0; console.log(`=== Stage: ${newStage} ===`); await agent.utils.job.submitTask( "running", await collectData(), false ); } async function collectData() { return { stage: currentStage, timestamp: new Date().toISOString(), // Add other data you want to track }; } async function main() { try { await setCurrentStage(Stage.Initialize); while (maxSteps-- > 0) { // Get current screen const screen = await agent.actions.screenContent(); const state = detectScreenState(screen); // Store for out-of-steps analysis await agent.utils.outOfSteps.storeScreen( screen, currentStage, state, maxSteps, state === ScreenState.Unknown ? ScreenshotRecord.HIGH_QUALITY : ScreenshotRecord.LOW_QUALITY ); // Check for stuck state await checkSameScreen(state); // Handle the state await handleScreenState(state, screen); // Check completion if (currentStage === Stage.Complete) { await agent.utils.job.submitTask("success", await collectData()); return; } // Small delay between iterations await sleep(500, 1000); } // Out of steps await agent.utils.outOfSteps.submit("outOfSteps"); throw new Error("OUT_OF_STEPS"); } catch (error) { console.error("Automation failed:", error); await agent.utils.job.submitTask("failed", { ...await collectData(), error: String(error) }); } finally { // Always stop the automation stopCurrentAutomation(); } } main(); success Task Submission → Submit results, collect data, and handle job tasks /docs/automation/guide/tasks Error Handling → Handle crashes, dialogs, and recovery /docs/automation/guide/error-handling -------------------------------------------------------------------------------- ## Guide / Task Submission Path: /docs/automation/guide/tasks Description: Submit results, collect data, and handle job tasks Sections: - Job Task Overview - Submitting Task Results - Status Values - Reporting Progress - Completing Tasks - File Attachments - Getting Current Task - Data Collection Pattern - Error Handling - Best Practices - Report Progress Regularly - Include Useful Data - Attach Screenshots on Failure - Use "declined" for Invalid Data - Next Steps Content: Job tasks allow your automation to process multiple items (accounts, orders, etc.) in sequence. You can submit results, collect data, and request new tasks. Job Task Overview Access job task methods through agent.utils.job ; useAnotherTask(): Promise ; getCurrentTask(): Promise Submitting Task Results Use submitTask() to report progress or complete a task: Status Values Status Description Finish? "running" Task is in progress, reporting intermediate data false "success" Task completed successfully true "failed" Task failed (error, blocked, etc.) "declined" Task was declined (invalid data, etc.) Reporting Progress Completing Tasks File Attachments Attach files (screenshots, logs, etc.) to task submissions: Getting Current Task Get information about the current job task: Data Collection Pattern A common pattern for collecting and submitting data throughout the automation: Error Handling Best Practices Report Progress Regularly Submit status on stage changes. This shows progress in the dashboard and helps with debugging. Include Useful Data Include timestamps, stage info, and any data that helps understand what happened. This is invaluable for debugging failed tasks. Attach Screenshots on Failure When a task fails, try to capture a screenshot of the current screen. This makes it much easier to understand what went wrong. Use "declined" for Invalid Data for actual errors. Next Steps next Task submission · Automation Task Submission Submit results, collect data, and handle job tasks agent.utils.job = { submitTask(status, data, finish, files): Promise; useAnotherTask(): Promise; getCurrentTask(): Promise; } submitTask Parameters await agent.utils.job.submitTask( status, // "running" | "success" | "failed" | "declined" data, // Record - data to store finish?, // boolean - true to complete the task (default: true) files? // File[] - files to upload (default: []) ); // Important notes: // - Once finish=true is passed, you cannot submit again for the same task // - The files parameter is ignored when finish=false Progress updates // Report progress without finishing // Note: files parameter is ignored when finish=false await agent.utils.job.submitTask( "running", { stage: "login", progress: 25, timestamp: Date.now() }, false // Don't finish - must specify when not finishing! ); // Later, report more progress await agent.utils.job.submitTask( "running", { stage: "processing", progress: 75, itemsProcessed: 10 }, false // Must be false for progress updates ); Success and failure // Complete with success (files and finish default to [] and true) await agent.utils.job.submitTask("success", { email: "user@example.com", orderId: "12345", completedAt: new Date().toISOString(), itemsProcessed: 15 }); // Don't forget to stop the automation! stopCurrentAutomation(); // Complete with failure await agent.utils.job.submitTask("failed", { email: "user@example.com", error: "Account locked", stage: "login", failedAt: new Date().toISOString() }); stopCurrentAutomation(); Submitting with files // Collect files during automation const files: { name: string; extension: string; base64Data: string }[] = []; // Take a screenshot const screenshot = await agent.actions.screenshot(1080, 1920, 80); files.push({ name: "final_screen", extension: "jpg", base64Data: screenshot.base64 }); // Submit with files (finish defaults to true) await agent.utils.job.submitTask( "success", { email: "user@example.com", orderPlaced: true }, true, // Final submission files // Pass files array ); stopCurrentAutomation(); getCurrentTask const task = await agent.utils.job.getCurrentTask(); if (task.success) { // Task data is available const { job_proof, ...otherData } = task; console.log("Job proof:", job_proof); // Job variables are also available const { email, password } = agent.arguments.jobVariables; // Process the task... } else { console.log("No task available or error:", task.error); } Data collection // Global data object const collectedData: Record = { startedAt: new Date().toISOString(), stages: [], errors: [], metrics: { screensProcessed: 0, actionsPerformed: 0 } }; function recordStage(stage: string, data?: object) { collectedData.stages.push({ stage, timestamp: Date.now(), ...data }); } function recordError(error: string, context?: object) { collectedData.errors.push({ error, timestamp: Date.now(), ...context }); } function recordMetric(metric: string, value: number) { collectedData.metrics[metric] = (collectedData.metrics[metric] || 0) + value; } // Usage during automation recordStage("login_started"); recordMetric("screensProcessed", 1); // Submit progress (files ignored when finish=false) await agent.utils.job.submitTask( "running", { ...collectedData, currentStage: "login" }, false // Don't finish for progress updates ); // Submit final collectedData.completedAt = new Date().toISOString(); await agent.utils.job.submitTask("success", collectedData); // Always stop the automation when done stopCurrentAutomation(); Robust task handling async function runTask() { const files: File[] = []; try { // Get task info const task = await agent.utils.job.getCurrentTask(); if (!task.success) { throw new Error("No task available"); } // Process... const result = await processAutomation(); // Success await agent.utils.job.submitTask("success", result, true, files); } catch (error) { console.error("Task failed:", error); // Take failure screenshot try { const screenshot = await agent.actions.screenshot(1080, 1920, 80); files.push({ name: "error_screen", extension: "jpg", base64Data: screenshot.base64 }); } catch (e) { // Ignore screenshot errors } // Determine failure type const errorMessage = String(error); let status: "failed" | "declined" = "failed"; if (errorMessage.includes("invalid") || errorMessage.includes("not found")) { status = "declined"; // Invalid input data } // Submit failure await agent.utils.job.submitTask(status, { error: errorMessage, stage: currentStage, timestamp: Date.now() }, true, files); } finally { // Always stop the automation stopCurrentAutomation(); } } success Error Handling → Handle crashes, dialogs, and recovery /docs/automation/guide/error-handling Running Automations → Execute automations on devices and collect results /docs/automation/guide/running -------------------------------------------------------------------------------- ## Guide / Error Handling Path: /docs/automation/guide/error-handling Description: Handle crashes, dialogs, network issues, and recovery Sections: - System Interruptions - Crash Dialogs - Phone Dialogs - Notification Shade - Network Connectivity - Complete Error Handler - Retry Strategies - Simple Retry - Conditional Retry - Recovery Actions - Global Error Handler - Best Practices - Always Check System States First - Set Up Network Callback Early - Capture Screenshots on Failure - Use Meaningful Error Reasons - Clean Up Resources - Next Steps Content: Robust error handling is critical for production automations. This guide covers common error scenarios and how to handle them gracefully. System Interruptions System dialogs can appear at any time. Check for and handle them before processing app screens. Crash Dialogs App crashes show a system dialog with options to close or report: Phone Dialogs Incoming calls or phone-related dialogs can interrupt automation: Notification Shade Dismiss expanded notifications that might be blocking the UI: Network Connectivity Complete Error Handler Put it all together in a system error handler: Retry Strategies Simple Retry Promise , maxAttempts: number = 3 ): Promise Conditional Retry boolean, maxAttempts: number = 3 ): Promise Recovery Actions Global Error Handler Best Practices Always Check System States First Before processing any app screen, check for crashes, dialogs, and notification shade. These can appear at any time and block automation. Set Up Network Callback Early Register the network callback at the start of your automation. Track total downtime for reporting. Capture Screenshots on Failure Always try to capture a screenshot when an error occurs. This is invaluable for debugging what went wrong. Use Meaningful Error Reasons Categorize errors into meaningful reasons (OUT_OF_STEPS, NETWORK_ERROR, etc.). This makes it easier to analyze failure patterns. Clean Up Resources Use try/finally to clean up callbacks and resources. This prevents memory leaks and unexpected behavior. Next Steps next Error handling · Automation Error Handling Handle crashes, dialogs, network issues, and recovery Crash dialog handling // Detect crash dialog function isCrashDialog(screen: AndroidNode): boolean { return !!findNodesById(screen, "android:id/aerr_close") .find(n => n.clickable); } // Handle crash dialog async function handleCrashDialog(screen: AndroidNode): Promise { const closeButton = findNodesById(screen, "android:id/aerr_close") .find(n => n.clickable); if (closeButton) { console.warn("Crash dialog detected, closing..."); await closeButton.performAction(agent.constants.ACTION_CLICK); await sleep(2000); return true; } return false; } Phone dialog handling const PHONE_PACKAGE = "com.android.phone"; const PHONE_DIALOG_IDS = [ "com.android.phone:id/floating_end_call_action_button", "com.android.phone:id/declineButton", "com.android.phone:id/dismiss_button", ]; function isPhoneDialog(screen: AndroidNode): boolean { const allNodes = getAllNodes(screen); // All nodes are from phone package return allNodes.every(n => n.packageName === PHONE_PACKAGE); } async function handlePhoneDialog(screen: AndroidNode): Promise { const allNodes = getAllNodes(screen); // Look for dismiss/decline button const dismissButton = allNodes.find(n => PHONE_DIALOG_IDS.includes(n.viewId || "") ); if (dismissButton) { console.warn("Phone dialog detected, dismissing..."); if (dismissButton.actions.includes(agent.constants.ACTION_CLICK)) { await dismissButton.performAction(agent.constants.ACTION_CLICK); } else { // Fallback to random tap within bounds dismissButton.randomClick(); } await sleep(2000); return true; } return false; } Notification shade handling async function dismissNotificationShade(): Promise { // Get all screens (including notification shade) const screens = await agent.actions.allScreensContent(); const allNodes = screens.flatMap(s => getAllNodes(s)); // Look for notification rows const notificationRow = allNodes.find(n => n.viewId === "com.android.systemui:id/expandableNotificationRow" && n.actions.includes(agent.constants.ACTION_DISMISS) ); if (notificationRow) { console.log("Dismissing notification..."); await notificationRow.performAction(agent.constants.ACTION_DISMISS); await sleep(500); return true; } // Check if shade is open (swipe down to close) const shadePanel = allNodes.find(n => n.viewId === "com.android.systemui:id/notification_panel" ); if (shadePanel) { console.log("Closing notification shade..."); await agent.actions.swipe(540, 1500, 540, 500, 300); await sleep(500); return true; } return false; } Network monitoring let isNetworkAvailable = true; let networkDownTime = 0; let networkDownTimestamp: number | undefined; // Set up network callback agent.utils.setNetworkCallback((available) => { isNetworkAvailable = available; if (!available && networkDownTimestamp === undefined) { networkDownTimestamp = Date.now(); console.warn("Network disconnected!"); } else if (available && networkDownTimestamp) { networkDownTime += Date.now() - networkDownTimestamp; networkDownTimestamp = undefined; console.log("Network restored"); } }); // Handle no internet screen async function handleNoInternet(screen: AndroidNode): Promise { const allNodes = getAllNodes(screen); const noInternetIndicator = allNodes.find(n => n.text?.toLowerCase()?.includes("no internet") || n.text?.toLowerCase()?.includes("you're offline") || n.text?.toLowerCase()?.includes("check your connection") ); if (noInternetIndicator || !isNetworkAvailable) { console.warn("No internet, waiting..."); // Wait for network to come back for (let i = 0; i < 30; i++) { if (isNetworkAvailable) { // Try to refresh const retryButton = allNodes.find(n => n.clickable && (n.text?.toLowerCase() === "retry" || n.text?.toLowerCase() === "try again") ); if (retryButton) { await retryButton.performAction(agent.constants.ACTION_CLICK); } await sleep(2000); return true; } await sleep(2000); } // Network didn't come back throw new Error("NETWORK_TIMEOUT"); } return false; } System error handler // Call this at the start of each main loop iteration async function handleSystemErrors(screen: AndroidNode): Promise { // Check and dismiss notification shade first if (await dismissNotificationShade()) { return true; // Screen changed, re-check } // Check for crash dialog if (await handleCrashDialog(screen)) { return true; } // Check for phone dialog if (await handlePhoneDialog(screen)) { return true; } // Check for no internet if (await handleNoInternet(screen)) { return true; } return false; // No system errors found } // Usage in main loop async function mainLoop() { while (maxSteps-- > 0) { const screen = await agent.actions.screenContent(); // Handle system errors first if (await handleSystemErrors(screen)) { continue; // Re-check after handling } // Get screen state and process... const state = detectScreenState(screen); await handleScreenState(state, screen); } } Basic retry async function withRetry( fn: () => Promise, maxAttempts: number = 3 ): Promise { let lastError: Error | undefined; for (let attempt = 1; attempt <= maxAttempts; attempt++) { try { return await fn(); } catch (error) { lastError = error as Error; console.warn(`Attempt ${attempt} failed: ${lastError.message}`); if (attempt < maxAttempts) { await sleep(1000 * attempt); // Increasing delay } } } throw lastError; } Conditional retry async function retryOnCondition( fn: () => Promise, shouldRetry: (error: Error) => boolean, maxAttempts: number = 3 ): Promise { let lastError: Error | undefined; for (let attempt = 1; attempt <= maxAttempts; attempt++) { try { return await fn(); } catch (error) { lastError = error as Error; if (!shouldRetry(lastError) || attempt >= maxAttempts) { throw lastError; } console.warn(`Retrying after: ${lastError.message}`); await sleep(2000); } } throw lastError; } // Usage: Only retry on network errors await retryOnCondition( () => performNetworkOperation(), (error) => error.message.includes("network") || error.message.includes("timeout"), 5 ); Recovery strategies async function attemptRecovery( state: ScreenState, screen: AndroidNode ): Promise { console.log(`Attempting recovery from ${state}`); switch (state) { case ScreenState.Unknown: // Try going back await agent.actions.goBack(); await sleep(1000); return true; case ScreenState.Loading: // Wait longer for loading await sleep(5000); return true; case ScreenState.NoInternet: // Toggle airplane mode to refresh connection await agent.actions.airplane(); await sleep(5000); return true; case ScreenState.RateLimited: // Wait before retrying console.log("Rate limited, waiting 60 seconds..."); await sleep(60000); return true; case ScreenState.ErrorDialog: // Try to dismiss error const okButton = getAllNodes(screen).find(n => n.clickable && (n.text?.toLowerCase() === "ok" || n.text?.toLowerCase() === "dismiss") ); if (okButton) { await okButton.performAction(agent.constants.ACTION_CLICK); await sleep(1000); return true; } break; } return false; } Wrapping the automation async function runAutomation() { const files: File[] = []; try { // Set up callbacks agent.utils.setNetworkCallback(handleNetworkChange); agent.notifications.setNotificationCallback(handleNotification); // Run main automation await main(); // Success await agent.utils.job.submitTask("success", await collectFinalData(), true, files); } catch (error) { console.error("Automation error:", error); // Capture error screenshot try { const screenshot = await agent.actions.screenshot(1080, 1920, 80); files.push({ name: "error_screenshot", extension: "jpg", base64Data: screenshot.base64 }); } catch (e) { console.error("Failed to capture screenshot:", e); } // Determine failure type const errorMessage = String(error); let failureReason = "UNKNOWN_ERROR"; if (errorMessage.includes("OUT_OF_STEPS")) { failureReason = "OUT_OF_STEPS"; } else if (errorMessage.includes("NETWORK")) { failureReason = "NETWORK_ERROR"; } else if (errorMessage.includes("not found")) { failureReason = "ELEMENT_NOT_FOUND"; } // Submit failure await agent.utils.job.submitTask("failed", { error: errorMessage, reason: failureReason, stage: currentStage, networkDownTime }, true, files); } finally { // Clean up callbacks agent.utils.setNetworkCallback(null); agent.notifications.setNotificationCallback(null); // Always stop the automation stopCurrentAutomation(); } } // Start automation runAutomation(); success Running Automations → Execute automations on devices and collect results /docs/automation/guide/running Full Tutorial → Complete example putting everything together /docs/automation/guide/tutorial -------------------------------------------------------------------------------- ## Guide / Running Automations Path: /docs/automation/guide/running Description: Execute automations on devices and collect results Sections: - The Automation Runner - Select Devices - Configure Automation Parameters - Provide Job Variables - Monitor & Collect Results - Execution Flow - Parameter Validation - Data Collection - Downloading Results - Start and Stop Commands - Multi-Device Execution - Same Parameters - Same Job Variables - Independent Execution - Status Updates - Monitoring logs - Debugging Tips - Check Device Connection - Review Parameter Values - Check Console Output - Review Submitted Data - Requirements Check - Version Requirements - Next Steps Content: Once your automation is ready, you can run it on connected devices through the dashboard. This guide covers the execution flow and data collection. The Automation Runner Access the runner from the Select Devices Choose one or more connected devices to run the automation on. Each device will execute the automation independently. Configure Automation Parameters Fill in automation parameters defined in your project. These are validated before execution starts. Provide Job Variables Enter job-specific data like account credentials. These values are accessible via agent.arguments.jobVariables Monitor & Collect Results View collected data, download results, and track automation status in the data section. Execution Flow Select automation from your available automations Select devices to run on (must be connected) Fill parameters as defined in your automation schema Fill job variables for the specific task Click Start to begin execution Monitor progress via status updates View/download results when complete Parameter Validation Parameters are validated against your schema before execution: Data Collection Data submitted via submitTask() is collected and displayed in the runner: Downloading Results Collected data can be downloaded as CSV for analysis: Click the Download button in the data section to export all collected data as a CSV file. The file includes all fields submitted via Start and Stop Commands Multi-Device Execution Running on multiple devices simultaneously: Same Parameters All selected devices receive the same automation parameters. Use this for consistent configuration across devices. Same Job Variables Job variables are also shared. If you need different data per device, you will need to run separate automation jobs with different job variable sets. Independent Execution Each device executes independently. One device failing doesn't affect others. Results are collected separately per device. Status Updates The runner shows submitted data for each execution (click refresh to update). Monitoring logs Debugging Tips Check Device Connection Ensure devices show as "Connected" before starting. Disconnected devices won't receive the automation command. Review Parameter Values Double-check parameter values before starting. Invalid values will cause validation errors or unexpected behavior. Check Console Output console.log() liberally for debugging. Review Submitted Data Check the collected data table for insights. Failed tasks often include error details in the submitted data. Requirements Check Before execution, the system verifies: Device meets minimum Android version requirement (only when running through a job) Device has minimum app version installed (only when running through a job) User has execution permission for the automation All required parameters are provided and valid Version Requirements If your automation uses features like dpad() inputKey() , set the minimum Android version to 13+ (API 33) in your project configuration. Next Steps lucide-react next Running automations · Automation Running Automations Execute automations on devices and collect results -> Schema validation json // Your schema definition { "fields": [ { "name": "maxRetries", "type": "number", "required": true, "min": 1, "max": 10, "integer": true }, { "name": "targetUrl", "type": "string", "required": true, "pattern": "^https?://.*" } ] } // Validation errors shown to user: // - "maxRetries: Value must be at least 1" // - "targetUrl: Value does not match pattern" Data flow // In your automation await agent.utils.job.submitTask("success", { email: "user@example.com", orderId: "12345", itemsProcessed: 10, completedAt: new Date().toISOString() }); // Don't forget to stop the automation! stopCurrentAutomation(); // This data appears in the runner as: // | Device | email | orderId | itemsProcessed | completedAt | // |------------|------------------|---------|----------------|----------------------| // | Device-001 | user@example.com | 12345 | 10 | 2024-01-15T10:30:00Z | Runner commands // When you click "Start": POST /api/v1/device/automation { "device_ids": ["device-001", "device-002"], "automationId": "automation-uuid", "command": "start", "automationParameters": { ... }, "jobVariables": { ... } } // When you click "Stop": POST /api/v1/device/automation { "device_ids": ["device-001", "device-002"], "automationId": "automation-uuid", "command": "stop" } info Live Control success warning Full Tutorial → Complete MySocial example putting everything together /docs/automation/guide/tutorial API Reference → Complete documentation of all available methods /docs/automation/reference -------------------------------------------------------------------------------- ## Guide / Full Tutorial Path: /docs/automation/guide/tutorial Description: Build a complete automation from scratch Sections: - Step 1: Define Stages - Step 2: Define Screen States - Step 3: Utility Functions - Step 4: Screen Detection - Step 5: Stage Handlers - Step 6: Main Entry Point - Configuration Schema - Key Patterns Used - Stage-Based Organization - Comprehensive Screen Detection - Error Recovery - Data Collection - Out-of-Steps Handling - ES6 Module Organization - Running the Automation - Next Steps Content: This tutorial walks through building a complete automation for a fictional social media app called "MySocial". The automation will: Launch the app and verify login state Navigate to the Messages section Process unread conversations Send automated responses Like posts in the feed Log all interactions to task data This example uses ES6 imports to organize code across multiple files: Step 1: Define Stages First, define the automation stages: Step 2: Define Screen States Define all possible screen states: Step 3: Utility Functions Create shared utility functions: Step 4: Screen Detection Implement screen state detection: Step 5: Stage Handlers Implement handlers for each stage: Step 6: Main Entry Point Finally, create the main entry point that ties everything together: Configuration Schema Configure the automation parameters and job variables in the Options panel: Key Patterns Used Stage-Based Organization The automation is divided into clear stages (Initialize, LaunchApp, HandleLogin, etc.) with step counters reset at each transition. Comprehensive Screen Detection Detection handles system states (crash, phone, permissions) before app states, using multiple conditions for reliable identification. Error Recovery Stuck state detection with automatic recovery (go back), notification dismissal, and network monitoring throughout execution. Data Collection submitTask("running", ...) , and final results include all collected metrics. Out-of-Steps Handling storeScreen() , and out-of-steps reports are submitted for debugging. ES6 Module Organization Code is organized into separate files with clean imports, making it easy to maintain and test individual components. Running the Automation Save all files in the IDE Configure automation parameters and job variables schemas Go to Devices → Automation Runner Select your automation and target device(s) Fill in parameters (response message, max likes) Fill in job variables (email, password) Click Start and monitor progress You've built a complete automation with all the essential patterns. Use this as a template for your own automations, adapting the stages, screen states, and handlers for your target app. Next Steps next Full tutorial · Automation Full Tutorial: MySocial Auto-Responder Build a complete automation from scratch Complete Example info Project Structure mysocial-responder/ ├── main.ts # Entry point & main loop ├── stages.ts # Stage enum ├── screenStates.ts # ScreenState enum ├── detection.ts # Screen detection logic ├── handlers.ts # Stage handlers └── utils.ts # Shared utilities stages.ts // Define all automation stages export enum Stage { Initialize = "Initialize", LaunchApp = "LaunchApp", HandleLogin = "HandleLogin", NavigateToMessages = "NavigateToMessages", SelectUnreadChat = "SelectUnreadChat", ProcessMessages = "ProcessMessages", NavigateToFeed = "NavigateToFeed", LikePosts = "LikePosts", Complete = "Complete", } screenStates.ts // System states (can appear anytime) export enum ScreenState { Unknown = "Unknown", Crash = "Crash", PhoneDialog = "PhoneDialog", NoInternet = "NoInternet", Loading = "Loading", // Login states SplashScreen = "SplashScreen", LoginScreen = "LoginScreen", LoginEnterPassword = "LoginEnterPassword", LoginTwoFactor = "LoginTwoFactor", LoginError = "LoginError", // Main app states HomeTab = "HomeTab", SearchTab = "SearchTab", MessagesTab = "MessagesTab", NotificationsTab = "NotificationsTab", ProfileTab = "ProfileTab", // Message states ChatList = "ChatList", ChatListEmpty = "ChatListEmpty", ChatConversation = "ChatConversation", NewMessageDialog = "NewMessageDialog", // Feed states PostDetail = "PostDetail", CommentSheet = "CommentSheet", // Dialog states PermissionDialog = "PermissionDialog", UpdateRequired = "UpdateRequired", RateLimited = "RateLimited", ErrorDialog = "ErrorDialog", } utils.ts // App package name export const APP_PACKAGE = "com.mysocial.app"; // Sleep with optional random range export function sleep(min: number, max?: number): Promise { const ms = max ? Math.floor(Math.random() * (max - min) + min) : min; return new Promise(resolve => setTimeout(resolve, ms)); } // Random number in range export function randomRange(min: number, max: number): number { return Math.floor(Math.random() * (max - min) + min); } // Collected data storage export const collectedData: { messagesResponded: number; postsLiked: number; errors: string[]; stages: { stage: string; timestamp: number }[]; } = { messagesResponded: 0, postsLiked: 0, errors: [], stages: [] }; export function recordStage(stage: string) { collectedData.stages.push({ stage, timestamp: Date.now() }); } export function recordError(error: string) { collectedData.errors.push(error); console.error(error); } detection.ts import { ScreenState } from "./screenStates.js"; import { APP_PACKAGE } from "./utils.js"; export function detectScreenState(screen: AndroidNode): ScreenState { const allNodes = getAllNodes(screen); // === SYSTEM STATES (check first) === // Crash dialog if (findNodesById(screen, "android:id/aerr_close").find(n => n.clickable)) { return ScreenState.Crash; } // Phone dialog if (allNodes.every(n => n.packageName === "com.android.phone")) { return ScreenState.PhoneDialog; } // No internet if (allNodes.find(n => n.text?.toLowerCase()?.includes("no internet") || n.text?.toLowerCase()?.includes("offline") )) { return ScreenState.NoInternet; } // Loading screen (minimal nodes with progress indicator) if (allNodes.length <= 5 && allNodes.find(n => n.className?.includes("ProgressBar") )) { return ScreenState.Loading; } // Permission dialog if (allNodes.find(n => n.packageName === "com.android.permissioncontroller" )) { return ScreenState.PermissionDialog; } // === APP STATES === const appNodes = allNodes.filter(n => n.packageName === APP_PACKAGE); // Splash screen if (appNodes.find(n => n.viewId === `${APP_PACKAGE}:id/splash_logo` )) { return ScreenState.SplashScreen; } // Login screen (email field) if (appNodes.find(n => n.className === "android.widget.EditText" && (n.hintText?.toLowerCase()?.includes("email") || n.hintText?.toLowerCase()?.includes("username")) )) { return ScreenState.LoginScreen; } // Password screen if (appNodes.find(n => n.isPassword)) { return ScreenState.LoginEnterPassword; } // Two-factor screen if (appNodes.find(n => n.text?.toLowerCase()?.includes("verification code") || n.text?.toLowerCase()?.includes("2fa") )) { return ScreenState.LoginTwoFactor; } // Check bottom navigation tabs const homeTab = appNodes.find(n => n.description === "Home" && n.className === "android.widget.Button" ); const messagesTab = appNodes.find(n => n.description === "Messages" && n.className === "android.widget.Button" ); // Messages tab selected if (messagesTab?.isSelected) { // Chat conversation (has message input) if (appNodes.find(n => n.hintText?.toLowerCase()?.includes("message") && n.className === "android.widget.EditText" )) { return ScreenState.ChatConversation; } // Chat list (has conversation items) if (appNodes.find(n => n.viewId === `${APP_PACKAGE}:id/chat_list` )) { const hasUnread = appNodes.find(n => n.viewId === `${APP_PACKAGE}:id/unread_badge` ); return hasUnread ? ScreenState.ChatList : ScreenState.ChatListEmpty; } return ScreenState.MessagesTab; } // Home tab selected if (homeTab?.isSelected) { return ScreenState.HomeTab; } // Profile tab if (appNodes.find(n => n.description === "Profile" && n.isSelected )) { return ScreenState.ProfileTab; } // Error dialog if (appNodes.find(n => n.text?.toLowerCase()?.includes("error") || n.text?.toLowerCase()?.includes("something went wrong") ) && appNodes.find(n => n.text?.toLowerCase() === "ok" && n.clickable )) { return ScreenState.ErrorDialog; } // Rate limited if (appNodes.find(n => n.text?.toLowerCase()?.includes("rate limit") || n.text?.toLowerCase()?.includes("try again later") )) { return ScreenState.RateLimited; } return ScreenState.Unknown; } handlers.ts import { Stage } from "./stages.js"; import { ScreenState } from "./screenStates.js"; import { APP_PACKAGE, sleep, collectedData, recordError } from "./utils.js"; // State variables let currentStage: Stage = Stage.Initialize; let maxSteps = 48; let isNetworkAvailable = true; export function getCurrentStage() { return currentStage; } export function getMaxSteps() { return maxSteps; } export async function setCurrentStage(newStage: Stage) { currentStage = newStage; maxSteps = 48; // Reset step counter console.log(`=== Stage: ${newStage} ===`); await agent.utils.job.submitTask( "running", { stage: newStage, ...collectedData }, false // Don't finish the task ); } // Network callback agent.utils.setNetworkCallback((available) => { isNetworkAvailable = available; if (!available) { recordError("Network disconnected"); } }); // === SYSTEM HANDLERS === export async function handleCrash(screen: AndroidNode): Promise { const closeBtn = findNodesById(screen, "android:id/aerr_close") .find(n => n.clickable); if (closeBtn) { console.log("Closing crash dialog..."); await closeBtn.performAction(agent.constants.ACTION_CLICK); await sleep(2000); return true; } return false; } export async function handlePhoneDialog(screen: AndroidNode): Promise { const allNodes = getAllNodes(screen); const dismissBtn = allNodes.find(n => n.viewId?.includes("dismiss") || n.viewId?.includes("decline") ); if (dismissBtn) { console.log("Dismissing phone dialog..."); dismissBtn.randomClick(); await sleep(2000); return true; } return false; } export async function handlePermissionDialog(screen: AndroidNode): Promise { const allNodes = getAllNodes(screen); const allowBtn = allNodes.find(n => n.text?.toLowerCase()?.includes("allow") && n.clickable ); if (allowBtn) { console.log("Granting permission..."); await allowBtn.performAction(agent.constants.ACTION_CLICK); await sleep(1000); return true; } return false; } export async function handleErrorDialog(screen: AndroidNode): Promise { const allNodes = getAllNodes(screen); const okBtn = allNodes.find(n => n.text?.toLowerCase() === "ok" && n.clickable ); if (okBtn) { console.log("Dismissing error dialog..."); await okBtn.performAction(agent.constants.ACTION_CLICK); await sleep(1000); return true; } return false; } // === STAGE HANDLERS === export async function handleInitialize() { // Verify app is installed const apps = await agent.actions.listApps(); if (!apps[APP_PACKAGE]) { throw new Error("MySocial app not installed"); } console.log("App installed, proceeding..."); await setCurrentStage(Stage.LaunchApp); } export async function handleLaunchApp( state: ScreenState, screen: AndroidNode ) { if (state === ScreenState.HomeTab || state === ScreenState.MessagesTab) { // Already in app await setCurrentStage(Stage.NavigateToMessages); return; } if (state === ScreenState.LoginScreen || state === ScreenState.LoginEnterPassword) { await setCurrentStage(Stage.HandleLogin); return; } if (state === ScreenState.SplashScreen || state === ScreenState.Loading) { await sleep(2000, 3000); return; } // Launch the app console.log("Launching MySocial..."); await agent.actions.launchApp(APP_PACKAGE); await sleep(3000); } export async function handleLogin( state: ScreenState, screen: AndroidNode ) { const allNodes = getAllNodes(screen); const { email, password } = agent.arguments.jobVariables; if (state === ScreenState.LoginScreen) { // Enter email const emailField = allNodes.find(n => n.className === "android.widget.EditText" && n.hintText?.toLowerCase()?.includes("email") ); if (emailField) { await emailField.performAction(agent.constants.ACTION_FOCUS); await sleep(500); await agent.actions.writeText(email); await sleep(500); // Click next/continue const nextBtn = allNodes.find(n => n.clickable && (n.text?.toLowerCase() === "next" || n.text?.toLowerCase() === "continue") ); if (nextBtn) { await nextBtn.performAction(agent.constants.ACTION_CLICK); } await sleep(2000); } return; } if (state === ScreenState.LoginEnterPassword) { // Enter password const passwordField = allNodes.find(n => n.isPassword); if (passwordField) { await passwordField.performAction(agent.constants.ACTION_FOCUS); await sleep(500); await agent.actions.writeText(password); await sleep(500); await agent.actions.hideKeyboard(); // Click login const loginBtn = allNodes.find(n => n.clickable && (n.text?.toLowerCase() === "login" || n.text?.toLowerCase() === "sign in") ); if (loginBtn) { await loginBtn.performAction(agent.constants.ACTION_CLICK); } await sleep(3000); } return; } if (state === ScreenState.HomeTab || state === ScreenState.MessagesTab) { console.log("Login successful!"); await setCurrentStage(Stage.NavigateToMessages); } } export async function handleNavigateToMessages( state: ScreenState, screen: AndroidNode ) { if (state === ScreenState.MessagesTab || state === ScreenState.ChatList) { await setCurrentStage(Stage.SelectUnreadChat); return; } // Click messages tab const allNodes = getAllNodes(screen); const messagesTab = allNodes.find(n => n.description === "Messages" && n.className === "android.widget.Button" && n.clickable ); if (messagesTab) { console.log("Navigating to messages..."); await messagesTab.performAction(agent.constants.ACTION_CLICK); await sleep(2000); } } export async function handleSelectUnreadChat( state: ScreenState, screen: AndroidNode ) { if (state === ScreenState.ChatConversation) { await setCurrentStage(Stage.ProcessMessages); return; } if (state === ScreenState.ChatListEmpty) { console.log("No unread messages, moving to feed..."); await setCurrentStage(Stage.NavigateToFeed); return; } const allNodes = getAllNodes(screen); // Find unread chat const unreadChat = allNodes.find(n => n.viewId === `${APP_PACKAGE}:id/chat_item` && getAllNodes(n).find(child => child.viewId === `${APP_PACKAGE}:id/unread_badge` ) ); if (unreadChat) { console.log("Opening unread chat..."); unreadChat.randomClick(); await sleep(2000); } else { // No more unread, move to feed console.log("No more unread chats, moving to feed..."); await setCurrentStage(Stage.NavigateToFeed); } } export async function handleProcessMessages( state: ScreenState, screen: AndroidNode ) { const allNodes = getAllNodes(screen); const { responseMessage } = agent.arguments.automationParameters; // Find message input const messageInput = allNodes.find(n => n.hintText?.toLowerCase()?.includes("message") && n.className === "android.widget.EditText" ); if (messageInput) { // Type response await messageInput.performAction(agent.constants.ACTION_FOCUS); await sleep(500); await agent.actions.writeText(responseMessage || "Thanks for your message!"); await sleep(500); // Send message const sendBtn = allNodes.find(n => (n.description?.toLowerCase() === "send" || n.viewId?.includes("send")) && n.clickable ); if (sendBtn) { await sendBtn.performAction(agent.constants.ACTION_CLICK); collectedData.messagesResponded++; console.log(`Message sent! Total: ${collectedData.messagesResponded}`); await sleep(1500); } } // Go back to chat list await agent.actions.goBack(); await sleep(1000); await setCurrentStage(Stage.SelectUnreadChat); } export async function handleNavigateToFeed( state: ScreenState, screen: AndroidNode ) { if (state === ScreenState.HomeTab) { await setCurrentStage(Stage.LikePosts); return; } const allNodes = getAllNodes(screen); const homeTab = allNodes.find(n => n.description === "Home" && n.className === "android.widget.Button" && n.clickable ); if (homeTab) { console.log("Navigating to feed..."); await homeTab.performAction(agent.constants.ACTION_CLICK); await sleep(2000); } } export async function handleLikePosts( state: ScreenState, screen: AndroidNode ) { const { maxLikes } = agent.arguments.automationParameters; if (collectedData.postsLiked >= (maxLikes || 5)) { console.log("Reached max likes, completing..."); await setCurrentStage(Stage.Complete); return; } const allNodes = getAllNodes(screen); // Find like button (not already liked) const likeBtn = allNodes.find(n => n.viewId === `${APP_PACKAGE}:id/like_button` && n.description !== "Unlike" && n.clickable ); if (likeBtn) { console.log("Liking post..."); await likeBtn.performAction(agent.constants.ACTION_CLICK); collectedData.postsLiked++; await sleep(1000, 2000); // Scroll to next post await agent.actions.swipe(540, 1500, 540, 800, 500); await sleep(1000); } else { // Scroll to find more posts await agent.actions.swipe(540, 1500, 540, 500, 500); await sleep(1500); } } main.ts import { Stage } from "./stages.js"; import { ScreenState } from "./screenStates.js"; import { detectScreenState } from "./detection.js"; import { getCurrentStage, getMaxSteps, setCurrentStage, handleCrash, handlePhoneDialog, handlePermissionDialog, handleErrorDialog, handleInitialize, handleLaunchApp, handleLogin, handleNavigateToMessages, handleSelectUnreadChat, handleProcessMessages, handleNavigateToFeed, handleLikePosts } from "./handlers.js"; import { sleep, collectedData, recordError } from "./utils.js"; // File storage for screenshots const files: { name: string; extension: string; base64Data: string }[] = []; // Dismiss notification shade if visible async function dismissNotifications(): Promise { const screens = await agent.actions.allScreensContent(); const allNodes = screens.flatMap(s => getAllNodes(s)); const notif = allNodes.find(n => n.viewId === "com.android.systemui:id/expandableNotificationRow" && n.actions.includes(agent.constants.ACTION_DISMISS) ); if (notif) { await notif.performAction(agent.constants.ACTION_DISMISS); await sleep(500); return true; } return false; } // Handle system-level interruptions async function handleSystemStates( state: ScreenState, screen: AndroidNode ): Promise { // First check notifications if (await dismissNotifications()) { return true; } switch (state) { case ScreenState.Crash: return await handleCrash(screen); case ScreenState.PhoneDialog: return await handlePhoneDialog(screen); case ScreenState.PermissionDialog: return await handlePermissionDialog(screen); case ScreenState.ErrorDialog: return await handleErrorDialog(screen); case ScreenState.NoInternet: console.log("Waiting for network..."); await sleep(5000); return true; case ScreenState.Loading: await sleep(2000); return true; case ScreenState.RateLimited: console.log("Rate limited, waiting 60 seconds..."); await sleep(60000); return true; } return false; } // Main screen state handler async function handleScreenState( state: ScreenState, screen: AndroidNode ) { // Handle system states first if (await handleSystemStates(state, screen)) { return; } const currentStage = getCurrentStage(); switch (currentStage) { case Stage.Initialize: await handleInitialize(); break; case Stage.LaunchApp: await handleLaunchApp(state, screen); break; case Stage.HandleLogin: await handleLogin(state, screen); break; case Stage.NavigateToMessages: await handleNavigateToMessages(state, screen); break; case Stage.SelectUnreadChat: await handleSelectUnreadChat(state, screen); break; case Stage.ProcessMessages: await handleProcessMessages(state, screen); break; case Stage.NavigateToFeed: await handleNavigateToFeed(state, screen); break; case Stage.LikePosts: await handleLikePosts(state, screen); break; } } // Main automation function async function main() { console.log("=== MySocial Auto-Responder Starting ==="); try { await setCurrentStage(Stage.Initialize); let maxSteps = 48; let sameStateCount = 0; let lastState: ScreenState | null = null; while (maxSteps-- > 0) { // Get current screen const screen = await agent.actions.screenContent(); const state = detectScreenState(screen); console.log(`[Step ${48 - maxSteps}] State: ${state}, Stage: ${getCurrentStage()}`); // Store screen for debugging await agent.utils.outOfSteps.storeScreen( screen, getCurrentStage(), state, maxSteps, state === ScreenState.Unknown ? ScreenshotRecord.HIGH_QUALITY : ScreenshotRecord.LOW_QUALITY ); // Check for stuck state if (state === lastState) { sameStateCount++; if (sameStateCount >= 5) { console.warn(`Stuck on ${state}, attempting recovery...`); await agent.actions.goBack(); await sleep(1000); sameStateCount = 0; } } else { lastState = state; sameStateCount = 0; } // Handle the screen state await handleScreenState(state, screen); // Check if complete if (getCurrentStage() === Stage.Complete) { break; } // Delay between iterations await sleep(500, 1000); } // Check completion status if (getCurrentStage() === Stage.Complete) { console.log("=== Automation Complete! ==="); console.log(`Messages responded: ${collectedData.messagesResponded}`); console.log(`Posts liked: ${collectedData.postsLiked}`); await agent.utils.job.submitTask("success", { ...collectedData, completedAt: new Date().toISOString() }, true, files); } else { // Out of steps console.warn("Out of steps!"); const result = await agent.utils.outOfSteps.submit("outOfSteps"); await agent.utils.job.submitTask("failed", { ...collectedData, error: "OUT_OF_STEPS", outOfStepsId: result.success ? result.id : null }, true, files); } } catch (error) { console.error("Automation error:", error); // Capture error screenshot try { const screenshot = await agent.actions.screenshot(1080, 1920, 80); files.push({ name: "error_screenshot", extension: "jpg", base64Data: screenshot.base64 }); } catch (e) { // Ignore screenshot errors } await agent.utils.job.submitTask("failed", { ...collectedData, error: String(error), stage: getCurrentStage() }, true, files); } finally { // Always stop the automation stopCurrentAutomation(); } } // Start the automation main(); Automation Parameters Schema json { "fields": [ { "name": "responseMessage", "type": "string", "required": false, "description": "Message to send as response", "defaultValue": "Thanks for reaching out!" }, { "name": "maxLikes", "type": "number", "required": false, "description": "Maximum posts to like", "defaultValue": 5, "min": 1, "max": 20 } ] } Job Variables Schema { "fields": [ { "name": "email", "type": "string", "required": true, "description": "MySocial account email" }, { "name": "password", "type": "string", "required": true, "description": "MySocial account password" } ] } success Congratulations! API Reference → Explore all available methods and types /docs/automation/reference Actions Reference → All touch, navigation, and input actions /docs/automation/reference/agent/actions AndroidNode Reference → Screen content structure and filtering /docs/automation/reference/android-node -------------------------------------------------------------------------------- ## Reference / Overview Path: /docs/automation/reference Description: Complete reference for all automation interfaces, methods, and types Content: next Reference · Automation API Reference Complete reference for all automation interfaces, methods, and types The Automation API is accessed through the global agent object. It provides comprehensive control over Android devices including touch gestures, text input, app management, file operations, and screen content access. Entry Point // The agent object is globally available in automation scripts declare const agent: Agent; interface Agent { constants: AgentConstants; // Accessibility action constants actions: AgentActions; // Device automation methods utils: AgentUtils; // Utility functions & file operations info: AgentInfo; // Device & automation metadata control: AgentControl; // Automation control display: AgentDisplay; // HTML overlay display email: AgentEmail; // Email operations notifications: AgentNotifications; // Notification handling } Agent Namespaces Constants 44 accessibility action constants for nodeAction() operations /docs/automation/reference/agent/constants Actions 26 methods for device interaction: tap, swipe, screenshot, screenContent, launchApp, and more /docs/automation/reference/agent/actions Utils & Files Utility functions and 14 file operation methods /docs/automation/reference/agent/utils Info Get automation and device metadata /docs/automation/reference/agent/info Control Stop automation execution /docs/automation/reference/agent/control Display Show HTML overlays on screen /docs/automation/reference/agent/display Email Read emails via IMAP /docs/automation/reference/agent/email Notifications Handle system notifications /docs/automation/reference/agent/notifications Screen Content AndroidNode Accessibility tree nodes with 35+ properties and 11 methods for traversing and querying UI elements /docs/automation/reference/android-node AndroidNodeFilter Builder pattern with 28 chainable methods for finding nodes: isButton(), hasText(), isClickable(), and more /docs/automation/reference/android-node/filter Other Types All supporting types: FileInfo, Email, OCR types, callbacks, and more /docs/automation/reference/types Helper Functions 5 standalone utility functions for working with nodes /docs/automation/reference/helpers -------------------------------------------------------------------------------- ## Reference / agent Path: /docs/automation/reference/agent Description: The main entry point for all automation functionality Content: next agent · Reference Agent Interface The main entry point for all automation functionality The agent object is globally available in all automation scripts. It provides access to device controls, screen content, file operations, and more through its namespaced properties. Agent Interface interface Agent { /** Accessibility action constants */ constants: AgentConstants; /** All automation actions */ actions: AgentActions; /** Utility functions */ utils: AgentUtils; /** Device and automation info */ info: AgentInfo; /** Automation control */ control: AgentControl; /** HTML overlay display */ display: AgentDisplay; /** Email operations */ email: AgentEmail; /** Notification handling */ notifications: AgentNotifications; /** Record usage statistics */ recordUsage(type: string, usage: number): void; } Namespaces agent.constants 44 accessibility action constants used with nodeAction() /docs/automation/reference/agent/constants agent.actions 26 methods for device interaction: touch gestures, text input, app management, screenshots, and more /docs/automation/reference/agent/actions agent.utils Utility functions including randomClick, randomSwipe, callbacks, and file operations /docs/automation/reference/agent/utils agent.info Get automation and device metadata /docs/automation/reference/agent/info agent.control Stop automation execution /docs/automation/reference/agent/control agent.display Display HTML overlays on screen /docs/automation/reference/agent/display agent.email Read emails via IMAP protocol /docs/automation/reference/agent/email agent.notifications Handle and respond to system notifications /docs/automation/reference/agent/notifications Quick Example // Tap at coordinates await agent.actions.tap(100, 200); // Get screen content and find elements const screen = await agent.actions.screenContent(); const button = screen.findTextOne("Submit"); // Perform action on a node if (button) { await button.performAction(agent.constants.ACTION_CLICK); } // Get device info const device = agent.info.getDeviceInfo(); console.log(device.brand, device.model); // Read a file const content = agent.utils.files.readFullFile("/sdcard/data.txt"); -------------------------------------------------------------------------------- ## Reference / agent.constants Path: /docs/automation/reference/agent/constants Description: Accessibility action constants for use with performAction() Content: node.performAction() method. data performAction() to pass additional arguments. next ACTION_FOCUS ACTION_CLEAR_FOCUS ACTION_SELECT Select the node ACTION_CLEAR_SELECTION Clear selection from the node ACTION_CLICK Perform a click action on the node ACTION_LONG_CLICK Perform a long click action on the node ACTION_ACCESSIBILITY_FOCUS ACTION_CLEAR_ACCESSIBILITY_FOCUS ACTION_NEXT_AT_MOVEMENT_GRANULARITY Move to next element at given granularity ACTION_PREVIOUS_AT_MOVEMENT_GRANULARITY Move to previous element at given granularity ACTION_NEXT_HTML_ELEMENT Navigate to next HTML element ACTION_PREVIOUS_HTML_ELEMENT Navigate to previous HTML element ACTION_SCROLL_FORWARD Scroll forward (down or right) ACTION_SCROLL_BACKWARD Scroll backward (up or left) ACTION_SCROLL_UP Scroll up ACTION_SCROLL_DOWN Scroll down ACTION_SCROLL_LEFT Scroll left ACTION_SCROLL_RIGHT Scroll right ACTION_SCROLL_TO_POSITION Scroll to a specific position ACTION_SCROLL_IN_DIRECTION Scroll in a specific direction ACTION_PAGE_UP Page up ACTION_PAGE_DOWN Page down ACTION_PAGE_LEFT Page left ACTION_PAGE_RIGHT Page right ACTION_CUT Cut text to clipboard ACTION_COPY Copy text to clipboard ACTION_PASTE Paste text from clipboard ACTION_SET_SELECTION Set text selection range ACTION_SET_TEXT Set the text content ACTION_EXPAND Expand a collapsible node ACTION_COLLAPSE Collapse an expanded node ACTION_DISMISS Dismiss/close a dismissable node ACTION_SHOW_ON_SCREEN Scroll to make the node visible ACTION_SET_PROGRESS Set progress value (e.g., SeekBar) ACTION_CONTEXT_CLICK Perform context click (right-click) ACTION_SHOW_TOOLTIP Show tooltip ACTION_HIDE_TOOLTIP Hide tooltip ACTION_PRESS_AND_HOLD Press and hold the node ACTION_IME_ENTER Submit via IME enter key ACTION_SHOW_TEXT_SUGGESTIONS Show text suggestions ACTION_MOVE_WINDOW Move a window ACTION_DRAG_START Start a drag operation ACTION_DRAG_DROP Drop at current position ACTION_DRAG_CANCEL Cancel the drag operation ACTION_ARGUMENT_MOVEMENT_GRANULARITY_INT Movement granularity for navigation ACTION_ARGUMENT_HTML_ELEMENT_STRING HTML element type for navigation ACTION_ARGUMENT_EXTEND_SELECTION_BOOLEAN Whether to extend selection ACTION_ARGUMENT_SELECTION_START_INT Selection start position ACTION_ARGUMENT_SELECTION_END_INT Selection end position ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE Text to set ACTION_ARGUMENT_MOVE_WINDOW_X Window X position ACTION_ARGUMENT_MOVE_WINDOW_Y Window Y position ACTION_ARGUMENT_ACCESSIBLE_CLICKABLE_SPAN android.view.accessibility.action.ACTION_ARGUMENT_ACCESSIBLE_CLICKABLE_SPAN Clickable span reference ARGUMENT_PRESS_AND_HOLD_DURATION_MILLIS_INT android.view.accessibility.action.ARGUMENT_PRESS_AND_HOLD_DURATION_MILLIS_INT Press and hold duration in ms ARGUMENT_DIRECTION_INT android.view.accessibility.action.ARGUMENT_DIRECTION_INT Direction for scroll operations ARGUMENT_SCROLL_AMOUNT_FLOAT android.view.accessibility.action.ARGUMENT_SCROLL_AMOUNT_FLOAT Scroll amount/distance // Click a node using accessibility action const screen = await agent.actions.screenContent(); const button = screen.findTextOne("Submit"); if (button) { await button.performAction(agent.constants.ACTION_CLICK); } // Set text on an input field const input = screen.findAdvanced(f => f.isEditText()); if (input) { await input.performAction( agent.constants.ACTION_SET_TEXT, { [agent.constants.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE]: "Hello World" } ); } agent.constants · Reference agent.constants Interface Accessibility action constants for use with node.performAction(). Usage Example Basic Actions Navigation Actions Scroll Actions Editing Actions Expand/Collapse Actions Advanced Actions Drag Actions Argument Constants -------------------------------------------------------------------------------- ## Reference / agent.actions Path: /docs/automation/reference/agent/actions Description: All automation actions for device interaction Sections: - Action Categories - Quick Example Content: ; swipe(x1: number, y1: number, x2: number, y2: number, duration: number): Promise ; hold(x: number, y: number, duration: number): Promise ; doubleTap(x: number, y: number, interval: number): Promise ; multiTap(sequence: MultiTapSequenceItem[]): Promise ; swipePoly(startX: number, startY: number, sequence: Point[], duration: number, bezier?: boolean): Promise ; // Navigation (dpad requires Android 13+) goHome(): Promise ; goBack(): Promise ; recents(): Promise ; dpad(direction: "up" | "down" | "left" | "right" | "center"): Promise ; // Text Input (inputKey requires Android 13+) writeText(text: string): Promise ; copyText(text: string): Promise ; paste(): Promise ; reverseCopy(): Promise ; hideKeyboard(): Promise ; inputKey(keyCode: number, duration?: number, state?: "down" | "up" | null): Promise ; // App Management launchApp(packageName: string, clearExisting?: boolean): Promise ; launchIntent(...): Promise ; listApps(): Promise ; browse(url: string, clearExistingData?: boolean): Promise ; // Screen Operations screenContent(): Promise ; allScreensContent(): Promise ; screenshot(maxWidth: number, maxHeight: number, quality: number, ...): Promise ; nodeAction(node: AndroidNode, actionInt: number, data?: object): Promise ; showNotification(title: string, message: string): Promise ; // File Operations saveFile(fileName: string, data: string, base64?: boolean): Promise ; // Network airplane(): Promise ; // Image Recognition recognizeText(imageBase64: string): Promise Access these methods through agent.actions . All action methods are asynchronous and return Promises. Action Categories Quick Example next agent.actions · Reference interface AgentActions { // Touch Gestures tap(x: number, y: number): Promise; swipe(x1: number, y1: number, x2: number, y2: number, duration: number): Promise; hold(x: number, y: number, duration: number): Promise; doubleTap(x: number, y: number, interval: number): Promise; multiTap(sequence: MultiTapSequenceItem[]): Promise; swipePoly(startX: number, startY: number, sequence: Point[], duration: number, bezier?: boolean): Promise; // Navigation (dpad requires Android 13+) goHome(): Promise; goBack(): Promise; recents(): Promise; dpad(direction: "up" | "down" | "left" | "right" | "center"): Promise; // Text Input (inputKey requires Android 13+) writeText(text: string): Promise; copyText(text: string): Promise; paste(): Promise; reverseCopy(): Promise; hideKeyboard(): Promise; inputKey(keyCode: number, duration?: number, state?: "down" | "up" | null): Promise; // App Management launchApp(packageName: string, clearExisting?: boolean): Promise; launchIntent(...): Promise; listApps(): Promise<{[packageName: string]: string}>; browse(url: string, clearExistingData?: boolean): Promise; // Screen Operations screenContent(): Promise; allScreensContent(): Promise; screenshot(maxWidth: number, maxHeight: number, quality: number, ...): Promise; nodeAction(node: AndroidNode, actionInt: number, data?: object): Promise<{actionPerformed: boolean}>; showNotification(title: string, message: string): Promise; // File Operations saveFile(fileName: string, data: string, base64?: boolean): Promise; // Network airplane(): Promise; // Image Recognition recognizeText(imageBase64: string): Promise; // ADB Actions (app version 2.141+) adb: AgentAdbActions; } // Get the current screen content const screen = await agent.actions.screenContent(); // Find a button with text "Submit" const submitBtn = screen.findTextOne("Submit"); // Tap the button if (submitBtn) { const { left, top, right, bottom } = submitBtn.boundsInScreen; await agent.actions.tap((left + right) / 2, (top + bottom) / 2); } // Or use performAction for accessibility-based click await submitBtn.performAction(agent.constants.ACTION_CLICK); Touch Actions tap, swipe, hold, doubleTap, multiTap, swipePoly /docs/automation/reference/agent/actions/touch Navigation goHome, goBack, recents, dpad (Android 13+) /docs/automation/reference/agent/actions/navigation Text Input writeText, copyText, paste, reverseCopy, hideKeyboard, inputKey (Android 13+) /docs/automation/reference/agent/actions/text App Management launchApp, launchIntent, listApps, browse /docs/automation/reference/agent/actions/apps Screen Operations screenContent, allScreensContent, screenshot, nodeAction, showNotification /docs/automation/reference/agent/actions/screen File Operations saveFile - save files to device storage /docs/automation/reference/agent/actions/files Network airplane - toggle airplane mode to refresh IP on mobile network /docs/automation/reference/agent/actions/network Image Recognition recognizeText - OCR using ML Kit /docs/automation/reference/agent/actions/recognition ADB Actions ADB shell-based alternatives: tap, swipe, hold, goHome, screenContent, listApps, and more (app v2.141+) /docs/automation/reference/agent/actions/adb AgentActions Interface All automation actions for device interaction AgentActions Interface Overview -------------------------------------------------------------------------------- ## Reference / agent.actions.touch Path: /docs/automation/reference/agent/actions/touch Description: Touch gestures for device interaction Methods: ### tap() Signature: tap(x: number, y: number): Promise Taps the screen at the given device-pixel coordinates. ### swipe() Signature: swipe(from: Point, to: Point, ms?: number): Promise Swipes between two points over the given duration. ### hold() Signature: hold(x: number, y: number, ms?: number): Promise Long-presses at the coordinates for the given duration. ### doubleTap() Signature: doubleTap(x: number, y: number, interval: number): Promise Performs a double tap at the specified coordinates. ### multiTap() Signature: multiTap(sequence: MultiTapSequenceItem[]): Promise Performs multiple taps in sequence with configurable delays. ### swipePoly() Signature: swipePoly(startX: number, startY: number, sequence: {x: number, y: number, duration?: number}[], duration: number, bezier?: boolean): Promise Performs a multi-point swipe through a sequence of coordinates. Each point can optionally specify its own segment duration for fine-grained timing control. Content: agent.actions . All touch methods are asynchronous and return Promises. next agent.actions.touch · Reference Touch Actions Actions Touch gestures for device interaction tap(x: number, y: number): Promise Taps the screen at the given device-pixel coordinates. number X coordinate in device pixels. Y coordinate in device pixels. Promise Resolves once the tap is dispatched. Simple tap await agent.actions.tap(100, 200); Tap center of a node const node = screen.findTextOne("Submit"); if (node) { const { left, top, right, bottom } = node.boundsInScreen; await agent.actions.tap((left + right) / 2, (top + bottom) / 2); } swipe swipe(from: Point, to: Point, ms?: number): Promise Swipes between two points over the given duration. from Point Start point { x, y }. End point { x, y }. Swipe duration. 400 Resolves once the swipe completes. Swipe up await agent.actions.swipe({ x: 500, y: 1500 }, { x: 500, y: 500 }, 300); Swipe right await agent.actions.swipe({ x: 100, y: 500 }, { x: 900, y: 500 }, 200); hold hold(x: number, y: number, ms?: number): Promise Long-presses at the coordinates for the given duration. X coordinate. Y coordinate. Hold duration. 600 Resolves once released. await agent.actions.hold(500, 500, 1000); // Hold for 1 second doubleTap doubleTap(x: number, y: number, interval: number): Promise Performs a double tap at the specified coordinates. X coordinate Y coordinate interval Interval between taps in milliseconds Resolves when double tap is complete await agent.actions.doubleTap(500, 500, 100); multiTap multiTap(sequence: MultiTapSequenceItem[]): Promise Performs multiple taps in sequence with configurable delays. sequence MultiTapSequenceItem[] Array of {x, y, delay} objects where delay is ms to wait after each tap Resolves when all taps are complete await agent.actions.multiTap([ { x: 100, y: 200, delay: 100 }, { x: 300, y: 400, delay: 100 }, { x: 500, y: 600, delay: 0 } ]); swipePoly swipePoly(startX: number, startY: number, sequence: {x: number, y: number, duration?: number}[], duration: number, bezier?: boolean): Promise Performs a multi-point swipe through a sequence of coordinates. Each point can optionally specify its own segment duration for fine-grained timing control. startX Starting X coordinate startY Starting Y coordinate {x, y, duration?}[] Array of points to swipe through. Each point can optionally include a duration (ms) for that segment. duration Total duration in milliseconds. Divided equally among segments if per-point durations are not specified. bezier boolean Use bezier curve interpolation Resolves when swipe is complete // Draw a zigzag pattern await agent.actions.swipePoly(100, 500, [ { x: 300, y: 400 }, { x: 500, y: 600 }, { x: 700, y: 400 } ], 500); With per-segment duration // Slow first segment, fast second await agent.actions.swipePoly(100, 100, [ { x: 100, y: 500, duration: 400 }, { x: 400, y: 500, duration: 100 } ], 0); -------------------------------------------------------------------------------- ## Reference / agent.actions.navigation Path: /docs/automation/reference/agent/actions/navigation Description: Device navigation and system button actions Methods: ### goHome() Signature: goHome(): Promise Returns to the home screen. ### goBack() Signature: goBack(): Promise Presses the system back button. ### recents() Signature: recents(): Promise Opens the recent apps screen. ### dpad() Signature: dpad(direction: "up" | "down" | "left" | "right" | "center"): Promise Sends a D-pad navigation event. Useful for navigating lists and menus. Requires Android 13+ (SDK level 33+). Content: Access these methods through agent.actions . Navigate between screens and interact with system buttons. Note: dpad() agent.info.getDeviceInfo().sdkVersion next goHome goHome(): Promise Returns to the home screen. Promise Resolves when navigation is complete. await agent.actions.goHome(); goBack goBack(): Promise Presses the system back button. Resolves when back action is complete. await agent.actions.goBack(); recents recents(): Promise Opens the recent apps screen. Resolves when recent apps screen is shown. await agent.actions.recents(); dpad dpad(direction: "up" | "down" | "left" | "right" | "center"): Promise Sends a D-pad navigation event. Useful for navigating lists and menus. Requires Android 13+ (SDK level 33+). direction "up" | "down" | "left" | "right" | "center" Direction to navigate. await agent.actions.dpad("down"); await agent.actions.dpad("center"); // Select/Enter agent.actions.navigation · Reference agent.actions.navigation Actions Device navigation and system button actions. warning -------------------------------------------------------------------------------- ## Reference / agent.actions.text Path: /docs/automation/reference/agent/actions/text Description: Keyboard input and clipboard operations Methods: ### writeText() Signature: writeText(text: string): Promise Types text using keyboard input. The keyboard must be visible. ### copyText() Signature: copyText(text: string): Promise Copies the specified text to the system clipboard. ### paste() Signature: paste(): Promise Pastes content from the clipboard at the current cursor position. ### reverseCopy() Signature: reverseCopy(): Promise<{text: string, data?: any, files?: {uri: string, mimeType: string, name: string, dataBase64: string}[]}> Gets the current clipboard content including text, data, and files. ### hideKeyboard() Signature: hideKeyboard(): Promise Hides the software keyboard if it's currently visible. ### inputKey() Signature: inputKey(keyCode: number, duration?: number, state?: "down" | "up" | null): Promise Sends a raw key event by Android KeyEvent code. Requires Android 13+ (SDK level 33+) and only works when the on-screen keyboard is visible. Content: Access these methods through agent.actions . Type text, manage clipboard, and handle keyboard interactions. Note: inputKey() requires Android 13+ (SDK level 33+) and will only work when the on-screen keyboard is visible. Tap on an input field first to show the keyboard. next writeText writeText(text: string): Promise Types text using keyboard input. The keyboard must be visible. text string Text to type. Promise Resolves when text is typed. await agent.actions.writeText("Hello World"); copyText copyText(text: string): Promise Copies the specified text to the system clipboard. Text to copy to clipboard. Resolves when copy is complete. await agent.actions.copyText("Text to copy"); paste paste(): Promise Pastes content from the clipboard at the current cursor position. Resolves when paste is complete. await agent.actions.paste(); reverseCopy reverseCopy(): Promise<{text: string, data?: any, files?: {uri: string, mimeType: string, name: string, dataBase64: string}[]}> Gets the current clipboard content including text, data, and files. {text, data?, files?} Clipboard content with text and optional binary data/files. const clipboard = await agent.actions.reverseCopy(); console.log(clipboard.text); if (clipboard.files) { console.log("Files:", clipboard.files.map(f => f.name)); } hideKeyboard hideKeyboard(): Promise Hides the software keyboard if it's currently visible. Resolves when keyboard is hidden. await agent.actions.hideKeyboard(); inputKey inputKey(keyCode: number, duration?: number, state?: "down" | "up" | null): Promise Sends a raw key event by Android KeyEvent code. Requires Android 13+ (SDK level 33+) and only works when the on-screen keyboard is visible. keyCode number Android KeyEvent code (e.g., 66 for ENTER). duration Press duration in ms. state "down" | "up" | null "down" for press, "up" for release, null for full press. Resolves when key event is sent. Press Enter await agent.actions.inputKey(66); Press and hold await agent.actions.inputKey(66, 500, "down"); agent.actions.text · Reference agent.actions.text Actions Keyboard input and clipboard operations. warning -------------------------------------------------------------------------------- ## Reference / agent.actions.apps Path: /docs/automation/reference/agent/actions/apps Description: Launch apps, manage intents, and browse URLs Methods: ### launchApp() Signature: launchApp(packageName: string, clearExisting?: boolean): Promise Launches an app by its package name. ### launchIntent() Signature: launchIntent(intentName: string, packageName: string | null, data: string | null, type: string | null, extras: object | null, flags: number, component: "activity" | "service" | "broadcast", isDataLocal?: boolean): Promise Launches an Android Intent with full configuration options. ### listApps() Signature: listApps(): Promise<{[packageName: string]: string}> Gets a list of all installed apps. ### browse() Signature: browse(url: string, clearExistingData?: boolean): Promise Opens a URL in the default browser. Content: Access these methods through agent.actions . Launch and manage applications on the device. next launchApp launchApp(packageName: string, clearExisting?: boolean): Promise Launches an app by its package name. packageName string The app's package name (e.g., 'com.android.settings'). clearExisting boolean Close the existing app before launching. false Promise Resolves when app is launched. await agent.actions.launchApp("com.android.settings"); Launch with fresh data await agent.actions.launchApp("com.example.app", true); launchIntent launchIntent(intentName: string, packageName: string | null, data: string | null, type: string | null, extras: object | null, flags: number, component: "activity" | "service" | "broadcast", isDataLocal?: boolean): Promise Launches an Android Intent with full configuration options. intentName Action name (e.g., "android.intent.action.VIEW"). string | null Target package name. data Intent data URI. type MIME type. extras object | null Extra data as key-value pairs. flags number Intent flags. component "activity" | "service" | "broadcast" Component type to launch. isDataLocal If true, data is a local file path. Resolves when intent is launched. Open a URL await agent.actions.launchIntent( "android.intent.action.VIEW", null, "https://example.com", null, null, 0, "activity" ); listApps listApps(): Promise<{[packageName: string]: string}> Gets a list of all installed apps. {[packageName: string]: string} Object mapping package names to app display names. const apps = await agent.actions.listApps(); console.log(apps["com.android.chrome"]); // "Chrome" browse browse(url: string, clearExistingData?: boolean): Promise Opens a URL in the default browser. URL to open. clearExistingData Clear browser data before opening. Resolves when browser is launched. await agent.actions.browse("https://example.com"); agent.actions.apps · Reference agent.actions.apps Actions Launch apps, manage intents, and browse URLs. -------------------------------------------------------------------------------- ## Reference / agent.actions.screen Path: /docs/automation/reference/agent/actions/screen Description: Screen content, screenshots, and node interactions Methods: ### screenContent() Signature: screenContent(): Promise Gets the accessibility tree of the currently focused window. Returns an AndroidNode representing the root of the UI hierarchy. ### allScreensContent() Signature: allScreensContent(): Promise Gets the accessibility trees from all visible windows (useful for dialogs, overlays). ### screenshot() Signature: screenshot(maxWidth: number, maxHeight: number, quality: number, cropX1?: number, cropY1?: number, cropX2?: number, cropY2?: number): Promise<{screenshot: string | null, compressedWidth: number, compressedHeight: number, originalWidth: number, originalHeight: number}> Takes a screenshot with optional scaling and cropping. ### nodeAction() Signature: nodeAction(node: AndroidNode | object, actionInt: number, data?: object, fieldsToIgnore?: string[]): Promise<{actionPerformed: boolean}> Performs an accessibility action on a node. ### showNotification() Signature: showNotification(title: string, message: string): Promise Shows a system notification. Content: Access these methods through agent.actions . Get screen content, take screenshots, and interact with UI nodes. next screenContent screenContent(): Promise Gets the accessibility tree of the currently focused window. Returns an AndroidNode representing the root of the UI hierarchy. Promise Root node of the accessibility tree. const screen = await agent.actions.screenContent(); // Find elements const button = screen.findTextOne("Submit"); const allButtons = screen.filterAdvanced(f => f.isButton()); const input = screen.findAdvanced(f => f.isEditText().isEditable()); allScreensContent allScreensContent(): Promise Gets the accessibility trees from all visible windows (useful for dialogs, overlays). Promise Array of root nodes for each window. const screens = await agent.actions.allScreensContent(); for (const screen of screens) { const dialog = screen.findTextOne("OK"); if (dialog) break; } screenshot screenshot(maxWidth: number, maxHeight: number, quality: number, cropX1?: number, cropY1?: number, cropX2?: number, cropY2?: number): Promise<{screenshot: string | null, compressedWidth: number, compressedHeight: number, originalWidth: number, originalHeight: number}> Takes a screenshot with optional scaling and cropping. maxWidth number Maximum width to scale to. maxHeight Maximum height to scale to. quality JPEG quality (1-100). cropX1 Crop region left. cropY1 Crop region top. cropX2 Crop region right. cropY2 Crop region bottom. {screenshot, compressedWidth, compressedHeight, originalWidth, originalHeight} Screenshot data as base64 string with dimensions. Full screenshot const result = await agent.actions.screenshot(1080, 1920, 80); Cropped screenshot const result = await agent.actions.screenshot(500, 500, 90, 100, 100, 600, 600); nodeAction nodeAction(node: AndroidNode | object, actionInt: number, data?: object, fieldsToIgnore?: string[]): Promise<{actionPerformed: boolean}> Performs an accessibility action on a node. node AndroidNode | object The node to perform action on. actionInt Action constant (use agent.constants.ACTION_*). data object Additional action data. fieldsToIgnore string[] Node fields to ignore when matching. {actionPerformed: boolean} Whether the action was successfully performed. Using node.performAction (preferred) const screen = await agent.actions.screenContent(); const button = screen.findTextOne("Submit"); if (button) { const result = await button.performAction(agent.constants.ACTION_CLICK); console.log("Clicked:", result.actionPerformed); } Using agent.actions.nodeAction (legacy) const result = await agent.actions.nodeAction( button, agent.constants.ACTION_CLICK ); showNotification showNotification(title: string, message: string): Promise Shows a system notification. title string Notification title. message Notification message. Promise Resolves when notification is shown. await agent.actions.showNotification("Task Complete", "Your automation has finished."); agent.actions.screen · Reference agent.actions.screen Actions Screen content, screenshots, and node interactions. -------------------------------------------------------------------------------- ## Reference / agent.actions.files Path: /docs/automation/reference/agent/actions/files Description: Save files to the device Methods: ### saveFile() Signature: saveFile(fileName: string, data: string, base64?: boolean): Promise Saves data to a file in the Downloads folder. Content: Access these methods through agent.actions . Save data to files on the device. Note: Utils > Files next saveFile saveFile(fileName: string, data: string, base64?: boolean): Promise Saves data to a file in the Downloads folder. fileName string Name of the file to create. data Content to save. base64 boolean If true, data is base64-encoded binary. false Promise Resolves when file is saved. Save text file await agent.actions.saveFile("output.txt", "Hello World"); Save JSON data const data = { name: "John", age: 30 }; await agent.actions.saveFile("data.json", JSON.stringify(data, null, 2)); Save binary file (base64) // Save a screenshot const { screenshot } = await agent.actions.screenshot(1080, 1920, 80); if (screenshot) { await agent.actions.saveFile("screenshot.jpg", screenshot, true); } Save CSV export const rows = [ ["Name", "Email", "Status"], ["John", "john@example.com", "Active"], ["Jane", "jane@example.com", "Inactive"] ]; const csv = rows.map(row => row.join(",")).join("\n"); await agent.actions.saveFile("export.csv", csv); async function exportResults(results: any[]) { // Save as JSON await agent.actions.saveFile( "results.json", JSON.stringify(results, null, 2) ); // Save as CSV const headers = Object.keys(results[0]); const csvRows = [ headers.join(","), ...results.map(r => headers.map(h => r[h]).join(",")) ]; await agent.actions.saveFile("results.csv", csvRows.join("\n")); // Take a confirmation screenshot const { screenshot } = await agent.actions.screenshot(1080, 1920, 80); if (screenshot) { await agent.actions.saveFile("confirmation.jpg", screenshot, true); } console.log("Results exported to Downloads folder"); } agent.actions.files · Reference agent.actions.files Actions Save files to the device. info /docs/automation/reference/agent/utils/files underline Complete Example: Export Automation Results -------------------------------------------------------------------------------- ## Reference / agent.actions.network Path: /docs/automation/reference/agent/actions/network Description: Network and connectivity operations Methods: ### airplane() Signature: airplane(): Promise Toggles airplane mode on and off to refresh the mobile network connection. This method turns airplane mode ON, waits briefly, then turns it OFF. Primarily used for changing IP address when connected to a mobile/cellular network. ### adb.airplane() Signature: adb.airplane(): Promise Toggles airplane mode on and off via ADB shell commands. Same effect as agent.actions.airplane() but does not require the app to be set as the device assistant — only a working ADB connection. Useful when the assistant role can't be granted but ADB is available. Content: Access these methods through agent.actions . Manage network connectivity and IP address changes. Important Notes Only works when the device is connected to a mobile/cellular network. Wi-Fi connections are not affected by this method. The new IP address is assigned by your mobile carrier. There may be a brief period of no connectivity during the toggle. next airplane airplane(): Promise Toggles airplane mode on and off to refresh the mobile network connection. This method turns airplane mode ON, waits briefly, then turns it OFF. Primarily used for changing IP address when connected to a mobile/cellular network. Promise Resolves after airplane mode cycle completes. Change IP address on mobile network // Refresh mobile network to get a new IP await agent.actions.airplane(); console.log("Mobile network refreshed with new IP"); Retry with new IP on failure async function fetchWithIPRefresh(url: string) { try { // First attempt return await fetch(url); } catch (error) { // Refresh IP and retry await agent.actions.airplane(); await agent.control.wait(2000); // Wait for network to stabilize return await fetch(url); } } adb.airplane adb.airplane(): Promise Toggles airplane mode on and off via ADB shell commands. Same effect as agent.actions.airplane() but does not require the app to be set as the device assistant — only a working ADB connection. Useful when the assistant role can't be granted but ADB is available. Promise Resolves with true if the toggle was kicked off, false if ADB is not connected. Rotate IP via ADB const ok = await agent.actions.adb.airplane(); if (!ok) { console.warn("ADB not connected — could not rotate IP"); } // When rate limited, get new IP and continue async function handleRateLimit() { console.log("Rate limited, refreshing IP..."); await agent.actions.airplane(); // Wait for network to fully reconnect const status = await agent.utils.isServerReachable(); if (!status.reachable) { await agent.control.wait(3000); } console.log("IP refreshed, continuing automation"); } agent.actions.network · Reference agent.actions.network Actions Network and connectivity operations. warning Use Case: Rate Limit Bypass -------------------------------------------------------------------------------- ## Reference / agent.actions.recognition Path: /docs/automation/reference/agent/actions/recognition Description: OCR and image analysis capabilities Sections: - Return Types Methods: ### recognizeText() Signature: recognizeText(imageBase64: string): Promise Performs OCR on an image using ML Kit. ### findImage() Signature: findImage(image: string, images: string[], threshold?: number): Promise Locates one or more template images within a source image using OpenCV multi-scale template matching. Useful for finding icons or fixed UI elements on a screenshot when accessibility nodes are unreliable. Requires Android 11+ (API 30+) and RemoteMobile app version code 195+. Content: Access these methods through agent.actions . Extract text and analyze images using ML Kit, or locate icons in screenshots with OpenCV template matching. Return Types next recognizeText recognizeText(imageBase64: string): Promise Performs OCR on an image using ML Kit. imageBase64 string Base64-encoded image. TextJSON Hierarchical text structure with confidence and bounding boxes. const { screenshot } = await agent.actions.screenshot(1080, 1920, 90); const result = await agent.actions.recognizeText(screenshot); console.log("Full text:", result.text); // Access individual text blocks for (const block of result.textBlocks) { console.log("Block:", block.text, "at", block.boundingBox); } findImage findImage(image: string, images: string[], threshold?: number): Promise Locates one or more template images within a source image using OpenCV multi-scale template matching. Useful for finding icons or fixed UI elements on a screenshot when accessibility nodes are unreliable. Requires Android 11+ (API 30+) and RemoteMobile app version code 195+. 195 image Base64-encoded source image to search within (e.g. a screenshot). images string[] Array of base64-encoded template images to find. threshold number Confidence threshold between 0.0 and 1.0 (default 0.7). FindImageResult Per-template results array. Each entry corresponds to the same index in the input images array. // Take a screenshot and look for an icon (e.g. a delete bin button) const device = agent.info.getDeviceInfo(); if (device.sdkVersion < 30 || device.appVersionCode < 195) { console.log("findImage not supported on this device"); return; } const { screenshot } = await agent.actions.screenshot(device.width, device.height, 90); const templates = await (await fetch("templates.json")).json(); const result = await agent.actions.findImage( screenshot, [templates.bin, templates.bin_dark], 0.7, ); for (const match of result.results) { if (match.found && match.bounds) { console.log(`Template ${match.index} found at (${match.x}, ${match.y}) with confidence ${match.confidence?.toFixed(3)}`); agent.utils.randomClick( match.bounds.left, match.bounds.top, match.bounds.right, match.bounds.bottom, ); } } Root level OCR result containing all recognized text. interface TextJSON { text: string; // Complete recognized text textBlocks: TextBlock[]; // Array of text blocks } TextBlock A block of text, typically a paragraph. interface TextBlock { text: string; boundingBox: BoundingBox; cornerPoints: Point[]; recognizedLanguages: string[]; lines: TextLine[]; } TextLine A line of text within a block. interface TextLine { text: string; boundingBox: BoundingBox; cornerPoints: Point[]; recognizedLanguages: string[]; elements: TextElement[]; confidence: number; angle: number; } TextElement Individual text element (usually a word). interface TextElement { text: string; boundingBox: BoundingBox; cornerPoints: Point[]; recognizedLanguages: string[]; symbols: TextSymbol[]; confidence: number; angle: number; } TextSymbol Individual character/symbol. interface TextSymbol { text: string; boundingBox: BoundingBox; cornerPoints: Point[]; confidence: number; angle: number; } BoundingBox interface BoundingBox { left: number; top: number; right: number; bottom: number; } Top-level result returned by findImage. interface FindImageResult { results: FindImageMatchResult[]; // One entry per template image (same order as input) } FindImageMatchResult Match information for a single template. Coordinates and bounds are only present when found is true. interface FindImageMatchResult { index: number; // Index of the template in the input images array found: boolean; // Whether a match was found above the threshold x?: number; // Center x-coordinate of the match (only if found) y?: number; // Center y-coordinate of the match (only if found) confidence?: number; // Match confidence 0.0 - 1.0 (only if found) bounds?: BoundingBox; // Bounding box of the matched region (only if found) scale?: number; // Scale at which the template was found (only if found) } agent.actions.recognition · Reference agent.actions.recognition Actions OCR and image analysis capabilities. -------------------------------------------------------------------------------- ## Reference / agent.utils Path: /docs/automation/reference/agent/utils Description: Utility functions, job management, and file operations Sections: - Utility Categories - Quick Examples Content: Access these utilities through agent.utils . Includes random gesture helpers, event callbacks, job task management, server connectivity, and comprehensive file operations. ; submit(type: "outOfSteps" | "timeout" | "debug"): Promise , finish: boolean, files: File[]): Promise ; useAnotherTask(): Promise ; getCurrentTask(): Promise ; set(data: Partial ): Promise ; set(data: Record Utility Categories For notification callbacks, see agent.notifications. Quick Examples next agent.utils · Reference Interface Utility functions, job management, and file operations AgentUtils Interface Overview interface AgentUtils { // Gesture helpers randomClick(x1: number, y1: number, x2: number, y2: number): void; randomSwipe(x1: number, y1: number, x2: number, y2: number, direction: Direction): void; // Server connectivity isServerReachable(): Promise<{ reachable: true } | { reachable: false; error: string }>; // Event callbacks (for notifications, see agent.notifications) setNetworkCallback(callback: NetworkCallback | null): void; toastCallback: ToastCallback | null; // Step tracking & debugging outOfSteps: { storeScreen(screen: AndroidNode, stage: string, screenState: string, remainingSteps: number, screenshotRecord: ScreenshotRecord): Promise; submit(type: "outOfSteps" | "timeout" | "debug"): Promise; }; // Job task management job: { submitTask(status: AutomationStatus, data: Record, finish: boolean, files: File[]): Promise; useAnotherTask(): Promise; getCurrentTask(): Promise; }; // Bucket storage (device+job scoped) bucket: { get(): Promise; set(data: Partial): Promise; }; // Device bucket storage (device-only scoped) deviceBucket: { get(): Promise; set(data: Record): Promise; }; // File operations files: AgentFiles; } agent.utils.helpers randomClick, randomSwipe, isServerReachable, waitForNode /docs/automation/reference/agent/utils/helpers agent.utils.callbacks setNetworkCallback, toastCallback /docs/automation/reference/agent/utils/callbacks agent.utils.job submitTask, useAnotherTask, getCurrentTask, addSubTasks /docs/automation/reference/agent/utils/job agent.utils.outOfSteps storeScreen, submit — debug automation failures /docs/automation/reference/agent/utils/out-of-steps agent.utils.bucket get, set — device+job scoped persistent storage /docs/automation/reference/agent/utils/bucket agent.utils.deviceBucket get, set — device-scoped persistent storage across all jobs /docs/automation/reference/agent/utils/device-bucket agent.utils.files exists, readFullFile, list, getHashes, and more /docs/automation/reference/agent/utils/files Note Human-like Interactions // Random tap within a button area (more human-like) const button = screen.findTextOne("Submit"); if (button) { button.randomClick(); } Job Task Processing // Get current task and submit results const task = await agent.utils.job.getCurrentTask(); if (task.success) { const proof = task.job_proof; // ... perform automation ... await agent.utils.job.submitTask( "success", { orderId: "12345", completed: true }, true, // Final submission [] ); } File Operations // Read and process files const files = agent.utils.files.list("/sdcard/Download"); for (const file of files) { if (file.name.endsWith(".json")) { const content = agent.utils.files.readFullFile(file.path); const data = JSON.parse(content); console.log(data); } } -------------------------------------------------------------------------------- ## Reference / agent.utils.helpers Path: /docs/automation/reference/agent/utils/helpers Description: Random gesture helpers, server connectivity, and node waiting utilities Methods: ### randomClick() Signature: randomClick(x1: number, y1: number, x2: number, y2: number): void Performs a tap at a random position within the specified rectangle. Useful for making automations appear more human-like. ### randomSwipe() Signature: randomSwipe(x1: number, y1: number, x2: number, y2: number, direction: "up" | "down" | "left" | "right"): void Performs a swipe starting from a random position within the rectangle, moving in the specified direction. ### isServerReachable() Signature: isServerReachable(): Promise<{ reachable: true } | { reachable: false; error: string }> Checks if the server is reachable. Useful for verifying connectivity before performing server-dependent operations. ### waitForNode() Signature: waitForNode(condition: (node: AndroidNode) => boolean, durationMs?: number, intervalMs?: number): Promise Waits for a node matching the condition to appear on screen. Polls the screen at regular intervals until the condition is met or timeout is reached. ### waitForNodeGone() Signature: waitForNodeGone(condition: (node: AndroidNode) => boolean, durationMs?: number, intervalMs?: number): Promise Waits for a node matching the condition to disappear from screen. Polls the screen at regular intervals until the node is gone or timeout is reached. Content: boolean, durationMs?: number, intervalMs?: number): Promise next randomClick randomClick(x1: number, y1: number, x2: number, y2: number): void Performs a tap at a random position within the specified rectangle. Useful for making automations appear more human-like. x1 number Left boundary y1 Top boundary x2 Right boundary y2 Bottom boundary Using node.randomClick (preferred) const button = screen.findTextOne("Submit"); if (button) { button.randomClick(); } Using agent.utils.randomClick (legacy) const { left, top, right, bottom } = button.boundsInScreen; agent.utils.randomClick(left, top, right, bottom); randomSwipe randomSwipe(x1: number, y1: number, x2: number, y2: number, direction: "up" | "down" | "left" | "right"): void Performs a swipe starting from a random position within the rectangle, moving in the specified direction. direction "up" | "down" | "left" | "right" Swipe direction Using node.randomSwipe (preferred) const scrollView = screen.findAdvanced(f => f.isScrollable()); if (scrollView) { scrollView.randomSwipe("up"); } Using agent.utils.randomSwipe (legacy) agent.utils.randomSwipe(100, 500, 900, 1500, "up"); isServerReachable isServerReachable(): Promise<{ reachable: true } | { reachable: false; error: string }> Checks if the server is reachable. Useful for verifying connectivity before performing server-dependent operations. { reachable: true } | { reachable: false; error: string } Object indicating server reachability status const status = await agent.utils.isServerReachable(); if (status.reachable) { console.log("Server is reachable"); } else { console.log("Server unreachable:", status.error); } waitForNode waitForNode(condition: (node: AndroidNode) => boolean, durationMs?: number, intervalMs?: number): Promise Waits for a node matching the condition to appear on screen. Polls the screen at regular intervals until the condition is met or timeout is reached. condition (node: AndroidNode) => boolean Function that returns true when the desired node is found durationMs Maximum time to wait in milliseconds 30000 intervalMs Polling interval in milliseconds 500 boolean true if the node was found, false if timeout was reached Wait for a button to appear const found = await agent.utils.waitForNode( node => node.text === "Continue" && node.isClickable, 10000, // 10 second timeout 500 // Check every 500ms ); if (found) { const screen = await agent.actions.screenContent(); screen.findTextOne("Continue")?.click(); } Wait for loading to complete const contentLoaded = await agent.utils.waitForNode( node => node.contentDescription?.includes("Main content"), 15000 ); if (!contentLoaded) { console.log("Timeout waiting for content to load"); } waitForNodeGone waitForNodeGone(condition: (node: AndroidNode) => boolean, durationMs?: number, intervalMs?: number): Promise Waits for a node matching the condition to disappear from screen. Polls the screen at regular intervals until the node is gone or timeout is reached. Function that identifies the node to wait for disappearance true if the node disappeared, false if timeout was reached Wait for loading spinner to disappear const spinnerGone = await agent.utils.waitForNodeGone( node => node.className?.includes("ProgressBar"), 20000 // 20 second timeout ); if (spinnerGone) { console.log("Loading complete"); } else { console.log("Loading took too long"); } Wait for dialog to close screen.findTextOne("Dismiss")?.click(); const dialogClosed = await agent.utils.waitForNodeGone( node => node.text === "Are you sure?", 5000 ); agent.utils.helpers · Reference agent.utils.helpers Random gesture helpers, server connectivity, and node waiting utilities. Accessed through agent.utils. -------------------------------------------------------------------------------- ## Reference / agent.utils.callbacks Path: /docs/automation/reference/agent/utils/callbacks Description: Network and toast event handlers Sections: - toastCallback Methods: ### setNetworkCallback() Signature: setNetworkCallback(callback: NetworkCallback | null): void Registers a callback to receive network state changes. Pass null to unregister. Content: For notification callbacks, see agent.notifications. toastCallback Set this property to receive toast messages shown by apps. next setNetworkCallback setNetworkCallback(callback: NetworkCallback | null): void Registers a callback to receive network state changes. Pass null to unregister. callback (networkAvailable: boolean) => void Callback function or null to unregister agent.utils.setNetworkCallback((networkAvailable) => { if (!networkAvailable) { console.log("Network disconnected!"); } }); agent.utils.callbacks · Reference agent.utils.callbacks Network and toast event handlers. Register callbacks through agent.utils to receive system events during automation. toastCallback: ((packageName: string, data: { message: string }) => void) | null Example agent.utils.toastCallback = (packageName, data) => { console.log("Toast from", packageName + ":", data.message); }; NetworkCallback type NetworkCallback = (networkAvailable: boolean) => void; ToastCallback type ToastCallback = ( packageName: string, data: { message: string } ) => void; -------------------------------------------------------------------------------- ## Reference / agent.utils.job Path: /docs/automation/reference/agent/utils/job Description: Manage job tasks, submit results, and request new tasks Sections: - Automation Variables Methods: ### submitTask() Signature: submitTask(automationStatus: "running" | "success" | "failed" | "declined", data: Record, finish: boolean, files: { name: string; extension: string; base64Data: string }[]): Promise<{ success: false; error: string } | { success: true }> Submits the current task result to the server. Use this to report progress, success, or failure of job tasks. Once you call submitTask with finish=true, you cannot submit again for the same task. The files parameter is ignored when finish=false. ### submitTaskToAnotherJob() Signature: submitTaskToAnotherJob(job_id: string, data: Record, status?: "pending" | "failed" | "declined", files?: { name: string; extension: string; base64Data: string }[]): Promise<{ success: false; error: string } | { success: true; job_task_id: string; job_id: string }> Writes a task into a separate "data-dump" job from inside a running automation. The target job must be a devices_automation job that has automation_id set and no planned tasks; the running job and the target must share the same automation_id; and the running job\'s owner must own (or have shared_view / shared_edit access to) the target. Use this to push structured data — leads, scrape results, side-channel facts — into a sink job while the main automation keeps running.\n\nDifferences from submitTask:\n• submitTask updates the currently running task; submitTaskToAnotherJob creates a brand-new task on a different job.\n• job_proof must be a JSON-serializable object (sent JSON-encoded).\n• Files use the {name, extension, base64Data} shape; their public URLs are inlined into the proof under proof[name] — no separate file fields.\n• status is the data-dump status: "pending" (default), "failed", or "declined". This is independent of the running automation status.\n• Skips manual-mode gates (rating / country / few_times / custom_vars / proof_files / positions / task_limit / Worker_Invites) and skips completion emails.\n• Does not affect the running task or the running automation\'s iteration / success-rate / blacklist analytics.\n\nBacked by POST /api/v2/tasks/submit (automation data-dump branch). ### useAnotherTask() Signature: useAnotherTask(): Promise<{ job_task_id: string; job_proof: string } | null> Accesses another task's data from the same job, allowing one job task to retrieve and use data from another. The target task must have set its automation variables with waiting: true via setAutomationVariables to be discoverable. Returns the task details or null if no waiting task is available. ### getCurrentTask() Signature: getCurrentTask(): Promise<{ success: false; error: string } | { success: true; parent_task_id: string; job_proof: any, timeout: number }> Gets information about the currently assigned task, including the parent task ID and job proof data. ### addSubTasks() Signature: addSubTasks(job_variables: JobVariables[], run_immediately?: boolean): Promise<...> Creates sub-tasks (planned tasks) for the current job or another job owned by the same employer. Each item in job_variables becomes a separate planned task. When job_id is omitted, the current task's job ID is used automatically (and cached). When targeting a different job, the parent task's owner must own or have shared_edit access to the target, and the target must have allow_sub_tasks enabled. Supports three call signatures: positional with JobVariables[], positional with a custom job_id, or a single options object (including remote_device_id). ### getSubTasks() Signature: getSubTasks(planned_task_ids: string[]): Promise<...> Gets the status and details of sub-tasks by their planned task IDs. Returns a map keyed by planned_task_id. Only returns data for planned tasks whose parent_task_id matches the current task. Status is "planned" if not yet assigned, "deleted" if the job task was removed, or the actual job task status (e.g. "running", "confirmed", "failed") otherwise. ### setAutomationVariables() Signature: setAutomationVariables(variables: object): Promise<{ success: false; error: string } | { success: true; automation_variables: any }> Sets automation variables for the current job task to coordinate between tasks in the same job. Set { waiting: true } to make the current task's data available to other tasks via useAnotherTask(). ### getAutomationVariables() Signature: getAutomationVariables(): Promise<{ success: false; error: string } | { success: true; automation_variables: any }> Retrieves the automation variables previously set for the current job task. Use this to check the current state of task coordination variables. Content: []", description: "Array of job variable objects; each becomes the input for one sub-task. Uses JobVariables when targeting the current job, or Record ; submitTaskToAnotherJob( job_id: string, data: Record ; useAnotherTask(): Promise ; getCurrentTask(): Promise ; addSubTasks( job_variables: JobVariables[], run_immediately?: boolean ): Promise ; addSubTasks( job_variables: Record [], run_immediately: boolean, job_id: string ): Promise ; getSubTasks(planned_task_ids: string[]): Promise ; getAutomationVariables(): Promise Automation Variables Accessed via agent.utils (not agent.utils.job), but used for coordinating between job tasks. next submitTask submitTask(automationStatus: "running" | "success" | "failed" | "declined", data: Record, finish: boolean, files: { name: string; extension: string; base64Data: string }[]): Promise<{ success: false; error: string } | { success: true }> Submits the current task result to the server. Use this to report progress, success, or failure of job tasks. Once you call submitTask with finish=true, you cannot submit again for the same task. The files parameter is ignored when finish=false. automationStatus "running" | "success" | "failed" | "declined" Current status of the task data Record Task result data as key-value pairs finish boolean Whether this is the final submission. Once true, no more submissions are allowed for this task. files { name, extension, base64Data }[] Files to upload with the task. Ignored when finish=false. { success: true } | { success: false; error: string } Success status or error message Submit successful task with data const result = await agent.utils.job.submitTask( "success", { orderId: "12345", totalAmount: 99.99, itemsProcessed: 3 }, true, // Final submission [] // No files ); if (result.success) { console.log("Task submitted successfully"); } Submit task with files const screenshot = await agent.actions.screenshot(1080, 1920, 80); const result = await agent.utils.job.submitTask( "success", { status: "completed" }, true, [{ name: "confirmation", extension: "jpg", base64Data: screenshot.screenshot || "" }] ); Report progress (not finished) // files parameter is ignored when finish=false await agent.utils.job.submitTask( "running", { currentStep: 3, totalSteps: 10 }, false, [] ); Decline a task await agent.utils.job.submitTask( "declined", { reason: "Item out of stock" }, true, [] ); submitTaskToAnotherJob submitTaskToAnotherJob(job_id: string, data: Record, status?: "pending" | "failed" | "declined", files?: { name: string; extension: string; base64Data: string }[]): Promise<{ success: false; error: string } | { success: true; job_task_id: string; job_id: string }> Writes a task into a separate "data-dump" job from inside a running automation. The target job must be a devices_automation job that has automation_id set and no planned tasks; the running job and the target must share the same automation_id; and the running job's owner must own (or have shared_view / shared_edit access to) the target. Use this to push structured data — leads, scrape results, side-channel facts — into a sink job while the main automation keeps running. Differences from submitTask: • submitTask updates the currently running task; submitTaskToAnotherJob creates a brand-new task on a different job. • job_proof must be a JSON-serializable object (sent JSON-encoded). • Files use the {name, extension, base64Data} shape; their public URLs are inlined into the proof under proof[name] — no separate file fields. • status is the data-dump status: "pending" (default), "failed", or "declined". This is independent of the running automation status. • Skips manual-mode gates (rating / country / few_times / custom_vars / proof_files / positions / task_limit / Worker_Invites) and skips completion emails. • Does not affect the running task or the running automation's iteration / success-rate / blacklist analytics. Backed by POST /api/v2/tasks/submit (automation data-dump branch). job_id string ID of the target data-dump job (devices_automation, automation_id set, no planned tasks). Proof data; serialized to JSON and stored as job_proof on the new task. status "pending" | "failed" | "declined" Status of the new task on the target job. Defaults to "pending". Optional files. Each is written to disk and its public URL embedded into the JSON proof under proof[name]. { success: true; job_task_id: string; job_id: string } | { success: false; error: string } On success, the created task and target job ids; otherwise an error. Push structured data into a sink job const result = await agent.utils.job.submitTaskToAnotherJob( "68868530c189957861cd698a", // target data-dump job_id { email: "lead@example.com", name: "Jane Doe", capturedAt: Date.now() }, ); if (result.success) { console.log("Wrote dump task", result.job_task_id, "to job", result.job_id); } Attach a screenshot to the dump const screenshot = await agent.actions.screenshot(1080, 1920, 80); await agent.utils.job.submitTaskToAnotherJob( "68868530c189957861cd698a", { note: "Profile page state at capture" }, "pending", [{ name: "snapshot", extension: ".jpg", base64Data: screenshot.screenshot || "" }], ); Mark a dump as failed await agent.utils.job.submitTaskToAnotherJob( "68868530c189957861cd698a", { reason: "captcha blocked" }, "failed", ); useAnotherTask useAnotherTask(): Promise<{ job_task_id: string; job_proof: string } | null> Accesses another task's data from the same job, allowing one job task to retrieve and use data from another. The target task must have set its automation variables with waiting: true via setAutomationVariables to be discoverable. Returns the task details or null if no waiting task is available. { job_task_id: string; job_proof: string } | null Task details from another waiting task, or null if none available Access data from another task in the same job // First, the other task marks itself as waiting: // await agent.utils.setAutomationVariables({ waiting: true }); const otherTask = await agent.utils.job.useAnotherTask(); if (otherTask) { console.log("Found waiting task:", otherTask.job_task_id); const proof = JSON.parse(otherTask.job_proof); const sharedData = proof.someSharedField; } else { console.log("No waiting task available"); } Coordinate between multiple tasks // Task A: Set up data and wait await agent.utils.job.submitTask( "running", { preparedData: "value", step: "waiting" }, false, [] ); await agent.utils.setAutomationVariables({ waiting: true }); // Task B: Access Task A's data const taskA = await agent.utils.job.useAnotherTask(); if (taskA) { const taskAData = JSON.parse(taskA.job_proof); console.log("Got data from Task A:", taskAData.preparedData); } getCurrentTask getCurrentTask(): Promise<{ success: false; error: string } | { success: true; parent_task_id: string; job_proof: any, timeout: number }> Gets information about the currently assigned task, including the parent task ID and job proof data. { success: true; parent_task_id: string; job_proof: any, timeout: number } | { success: false; error: string } Current task details or error const task = await agent.utils.job.getCurrentTask(); if (task.success) { console.log("Current task ID:", task.parent_task_id); const targetUrl = task.job_proof.url; const credentials = task.job_proof.credentials; } else { console.log("Error getting task:", task.error); } addSubTasks addSubTasks(job_variables: JobVariables[], run_immediately?: boolean): Promise<...> Creates sub-tasks (planned tasks) for the current job or another job owned by the same employer. Each item in job_variables becomes a separate planned task. When job_id is omitted, the current task's job ID is used automatically (and cached). When targeting a different job, the parent task's owner must own or have shared_edit access to the target, and the target must have allow_sub_tasks enabled. Supports three call signatures: positional with JobVariables[], positional with a custom job_id, or a single options object (including remote_device_id). job_variables JobVariables[] | Record[] Array of job variable objects; each becomes the input for one sub-task. Uses JobVariables when targeting the current job, or Record when a custom job_id is provided. run_immediately Whether to attempt immediate task assignment after creation. Defaults to true. Target job ID. If omitted, uses the current task's job ID (fetched once and cached). Different jobs require shared_edit access and allow_sub_tasks enabled. remote_device_id Pin sub-tasks to a specific device by its remote_device_id. Only available via the options object overload. { success: true; inserted_ids: { planned_task_id: string; input: string }[]; assignment_results?: { planned_task_id: string; assigned: boolean }[] } | { success: false; error: string } Inserted planned task IDs with their inputs, and optional assignment results when run_immediately is true Create sub-tasks for the current job const result = await agent.utils.job.addSubTasks([ { email: "user1@example.com", action: "verify" }, { email: "user2@example.com", action: "verify" }, ]); if (result.success) { const ids = result.inserted_ids.map(t => t.planned_task_id); } Create sub-tasks without immediate assignment const result = await agent.utils.job.addSubTasks( [{ url: "https://example.com/page1" }, { url: "https://example.com/page2" }], false // Don't assign immediately ); Create sub-tasks for a different job const result = await agent.utils.job.addSubTasks( [{ targetId: "abc123" }], true, "674a1b2c3d4e5f6a7b8c9d0e" // Different job ID ); Options object — pin sub-tasks to a specific device const result = await agent.utils.job.addSubTasks({ job_variables: [{ query: "search term 1" }, { query: "search term 2" }], run_immediately: true, remote_device_id: "a1b2c3d4-e5f6-7890-abcd-ef1234567890", }); Options object — all options const result = await agent.utils.job.addSubTasks({ job_variables: [{ action: "scrape", url: "https://example.com" }], run_immediately: false, job_id: "674a1b2c3d4e5f6a7b8c9d0e", remote_device_id: "a1b2c3d4-e5f6-7890-abcd-ef1234567890", }); getSubTasks getSubTasks(planned_task_ids: string[]): Promise<...> Gets the status and details of sub-tasks by their planned task IDs. Returns a map keyed by planned_task_id. Only returns data for planned tasks whose parent_task_id matches the current task. Status is "planned" if not yet assigned, "deleted" if the job task was removed, or the actual job task status (e.g. "running", "confirmed", "failed") otherwise. planned_task_ids string[] Array of planned task IDs to query (max 100), from addSubTasks. { success: true; tasks: Record } | { success: false; error: string } Map of planned_task_id to task details. Status is "planned", "deleted", or the job task status. Check status of sub-tasks const result = await agent.utils.job.getSubTasks([ "674a1b2c3d4e5f6a7b8c9d01", "674a1b2c3d4e5f6a7b8c9d02", ]); if (result.success) { for (const [plannedTaskId, task] of Object.entries(result.tasks)) { console.log(`Task ${plannedTaskId}: status=${task.status}`); if (task.status === "confirmed") console.log("Result:", task.job_proof); } } Poll sub-tasks until all complete const subTaskResult = await agent.utils.job.addSubTasks([ { query: "search term 1" }, { query: "search term 2" }, ]); if (!subTaskResult.success) throw new Error(subTaskResult.error); const ids = subTaskResult.inserted_ids.map(t => t.planned_task_id); while (true) { const status = await agent.utils.job.getSubTasks(ids); if (!status.success) break; const allDone = Object.values(status.tasks).every( t => ["confirmed", "failed", "declined", "deleted"].includes(t.status) ); if (allDone) break; await sleepRandom(5000, 10000); } setAutomationVariables setAutomationVariables(variables: object): Promise<{ success: false; error: string } | { success: true; automation_variables: any }> Sets automation variables for the current job task to coordinate between tasks in the same job. Set { waiting: true } to make the current task's data available to other tasks via useAnotherTask(). variables object Variables to set. Use { waiting: true } to mark this task as available for other tasks. { success: true; automation_variables: any } | { success: false; error: string } Success status with the stored variables, or error message Mark task as waiting for another task await agent.utils.setAutomationVariables({ waiting: true }); // Now another task in the same job can call useAnotherTask() Store custom variables for task coordination const result = await agent.utils.setAutomationVariables({ waiting: true, stage: "data_prepared", timestamp: Date.now() }); if (result.success) { console.log("Variables set:", result.automation_variables); } else { console.error("Failed to set variables:", result.error); } getAutomationVariables getAutomationVariables(): Promise<{ success: false; error: string } | { success: true; automation_variables: any }> Retrieves the automation variables previously set for the current job task. Use this to check the current state of task coordination variables. Check current automation variables const result = await agent.utils.getAutomationVariables(); if (result.success) { if (result.automation_variables?.waiting) { console.log("This task is marked as waiting"); } } else { console.error("Failed to get variables:", result.error); } Retrieve stored coordination state const result = await agent.utils.getAutomationVariables(); if (result.success && result.automation_variables) { const { stage, timestamp } = result.automation_variables; console.log(`Task is at stage: ${stage}, set at: ${timestamp}`); } agent.utils.job · Reference agent.utils.job Manage job tasks, submit results, and request new tasks. Accessed through agent.utils.job. JobUtils Interface interface JobUtils { submitTask( automationStatus: "running" | "success" | "failed" | "declined", data: Record, finish: boolean, files: { name: string; extension: string; base64Data: string }[] ): Promise<{ success: false; error: string } | { success: true }>; submitTaskToAnotherJob( job_id: string, data: Record, status?: "pending" | "failed" | "declined", files?: { name: string; extension: string; base64Data: string }[] ): Promise< { success: false; error: string } | { success: true; job_task_id: string; job_id: string } >; useAnotherTask(): Promise<{ job_task_id: string; job_proof: string } | null>; getCurrentTask(): Promise< { success: false; error: string } | { success: true; parent_task_id: string; job_proof: any, timeout: number } >; addSubTasks( job_variables: JobVariables[], run_immediately?: boolean ): Promise< { success: false; error: string } | { success: true; inserted_ids: { planned_task_id: string; input: string }[]; assignment_results?: { planned_task_id: string; assigned: boolean }[]; } >; addSubTasks( job_variables: Record[], run_immediately: boolean, job_id: string ): Promise< { success: false; error: string } | { success: true; inserted_ids: { planned_task_id: string; input: string }[]; assignment_results?: { planned_task_id: string; assigned: boolean }[]; } >; addSubTasks(options: { job_variables: Record[]; run_immediately?: boolean; job_id?: string; remote_device_id?: string; }): Promise< { success: false; error: string } | { success: true; inserted_ids: { planned_task_id: string; input: string }[]; assignment_results?: { planned_task_id: string; assigned: boolean }[]; } >; getSubTasks(planned_task_ids: string[]): Promise< { success: false; error: string } | { success: true; tasks: Record; } >; } // On agent.utils (for task coordination) interface AgentUtils { setAutomationVariables(variables: object): Promise< { success: false; error: string } | { success: true; automation_variables: any } >; getAutomationVariables(): Promise< { success: false; error: string } | { success: true; automation_variables: any } >; } -------------------------------------------------------------------------------- ## Reference / agent.utils.out-of-steps Path: /docs/automation/reference/agent/utils/out-of-steps Description: Track automation progress and debug failures Sections: - Supporting Types - Usage Pattern Methods: ### storeScreen() Signature: storeScreen(screen: AndroidNode, stage: string, screenState: string, remainingSteps: number, screenshotRecord: ScreenshotRecord): Promise Stores the current screen state for debugging purposes. Call this periodically during automation to track progress and help diagnose issues when automations fail. ### submit() Signature: submit(type: "outOfSteps" | "timeout" | "debug"): Promise<{ success: false; error: string } | { success: true; id: string }> Submits the collected screen states for analysis. Call this when the automation ends unexpectedly or for debugging. Content: agent.utils.outOfSteps . Used for tracking automation progress and debugging when automations run out of steps or timeout. ; submit( type: "outOfSteps" | "timeout" | "debug" ): Promise Supporting Types Usage Pattern next storeScreen storeScreen(screen: AndroidNode, stage: string, screenState: string, remainingSteps: number, screenshotRecord: ScreenshotRecord): Promise Stores the current screen state for debugging purposes. Call this periodically during automation to track progress and help diagnose issues when automations fail. screen AndroidNode The current screen content from screenContent() stage string Current stage/phase of the automation (e.g., 'login', 'checkout') screenState Description of the current screen state remainingSteps number Number of steps remaining in the automation screenshotRecord ScreenshotRecord Screenshot quality setting const screen = await agent.actions.screenContent(); // Store screen state for debugging await agent.utils.outOfSteps.storeScreen( screen, "login", "waiting_for_credentials", 50, ScreenshotRecord.LOW_QUALITY ); submit submit(type: "outOfSteps" | "timeout" | "debug"): Promise<{ success: false; error: string } | { success: true; id: string }> Submits the collected screen states for analysis. Call this when the automation ends unexpectedly or for debugging. type "outOfSteps" | "timeout" | "debug" Reason for submission { success: true; id: string } | { success: false; error: string } Result with submission ID on success or error message on failure Submit on timeout const result = await agent.utils.outOfSteps.submit("timeout"); if (result.success) { console.log("Debug data submitted with ID:", result.id); } else { console.log("Failed to submit:", result.error); } Submit for debugging // Collect screens during automation await agent.utils.outOfSteps.storeScreen(screen1, "step1", "initial", 100, ScreenshotRecord.HIGH_QUALITY); await agent.utils.outOfSteps.storeScreen(screen2, "step2", "processing", 80, ScreenshotRecord.LOW_QUALITY); // Submit for analysis await agent.utils.outOfSteps.submit("debug"); agent.utils.outOfSteps · Reference Step Tracking & Debugging Utils Track automation progress and debug failures OutOfStepsUtils Interface interface OutOfStepsUtils { storeScreen( screen: AndroidNode, stage: string, screenState: string, remainingSteps: number, screenshotRecord: ScreenshotRecord ): Promise; submit( type: "outOfSteps" | "timeout" | "debug" ): Promise<{ success: false; error: string } | { success: true; id: string }>; } Enum for screenshot quality settings. enum ScreenshotRecord { HIGH_QUALITY, // Full quality screenshot LOW_QUALITY, // Compressed screenshot (faster, smaller) NONE // No screenshot } Complete Debugging Flow async function runAutomation() { let remainingSteps = 100; while (remainingSteps > 0) { const screen = await agent.actions.screenContent(); // Store screen state periodically await agent.utils.outOfSteps.storeScreen( screen, getCurrentStage(), describeScreen(screen), remainingSteps, ScreenshotRecord.LOW_QUALITY ); // Perform automation step const result = await performStep(screen); if (!result.success) { // Submit debug data on failure await agent.utils.outOfSteps.submit("outOfSteps"); throw new Error("Automation failed"); } remainingSteps--; } // Submit on timeout if loop exits without success await agent.utils.outOfSteps.submit("timeout"); } -------------------------------------------------------------------------------- ## Reference / agent.utils.bucket Path: /docs/automation/reference/agent/utils/bucket Description: Device+job scoped persistent storage that survives across task iterations Sections: - Bucket Schema Methods: ### get() Signature: get(): Promise<{ success: false; error: string } | { success: true; bucket: Bucket }> Retrieves the current bucket data for this device+job combination. Returns the stored data object, or an empty object if no bucket exists yet. ### set() Signature: set(data: Partial): Promise<{ success: false; error: string } | { success: true; bucket: Bucket }> Merges the provided data into the existing bucket for this device+job combination. New keys are added and existing keys are overwritten. Keys not included in the provided data are preserved. If a bucket schema is defined on the automation, the merged data is validated against it. Content: ): Promise agent.utils.bucket . Bucket data is scoped to a specific device + job combination and persists across task iterations. Use it to store session tokens, login state, or any data that should survive between tasks on the same device. Bucket is only available when the automation is running through a job. Calling these methods outside of a job context will return an error. ; set(data: Partial Bucket Schema You can optionally define a bucket schema in the project editor under the Inputs tab. When a schema is defined: The Bucket type is generated with the correct fields, giving you autocomplete and type checking Data passed to set() is validated against the schema on the server Device Bucket next next/link get(): Promise<{ success: false; error: string } | { success: true; bucket: Bucket }> Retrieves the current bucket data for this device+job combination. Returns the stored data object, or an empty object if no bucket exists yet. { success: true; bucket: Bucket } | { success: false; error: string } The stored bucket data on success, or an error message Read bucket data const result = await agent.utils.bucket.get(); if (result.success) { console.log("Bucket data:", result.bucket); // Access stored values const token = result.bucket.sessionToken; const isLoggedIn = result.bucket.loggedIn; } else { console.error("Failed to get bucket:", result.error); } Check if a session exists before logging in const result = await agent.utils.bucket.get(); if (result.success && result.bucket.sessionToken) { console.log("Reusing existing session"); // Skip login, use stored session } else { console.log("No session found, performing login..."); // Perform login flow } set(data: Partial): Promise<{ success: false; error: string } | { success: true; bucket: Bucket }> Merges the provided data into the existing bucket for this device+job combination. New keys are added and existing keys are overwritten. Keys not included in the provided data are preserved. If a bucket schema is defined on the automation, the merged data is validated against it. data Partial Object containing the key-value pairs to merge into the bucket. Existing keys not present in this object are preserved. The full merged bucket data on success, or an error message Store a session token after login // After successful login, persist the session const result = await agent.utils.bucket.set({ sessionToken: "abc123xyz", loggedIn: true, loginTimestamp: Date.now(), }); if (result.success) { console.log("Session saved:", result.bucket); } Merge additional data (existing keys are preserved) // First call: store login info await agent.utils.bucket.set({ sessionToken: "abc123", loggedIn: true, }); // Second call: add more data without losing sessionToken or loggedIn await agent.utils.bucket.set({ lastChecked: Date.now(), itemsProcessed: 5, }); // Bucket now contains all four keys: // { sessionToken, loggedIn, lastChecked, itemsProcessed } Overwrite a specific key // Update just the session token (other keys are preserved) await agent.utils.bucket.set({ sessionToken: "newToken456", }); agent.utils.bucket · Reference Bucket Storage Utils Device+job scoped persistent storage that survives across task iterations Note BucketUtils Interface interface BucketUtils { get(): Promise< { success: false; error: string } | { success: true; bucket: Bucket } >; set(data: Partial): Promise< { success: false; error: string } | { success: true; bucket: Bucket } >; } Typical Usage Pattern // At the start of automation: check for existing state const bucket = await agent.utils.bucket.get(); if (bucket.success && bucket.bucket.loggedIn) { // Resume from previous session console.log("Resuming with token:", bucket.bucket.sessionToken); } else { // Fresh start - perform login const token = await performLogin(); // Save state for next iteration await agent.utils.bucket.set({ sessionToken: token, loggedIn: true, }); } /docs/automation/reference/agent/utils/device-bucket -------------------------------------------------------------------------------- ## Reference / agent.utils.deviceBucket Path: /docs/automation/reference/agent/utils/device-bucket Description: Device-scoped persistent storage shared across all jobs and automations Sections: - Comparison with Job Bucket Methods: ### get() Signature: get(): Promise<{ success: false; error: string } | { success: true; deviceBucket: Record }> Retrieves the current device bucket data. Returns the stored data object, or an empty object if no device bucket exists yet. ### set() Signature: set(data: Record): Promise<{ success: false; error: string } | { success: true; deviceBucket: Record }> Merges the provided data into the existing device bucket. New keys are added and existing keys are overwritten. Keys not included in the provided data are preserved. Content: ): Promise agent.utils.deviceBucket . Device bucket data is scoped to a specific device and persists across all jobs and automations running on that device. Use it to store account credentials, app configurations, or any data that should be shared across all automations on the same device. Unlike agent.utils.bucket, device bucket does not require a job context. It is available in all automation contexts, including direct-run automations. ; set(data: Record Comparison with Job Bucket Feature agent.utils.bucket next get(): Promise<{ success: false; error: string } | { success: true; deviceBucket: Record }> Retrieves the current device bucket data. Returns the stored data object, or an empty object if no device bucket exists yet. { success: true; deviceBucket: Record } | { success: false; error: string } The stored device bucket data on success, or an error message Read device bucket data const result = await agent.utils.deviceBucket.get(); if (result.success) { console.log("Device bucket:", result.deviceBucket); // Access stored values const token = result.deviceBucket.accountToken; const username = result.deviceBucket.username; } else { console.error("Failed to get device bucket:", result.error); } Check if account credentials exist before logging in const result = await agent.utils.deviceBucket.get(); if (result.success && result.deviceBucket.accountToken) { console.log("Reusing stored credentials"); // Skip login, use stored credentials } else { console.log("No credentials found, performing login..."); // Perform login flow } set(data: Record): Promise<{ success: false; error: string } | { success: true; deviceBucket: Record }> Merges the provided data into the existing device bucket. New keys are added and existing keys are overwritten. Keys not included in the provided data are preserved. data Record Object containing the key-value pairs to merge into the device bucket. Existing keys not present in this object are preserved. The full merged device bucket data on success, or an error message Store account credentials after login // After successful login, persist credentials for all automations const result = await agent.utils.deviceBucket.set({ accountToken: "abc123xyz", username: "user@example.com", loginTimestamp: Date.now(), }); if (result.success) { console.log("Credentials saved:", result.deviceBucket); } Merge additional data (existing keys are preserved) // First call: store credentials await agent.utils.deviceBucket.set({ accountToken: "abc123", username: "user1", }); // Second call: add more data without losing credentials await agent.utils.deviceBucket.set({ appVersion: "2.1.0", lastChecked: Date.now(), }); // Device bucket now contains all four keys: // { accountToken, username, appVersion, lastChecked } Scope Device + Job Device only Requires job context Yes No Schema validation Optional (via bucket_schema) None Shared between jobs No (separate per job) Yes (all jobs share it) Use case Job-specific session state Device-wide credentials, config agent.utils.deviceBucket · Reference Device Bucket Storage Utils Device-scoped persistent storage shared across all jobs and automations Note DeviceBucketUtils Interface interface DeviceBucketUtils { get(): Promise< { success: false; error: string } | { success: true; deviceBucket: Record } >; set(data: Record): Promise< { success: false; error: string } | { success: true; deviceBucket: Record } >; } Typical Usage Pattern // At the start of any automation: check for device-level credentials const db = await agent.utils.deviceBucket.get(); if (db.success && db.deviceBucket.accountToken) { // Reuse credentials from any previous automation console.log("Using stored credentials for:", db.deviceBucket.username); } else { // First time on this device - perform account setup const { token, username } = await performAccountSetup(); // Save for all future automations on this device await agent.utils.deviceBucket.set({ accountToken: token, username: username, setupDate: Date.now(), }); } -------------------------------------------------------------------------------- ## Reference / agent.utils.files Path: /docs/automation/reference/agent/utils/files Description: Read, write, list, upload, and manage files on the device Sections: - Basic Operations - Reading Files - Streaming Large Files - Directory Operations - File Management - File Download - Download & Read - File Integrity - File Upload - Data Conversion - Types Methods: ### exists() Signature: exists(path: string): boolean Checks if a file or directory exists at the specified path. ### getSize() Signature: getSize(filePath: string): number Gets the size of a file in bytes. ### getStorageRoot() Signature: getStorageRoot(): string Gets the root storage path for the device (typically '/sdcard' or '/storage/emulated/0'). ### readFullFile() Signature: readFullFile(filePath: string): string Reads the entire file content as UTF-8 text. ### readFullFileBase64() Signature: readFullFileBase64(filePath: string): string Reads the entire file content as a Base64-encoded string. Useful for binary files. ### readFileAsBlob() Signature: readFileAsBlob(filePath: string, mimeType?: string): Blob | null Reads a file directly as a Blob object. ### openStream() Signature: openStream(filePath: string): string Opens a file stream for reading large files in chunks. ### readChunk() Signature: readChunk(streamId: string, chunkSize: number): number[] Reads a chunk of data from an open file stream. ### closeStream() Signature: closeStream(streamId: string): void Closes an open file stream. ### list() Signature: list(dirPath: string): FileInfo[] Lists all files and directories in the specified directory. ### getPathInfo() Signature: getPathInfo(path: string): DirectoryInfo | FilePathInfo | PathNotFoundInfo Gets detailed information about a path, including whether it's a file or directory. ### deleteFile() Signature: deleteFile(path: string): boolean Deletes a file at the specified path. Only works on files, not directories. Returns false if the path doesn't exist, is a directory, or deletion fails. ### deleteDir() Signature: deleteDir(path: string): boolean Deletes a directory and all its contents recursively. Only works on directories, not files. Returns false if the path doesn't exist, is a file, or deletion fails. ### rename() Signature: rename(oldPath: string, newPath: string): boolean Renames or moves a file or directory from oldPath to newPath. Parent directories for the new path are created automatically. Fails if the source doesn't exist or destination already exists. ### getDirPath() Signature: getDirPath(type: "Download" | "Movies" | "Music" | "Pictures" | "DCIM" | "Documents" | "Ringtones" | "Alarms" | "Notifications" | "Podcasts"): string Gets the external public directory path for a given type. ### startDownload() Signature: startDownload(url: string, localPath: string, options?: DownloadRequestOptions): string Starts downloading a file from a URL to a local path on the device. Returns a unique download ID that can be used to track progress with getDownloadStatus. Parent directories are created automatically. Supports resume on retry. Optionally specify HTTP method, headers, and body. ### getDownloadStatus() Signature: getDownloadStatus(id: string): DownloadStatusInfo Returns the current status of a download. Status can be "downloading", "success", or "failed". Includes progress info (bytesDownloaded, totalBytes) and result info (filePath, fileSize) on success. ### retryDownload() Signature: retryDownload(id: string): boolean Retries a failed download. Only works if the download is in failed state. The download resumes from where it left off if the server supports Range requests. ### fetch2() Signature: fetch2(url: string, options?: Fetch2Options): Promise<{ success: true; content: string | Blob | Uint8Array; size: number } | { success: false; error: string }> Downloads a file from a URL and returns its content directly. Internally downloads to a temporary file, reads the content, then deletes the temp file. On failure, automatically retries using resume-capable retryDownload. ### getHashes() Signature: getHashes(filePath: string): FileHashes | FileHashError Calculates MD5, SHA-1, and SHA-256 hashes for a file. ### uploadTempFile() Signature: uploadTempFile(filename: string, base64Data: string): Promise Uploads a file to the server as a temporary file. The file will be automatically deleted after 15 minutes. This overload accepts base64-encoded file data. ### uploadTempFile (local file)() Signature: uploadTempFile(localFilePath: string): Promise Uploads a local file from the device to the server as a temporary file. The file will be automatically deleted after 15 minutes. This overload reads the file in chunks to handle large files efficiently. ### base64ToBytes() Signature: base64ToBytes(base64: string): Uint8Array Converts a Base64-encoded string to a Uint8Array. ### bytesToBlob() Signature: bytesToBlob(bytes: number[] | Uint8Array, mimeType?: string): Blob | null Converts a byte array to a Blob object. Content: Access file operations through agent.utils.files . Comprehensive file system access for automations. For uploading files to the server, use agent.utils.uploadTempFile ; uploadTempFile(localFilePath: string): Promise Basic Operations Reading Files Streaming Large Files Directory Operations File Management File Download Download & Read File Integrity File Upload Data Conversion Types next exists exists(path: string): boolean Checks if a file or directory exists at the specified path. path string Path to check boolean true if the path exists if (agent.utils.files.exists("/sdcard/Download/data.json")) { const content = agent.utils.files.readFullFile("/sdcard/Download/data.json"); } getSize getSize(filePath: string): number Gets the size of a file in bytes. filePath Path to the file number File size in bytes, or -1 if not found const size = agent.utils.files.getSize("/sdcard/video.mp4"); getStorageRoot getStorageRoot(): string Gets the root storage path for the device (typically '/sdcard' or '/storage/emulated/0'). Storage root path const root = agent.utils.files.getStorageRoot(); const downloadPath = root + "/Download"; readFullFile readFullFile(filePath: string): string Reads the entire file content as UTF-8 text. File content as text const config = agent.utils.files.readFullFile("/sdcard/config.json"); const data = JSON.parse(config); readFullFileBase64 readFullFileBase64(filePath: string): string Reads the entire file content as a Base64-encoded string. Useful for binary files. Base64-encoded file content const imageBase64 = agent.utils.files.readFullFileBase64("/sdcard/DCIM/photo.jpg"); const ocrResult = await agent.actions.recognizeText(imageBase64); readFileAsBlob readFileAsBlob(filePath: string, mimeType?: string): Blob | null Reads a file directly as a Blob object. mimeType MIME type for the blob Blob | null Blob object or null on failure const imageBlob = agent.utils.files.readFileAsBlob("/sdcard/image.png", "image/png"); openStream openStream(filePath: string): string Opens a file stream for reading large files in chunks. Stream ID for use with readChunk and closeStream const streamId = agent.utils.files.openStream("/sdcard/large_file.bin"); let chunk; while ((chunk = agent.utils.files.readChunk(streamId, 1024 * 1024)).length > 0) { // Process chunk... } agent.utils.files.closeStream(streamId); readChunk readChunk(streamId: string, chunkSize: number): number[] Reads a chunk of data from an open file stream. streamId Stream ID from openStream() chunkSize Maximum bytes to read number[] Array of byte values (empty if end of file) closeStream closeStream(streamId: string): void Closes an open file stream. list list(dirPath: string): FileInfo[] Lists all files and directories in the specified directory. dirPath Directory path FileInfo[] Array of file/directory information const files = agent.utils.files.list("/sdcard/Download"); for (const file of files) { console.log(file.name, file.isDirectory ? "(dir)" : file.size + " bytes"); } getPathInfo getPathInfo(path: string): DirectoryInfo | FilePathInfo | PathNotFoundInfo Gets detailed information about a path, including whether it's a file or directory. DirectoryInfo | FilePathInfo | PathNotFoundInfo Detailed path information based on path type const info = agent.utils.files.getPathInfo("/sdcard/Download"); if (info.exists && info.isDirectory) { console.log("Contains", info.totalItems, "items"); } deleteFile deleteFile(path: string): boolean Deletes a file at the specified path. Only works on files, not directories. Returns false if the path doesn't exist, is a directory, or deletion fails. 2.138 (150) Path to the file to delete true if the file was deleted successfully const deleted = agent.utils.files.deleteFile("/sdcard/Download/temp.txt"); if (deleted) { console.log("File deleted"); } deleteDir deleteDir(path: string): boolean Deletes a directory and all its contents recursively. Only works on directories, not files. Returns false if the path doesn't exist, is a file, or deletion fails. Path to the directory to delete true if the directory was deleted successfully const deleted = agent.utils.files.deleteDir("/sdcard/Download/temp_folder"); if (deleted) { console.log("Directory and all contents deleted"); } rename rename(oldPath: string, newPath: string): boolean Renames or moves a file or directory from oldPath to newPath. Parent directories for the new path are created automatically. Fails if the source doesn't exist or destination already exists. oldPath Current path (file or directory) newPath New path true if renamed/moved successfully // Rename a file agent.utils.files.rename("/sdcard/Download/old.txt", "/sdcard/Download/new.txt"); // Move a file to another directory agent.utils.files.rename("/sdcard/Download/photo.jpg", "/sdcard/Pictures/photo.jpg"); // Rename a directory agent.utils.files.rename("/sdcard/Download/old_folder", "/sdcard/Download/new_folder"); getDirPath getDirPath(type: "Download" | "Movies" | "Music" | "Pictures" | "DCIM" | "Documents" | "Ringtones" | "Alarms" | "Notifications" | "Podcasts"): string Gets the external public directory path for a given type. type Directory type: "Download", "Movies", "Music", "Pictures", "DCIM", "Documents", "Ringtones", "Alarms", "Notifications", "Podcasts" Absolute path (e.g., "/storage/emulated/0/Download") const downloadDir = agent.utils.files.getDirPath("Download"); const files = agent.utils.files.list(downloadDir); console.log("Downloads:", files.length, "files"); const musicDir = agent.utils.files.getDirPath("Music"); startDownload startDownload(url: string, localPath: string, options?: DownloadRequestOptions): string Starts downloading a file from a URL to a local path on the device. Returns a unique download ID that can be used to track progress with getDownloadStatus. Parent directories are created automatically. Supports resume on retry. Optionally specify HTTP method, headers, and body. The URL to download from localPath The local file path to save the downloaded file to options DownloadRequestOptions Optional HTTP request options (method, headers, body) A unique download ID for tracking progress Start a download and poll for progress const downloadDir = agent.utils.files.getDirPath("Download"); const id = agent.utils.files.startDownload( "https://example.com/large-file.zip", downloadDir + "/large-file.zip" ); // Poll for progress let status; do { await sleep(1000); status = agent.utils.files.getDownloadStatus(id); if (status.totalBytes > 0) { const percent = Math.round((status.bytesDownloaded / status.totalBytes) * 100); console.log("Progress:", percent + "%"); } } while (status.status === "downloading"); if (status.status === "success") { console.log("Downloaded to:", status.filePath, "Size:", status.fileSize); } else { console.error("Failed:", status.error); } Download with custom headers and POST method const id = agent.utils.files.startDownload( "https://api.example.com/export", "/sdcard/Download/export.csv", { method: "POST", headers: { "Authorization": "Bearer my-token", "Content-Type": "application/json", }, body: JSON.stringify({ format: "csv", dateRange: "last30days" }), } ); getDownloadStatus getDownloadStatus(id: string): DownloadStatusInfo Returns the current status of a download. Status can be "downloading", "success", or "failed". Includes progress info (bytesDownloaded, totalBytes) and result info (filePath, fileSize) on success. The download ID returned by startDownload DownloadStatusInfo Object with download status, progress, and result info Check download status const status = agent.utils.files.getDownloadStatus(downloadId); console.log("Status:", status.status); console.log("Progress:", status.bytesDownloaded, "/", status.totalBytes); retryDownload retryDownload(id: string): boolean Retries a failed download. Only works if the download is in failed state. The download resumes from where it left off if the server supports Range requests. true if retry was started, false if download not found or not in failed state Retry a failed download const status = agent.utils.files.getDownloadStatus(downloadId); if (status.status === "failed") { console.log("Download failed:", status.error); const retried = agent.utils.files.retryDownload(downloadId); if (retried) console.log("Retrying..."); } fetch2 fetch2(url: string, options?: Fetch2Options): Promise<{ success: true; content: string | Blob | Uint8Array; size: number } | { success: false; error: string }> Downloads a file from a URL and returns its content directly. Internally downloads to a temporary file, reads the content, then deletes the temp file. On failure, automatically retries using resume-capable retryDownload. Fetch2Options Download options Promise<{ success: true; content: string | Blob | Uint8Array; size: number } | { success: false; error: string }> File content on success, or error on failure Download and parse JSON const result = await agent.utils.fetch2("https://example.com/data.json"); if (result.success) { const data = JSON.parse(result.content as string); console.log("Downloaded", result.size, "bytes"); } Download as base64 for upload const result = await agent.utils.fetch2("https://example.com/image.png", { readAs: "base64", }); if (result.success) { await agent.utils.uploadTempFile("image.png", result.content as string); } Download with custom headers and timeout const result = await agent.utils.fetch2("https://api.example.com/export", { readAs: "text", method: "POST", headers: { "Authorization": "Bearer my-token" }, body: JSON.stringify({ format: "csv" }), timeoutMs: 60_000, maxRetries: 3, }); if (!result.success) { console.error("Download failed:", result.error); } getHashes getHashes(filePath: string): FileHashes | FileHashError Calculates MD5, SHA-1, and SHA-256 hashes for a file. FileHashes | FileHashError Hash values or error const hashes = agent.utils.files.getHashes("/sdcard/download.apk"); if (!("error" in hashes)) { console.log("MD5:", hashes.md5); console.log("SHA-256:", hashes.sha256); } uploadTempFile uploadTempFile(filename: string, base64Data: string): Promise Uploads a file to the server as a temporary file. The file will be automatically deleted after 15 minutes. This overload accepts base64-encoded file data. filename Name for the uploaded file (including extension) base64Data UploadTempFileResult | { success: false; error: string } Upload result with file URL on success, or error on failure Upload a screenshot const screenshot = await agent.actions.screenshot(1080, 1920, 80); if (screenshot.screenshot) { const result = await agent.utils.uploadTempFile( "screenshot.jpg", screenshot.screenshot ); if (result.success) { console.log("File URL:", result.data.url); console.log("Expires at:", result.data.expiresAt); } else { console.error("Upload failed:", result.error); } } Upload text data as file const jsonData = JSON.stringify({ results: [1, 2, 3] }); const base64 = btoa(jsonData); const result = await agent.utils.uploadTempFile("results.json", base64); if (result.success) { console.log("Uploaded to:", result.data.url); } uploadTempFile (local file) uploadTempFile(localFilePath: string): Promise Uploads a local file from the device to the server as a temporary file. The file will be automatically deleted after 15 minutes. This overload reads the file in chunks to handle large files efficiently. localFilePath Absolute path to the file on the device Upload a downloaded file const result = await agent.utils.uploadTempFile("/sdcard/Download/report.pdf"); if (result.success) { console.log("Uploaded:", result.data.originalName); console.log("Size:", result.data.size, "bytes"); console.log("URL:", result.data.url); } else { console.error("Upload failed:", result.error); } Upload and share URL const imagePath = "/sdcard/DCIM/Camera/photo.jpg"; if (agent.utils.files.exists(imagePath)) { const result = await agent.utils.uploadTempFile(imagePath); if (result.success) { // Use the URL (valid for 15 minutes) await agent.utils.job.submitTask("success", { imageUrl: result.data.url }, true, []); } } base64ToBytes base64ToBytes(base64: string): Uint8Array Converts a Base64-encoded string to a Uint8Array. base64 Base64-encoded string Uint8Array Decoded byte array bytesToBlob bytesToBlob(bytes: number[] | Uint8Array, mimeType?: string): Blob | null Converts a byte array to a Blob object. bytes number[] | Uint8Array Byte array agent.utils.files · Reference File Operations Utils Read, write, list, upload, and manage files on the device AgentFiles Interface interface AgentFiles { exists(path: string): boolean; getSize(filePath: string): number; readFullFileBase64(filePath: string): string; readFullFile(filePath: string): string; openStream(filePath: string): string; readChunk(streamId: string, chunkSize: number): number[]; closeStream(streamId: string): void; list(dirPath: string): FileInfo[]; getPathInfo(path: string): DirectoryInfo | FilePathInfo | PathNotFoundInfo; getStorageRoot(): string; getHashes(filePath: string): FileHashes | FileHashError; deleteFile(path: string): boolean; // Since 2.138 deleteDir(path: string): boolean; // Since 2.138 rename(oldPath: string, newPath: string): boolean; // Since 2.138 getDirPath(type: string): string; // Since 2.138 startDownload(url: string, localPath: string, options?: DownloadRequestOptions): string; // Since 2.138 getDownloadStatus(id: string): DownloadStatusInfo; // Since 2.138 retryDownload(id: string): boolean; // Since 2.138 base64ToBytes(base64: string): Uint8Array; bytesToBlob(bytes: number[] | Uint8Array, mimeType?: string): Blob | null; readFileAsBlob(filePath: string, mimeType?: string): Blob | null; } // On agent.utils (for file upload) interface AgentUtils { uploadTempFile(filename: string, base64Data: string): Promise; uploadTempFile(localFilePath: string): Promise; } Status information for a file download. interface DownloadStatusInfo { id: string; // Unique download ID status: "downloading" | "success" | "failed"; error: string | null; // Error message if failed bytesDownloaded: number; // Bytes downloaded so far totalBytes: number; // Total size (-1 if unknown) fileSize: number; // Final file size on success (-1 otherwise) filePath: string | null; // Absolute path on success } Optional HTTP request options for startDownload. interface DownloadRequestOptions { method?: string; // HTTP method (default: "GET") headers?: Record; // HTTP headers body?: string; // Request body (for POST, PUT, etc.) } Options for the fetch2 utility method. interface Fetch2Options { readAs?: "text" | "base64" | "blob" | "bytes"; // Default: "text" (UTF-8) timeoutMs?: number; // Max wait time in ms (default: 120000) maxRetries?: number; // Retry attempts on failure (default: 2) method?: string; // HTTP method (default: "GET") headers?: Record; // HTTP headers body?: string; // Request body (for POST, PUT, etc.) } FileInfo interface FileInfo { name: string; // File or directory name path: string; // Full path isDirectory: boolean; isFile: boolean; size: number; // Size in bytes (0 for directories) lastModified: number; // Timestamp } DirectoryInfo interface DirectoryInfo { exists: true; path: string; name: string; isDirectory: true; isFile: false; lastModified: number; canRead: boolean; canWrite: boolean; fileCount: number; // Number of files directoryCount: number; // Number of subdirectories totalItems: number; // Total items } FilePathInfo interface FilePathInfo { exists: true; path: string; name: string; isDirectory: false; isFile: true; lastModified: number; canRead: boolean; canWrite: boolean; size: number; } PathNotFoundInfo interface PathNotFoundInfo { exists: false; error?: string; } FileHashes interface FileHashes { md5: string; sha1: string; sha256: string; size: number; } UploadTempFileResult Result returned when a file is successfully uploaded. interface UploadTempFileResult { success: true; message: string; // "File uploaded successfully" data: { filename: string; // Server-assigned filename (e.g., "1234567890_example.pdf") originalName: string; // Original filename provided size: number; // File size in bytes url: string; // Full URL to access the file expiresAt: string; // ISO 8601 expiration timestamp (15 minutes from upload) }; } -------------------------------------------------------------------------------- ## Reference / agent.info Path: /docs/automation/reference/agent/info Description: Get automation and device metadata Sections: - Methods - Types - Usage Example Methods: ### getAutomationInfo() Signature: getAutomationInfo(): AutomationInfo Returns metadata about the currently running automation. ### getDeviceInfo() Signature: getDeviceInfo(): DeviceInfo Returns information about the device hardware and configuration. ### getPhoneNumber() Signature: getPhoneNumber(): string | null Returns the phone number of the device's SIM card. Returns null if unavailable (no SIM inserted, permission not granted, or carrier doesn't provide the number). Requires phone/SMS permissions to be granted on the device. ### getVerifiedPhoneNumber() Signature: getVerifiedPhoneNumber(): Promise Returns the phone number stored on the device record on the server, only if it has been marked verified there. Resolves to null when the device has no verified phone number, the agent token is missing (i.e. the automation is not running through a job/agent), or the request fails. Differs from getPhoneNumber, which reads the SIM directly on the device. Backed by GET /api/v2/devices/verified-phone-number. Content: Access these methods through agent.info . Provides information about the current automation and the device it's running on. Methods Types Usage Example next getAutomationInfo getAutomationInfo(): AutomationInfo Returns metadata about the currently running automation. AutomationInfo Automation metadata const info = agent.info.getAutomationInfo(); console.log("Running:", info.name); console.log("Launch ID:", info.launchId); console.log("Server:", info.serverBaseUrl); getDeviceInfo getDeviceInfo(): DeviceInfo Returns information about the device hardware and configuration. DeviceInfo Device specifications const device = agent.info.getDeviceInfo(); console.log("Device:", device.brand, device.model); console.log("Screen:", device.width, "x", device.height); console.log("Android SDK:", device.sdkVersion); console.log("Is Emulator:", device.isEmulator); getPhoneNumber getPhoneNumber(): string | null Returns the phone number of the device's SIM card. Returns null if unavailable (no SIM inserted, permission not granted, or carrier doesn't provide the number). Requires phone/SMS permissions to be granted on the device. 2.138 (150) string | null Phone number string or null if unavailable const phoneNumber = agent.info.getPhoneNumber(); if (phoneNumber) { console.log("Device phone number:", phoneNumber); } else { console.log("Phone number not available"); } getVerifiedPhoneNumber getVerifiedPhoneNumber(): Promise Returns the phone number stored on the device record on the server, only if it has been marked verified there. Resolves to null when the device has no verified phone number, the agent token is missing (i.e. the automation is not running through a job/agent), or the request fails. Differs from getPhoneNumber, which reads the SIM directly on the device. Backed by GET /api/v2/devices/verified-phone-number. Promise Verified phone number from the device record, or null if not verified / unavailable const verified = await agent.info.getVerifiedPhoneNumber(); if (verified) { console.log("Verified phone number:", verified); } else { console.log("No verified phone number on file"); } name string The name of the automation description Description of what the automation does launchId Unique identifier for this execution agent object Agent details when running in agent mode (id, commitId, token, jobTaskId) serverBaseUrl Base URL of the server for API calls timeout number Optional timeout for the automation in minutes (since v2.123 (135)) Unique identifier for the device brand Device manufacturer (e.g., 'Samsung', 'Google') model Device model name sdkVersion Android SDK API level (e.g., 33 for Android 13) processor CPU/processor information numberOfCores Number of CPU cores ramMb Total RAM in megabytes country Device country/region setting isEmulator boolean Whether the device is an emulator width Screen width in pixels height Screen height in pixels appVersionName RemoteMobile app version name string (e.g., "2.141") appVersionCode RemoteMobile app version code integer (e.g., 195). Use to gate features added in a specific app version. agent.info · Reference AgentInfo Interface Get automation and device metadata AgentInfo Interface interface AgentInfo { getAutomationInfo(): AutomationInfo; getDeviceInfo(): DeviceInfo; getPhoneNumber(): string | null; // Since 2.138 getVerifiedPhoneNumber(): Promise; } Contains metadata about the running automation. interface AutomationInfo { name: string; // Automation name description: string; // Automation description launchId: string; // Unique ID for this launch agent?: { // Optional agent details id: string; commitId: string; token: string; jobTaskId: string; }; serverBaseUrl: string; // Server URL for API calls timeout?: number; // Optional timeout in minutes available since app v2.123 (135) } AutomationInfo Properties Contains hardware and configuration information about the device. interface DeviceInfo { id: string; // Unique device ID brand: string; // Device brand (e.g., "Samsung") model: string; // Device model (e.g., "SM-G991B") sdkVersion: number; // Android SDK version (e.g., 33) processor: string; // CPU info numberOfCores: number; // CPU core count ramMb: number; // RAM in megabytes country: string; // Device country isEmulator: boolean; // true if running on emulator width: number; // Screen width in pixels height: number; // Screen height in pixels appVersionName: string; // RemoteMobile app version name (e.g., "2.141") appVersionCode: number; // RemoteMobile app version code (e.g., 195) } DeviceInfo Properties // Adapt automation to device capabilities const device = agent.info.getDeviceInfo(); // Calculate center of screen const centerX = device.width / 2; const centerY = device.height / 2; // Check Android version for feature availability if (device.sdkVersion >= 33) { // Use Android 13+ features } // Log automation context const automation = agent.info.getAutomationInfo(); console.log(`Running "${automation.name}" on ${device.brand} ${device.model}`); -------------------------------------------------------------------------------- ## Reference / agent.control Path: /docs/automation/reference/agent/control Description: Control automation execution Methods: ### stopCurrentAutomation() Signature: stopCurrentAutomation(): void Immediately stops the current automation execution. Use this to gracefully terminate an automation when a condition is met or an error occurs. Content: When stopCurrentAutomation() is called, any code after the call will not execute. The automation terminates immediately. next stopCurrentAutomation stopCurrentAutomation(): void Immediately stops the current automation execution. Use this to gracefully terminate an automation when a condition is met or an error occurs. Stop on error try { await performCriticalAction(); } catch (error) { console.log("Critical error, stopping automation"); agent.control.stopCurrentAutomation(); } Stop when goal is achieved const screen = await agent.actions.screenContent(); const successMessage = screen.findTextOne("Order Confirmed"); if (successMessage) { console.log("Task completed successfully!"); agent.control.stopCurrentAutomation(); } Stop with cleanup function cleanup() { // Save state, close resources, etc. console.log("Cleaning up..."); } // On critical failure cleanup(); agent.control.stopCurrentAutomation(); agent.control · Reference agent.control Interface Control the automation execution flow. AgentControl Interface interface AgentControl { stopCurrentAutomation(): void; } warning -------------------------------------------------------------------------------- ## Reference / agent.display Path: /docs/automation/reference/agent/display Description: Display HTML overlays on screen Sections: - Processing... - Use Cases Methods: ### displayHTMLCode() Signature: displayHTMLCode(htmlCode: string, x1: number, y1: number, x2: number, y2: number, opacity: number): void Displays an HTML overlay on the screen within the specified rectangle. ### hideHTMLCode() Signature: hideHTMLCode(): void Hides any currently displayed HTML overlay. Content: ' + ' Processing... Please wait while the automation runs. Loading... Working... Use Cases HTML overlays are rendered on top of the screen content but don't intercept touch events. The overlay is purely visual and won't affect automation interactions with the underlying UI. next displayHTMLCode displayHTMLCode(htmlCode: string, x1: number, y1: number, x2: number, y2: number, opacity: number): void Displays an HTML overlay on the screen within the specified rectangle. htmlCode string HTML content to render x1 number Left boundary of the overlay y1 Top boundary of the overlay x2 Right boundary of the overlay y2 Bottom boundary of the overlay opacity Opacity (0.0 to 1.0) Show status message Show progress Full-screen overlay const device = agent.info.getDeviceInfo(); agent.display.displayHTMLCode( '
Loading...
', 0, 0, device.width, device.height, 1.0 ); hideHTMLCode hideHTMLCode(): void Hides any currently displayed HTML overlay. // Show overlay agent.display.displayHTMLCode("
Working...
", 100, 100, 500, 200, 0.8); // Do some work await performTask(); // Hide overlay when done agent.display.hideHTMLCode(); Status Display Show the current status of the automation to the user, especially for long-running tasks. Progress Indicators Display progress bars or percentage completion for multi-step processes. Error Messages Show error messages or warnings that require user attention. Debug Information Display debug info during development to understand automation behavior. agent.display · Reference agent.display Interface Render HTML content as an overlay on the device screen. AgentDisplay Interface interface AgentDisplay { displayHTMLCode(htmlCode: string, x1: number, y1: number, x2: number, y2: number, opacity: number): void; hideHTMLCode(): void; } -------------------------------------------------------------------------------- ## Reference / agent.email Path: /docs/automation/reference/agent/email Description: Read emails via IMAP protocol Methods: ### readIMAPEmails() Signature: readIMAPEmails(email: string, password: string, host?: string, port?: number, skip?: number, limit?: number, proxyHost?: string, proxyPort?: number, proxyUser?: string, proxyPassword?: string): Promise Reads emails from an IMAP server. Useful for automations that need to verify email content (e.g., verification codes, confirmation emails). Content: App Password next readIMAPEmails readIMAPEmails(email: string, password: string, host?: string, port?: number, skip?: number, limit?: number, proxyHost?: string, proxyPort?: number, proxyUser?: string, proxyPassword?: string): Promise Reads emails from an IMAP server. Useful for automations that need to verify email content (e.g., verification codes, confirmation emails). email string Email address to read from password Email account password or app-specific password host IMAP server hostname (auto-detected for common providers) port number IMAP server port (default: 993) skip Number of emails to skip (for pagination) limit Maximum number of emails to return proxyHost Proxy server hostname. Available since app version 2.123 (135) proxyPort Proxy server port. Available since app version 2.123 (135) proxyUser Proxy username for authentication. Available since app version 2.123 (135) proxyPassword Proxy password for authentication. Available since app version 2.123 (135) Promise Array of email messages Read latest emails const emails = await agent.email.readIMAPEmails( "user@gmail.com", "app-password-here", undefined, // auto-detect host undefined, // default port 0, // no skip 10 // limit to 10 emails ); for (const email of emails) { console.log("From:", email.from); console.log("Subject:", email.subject); console.log("---"); } Find verification code const emails = await agent.email.readIMAPEmails( "user@gmail.com", "app-password", undefined, undefined, 0, 5 ); // Find email with verification code const verificationEmail = emails.find(e => e.subject.includes("Verification") || e.subject.includes("Code") ); if (verificationEmail) { // Extract code from email body const codeMatch = verificationEmail.body.match(/\b\d{6}\b/); if (codeMatch) { console.log("Verification code:", codeMatch[0]); } } Custom IMAP server const emails = await agent.email.readIMAPEmails( "user@company.com", "password", "imap.company.com", // custom host 993, // custom port 0, 20 ); Using a proxy server (since v2.123 (135)) const emails = await agent.email.readIMAPEmails( "user@gmail.com", "app-password", undefined, // auto-detect host undefined, // default port 0, // no skip 10, // limit "proxy.example.com", // proxy host 1080, // proxy port "proxyuser", // proxy username (optional) "proxypassword" // proxy password (optional) ); Unique identifier for the email subject Email subject line from Sender's email address fromName Sender's display name string[] Array of recipient email addresses Array of CC recipient addresses Array of BCC recipient addresses date Email timestamp in Unix milliseconds body Email body content (HTML or plain text) isHtml boolean Whether the body content is HTML isRead Whether the email has been read hasAttachments Whether the email has attachments attachmentNames Names of attached files agent.email · Reference agent.email Interface Read emails from IMAP-enabled email accounts. AgentEmail Interface interface AgentEmail { readIMAPEmails( email: string, password: string, host?: string, port?: number, skip?: number, limit?: number, proxyHost?: string, proxyPort?: number, proxyUser?: string, proxyPassword?: string ): Promise; } Email Represents an email message. interface Email { id: string; // Unique email ID subject: string; // Email subject from: string; // Sender email address fromName: string; // Sender display name to: string[]; // Recipients cc: string[]; // CC recipients bcc: string[]; // BCC recipients date: number; // Timestamp (Unix ms) body: string; // Email body content isHtml: boolean; // true if body is HTML isRead: boolean; // Read status hasAttachments: boolean; // Has attachments attachmentNames: string[]; // Attachment file names } Email Properties warning Security Note https://support.google.com/accounts/answer/185833 _blank noopener noreferrer -------------------------------------------------------------------------------- ## Reference / agent.sms Path: /docs/automation/reference/agent/sms Description: Read and send SMS messages from the device Methods: ### readSMS() Signature: readSMS(options?: ReadSMSOptions): Promise Reads SMS messages from the device. Useful for automations that need to read verification codes or other SMS content. All options are optional - calling with no arguments returns the 20 most recent messages. ### sendSMS() Signature: sendSMS(phoneNumber: string, message: string): Promise Sends an SMS message from the device. Long messages are automatically split into multiple parts. Requires SEND_SMS permission. Content: ; sendSMS(phoneNumber: string, message: string): Promise SMS operations require the READ_SMS SEND_SMS permissions to be granted on the device. The app requests these automatically during setup. If permissions are denied, the corresponding methods return an error. next readSMS readSMS(options?: ReadSMSOptions): Promise Reads SMS messages from the device. Useful for automations that need to read verification codes or other SMS content. All options are optional - calling with no arguments returns the 20 most recent messages. 2.138 (150) options ReadSMSOptions Optional filtering and pagination options Promise Array of SMS messages Read latest SMS messages const messages = await agent.sms.readSMS(); for (const msg of messages) { console.log("From:", msg.address); console.log("Body:", msg.body); console.log("Date:", new Date(msg.date).toLocaleString()); console.log("---"); } Find OTP/verification code // Get recent inbox messages from the last 5 minutes const fiveMinAgo = Date.now() - 5 * 60 * 1000; const messages = await agent.sms.readSMS({ type: 1, // inbox only minDate: fiveMinAgo, limit: 10, }); // Find message with OTP code const otpMsg = messages.find(m => m.body.includes("code") || m.body.includes("OTP") ); if (otpMsg) { const codeMatch = otpMsg.body.match(/\b\d{4,6}\b/); if (codeMatch) { console.log("OTP code:", codeMatch[0]); } } Filter by phone number const messages = await agent.sms.readSMS({ phoneNumber: "+1234567890", limit: 5, }); console.log("Messages from +1234567890:", messages.length); Paginate through messages // First page const page1 = await agent.sms.readSMS({ limit: 10, skip: 0 }); // Second page const page2 = await agent.sms.readSMS({ limit: 10, skip: 10 }); Read sent messages (oldest first) const sentMessages = await agent.sms.readSMS({ type: 2, // sent messages sortOrder: "asc", // oldest first limit: 20, }); sendSMS sendSMS(phoneNumber: string, message: string): Promise Sends an SMS message from the device. Long messages are automatically split into multiple parts. Requires SEND_SMS permission. phoneNumber string Recipient phone number message Text message to send Promise Resolves to true if the message was sent successfully Send a simple SMS const success = await agent.sms.sendSMS("+1234567890", "Hello from automation!"); console.log("SMS sent:", success); Send SMS with error handling try { await agent.sms.sendSMS("+1234567890", "Your verification code is 1234"); console.log("SMS sent successfully"); } catch (error) { console.error("Failed to send SMS:", error.message); } Send SMS using job variables const { phoneNumber, messageTemplate } = agent.arguments.jobVariables; await agent.sms.sendSMS(phoneNumber, messageTemplate); Filter messages by phone number. Uses partial matching so both full and partial numbers work. limit number Maximum number of messages to return. Defaults to 20. skip Number of messages to skip for pagination. Defaults to 0. sortOrder "asc" | "desc" Sort order by date. "desc" returns newest first (default), "asc" returns oldest first. type Filter by SMS type: 1 = inbox (received), 2 = sent, 3 = draft. Omit to include all types. minDate Only return messages after this timestamp (Unix milliseconds). maxDate Only return messages before this timestamp (Unix milliseconds). Unique identifier for the SMS message address Phone number - sender for inbox messages, recipient for sent messages body The text content of the message date Message timestamp in Unix milliseconds Message type: 1 = inbox (received), 2 = sent, 3 = draft read boolean Whether the message has been read seen Whether the message has been seen by the user agent.sms · Reference agent.sms Interface Read and send SMS messages on the device. Requires SMS permissions. AgentSMS Interface interface AgentSMS { readSMS(options?: ReadSMSOptions): Promise; sendSMS(phoneNumber: string, message: string): Promise; } interface ReadSMSOptions { phoneNumber?: string; // Filter by phone number (partial match) limit?: number; // Max messages (default: 20) skip?: number; // Pagination offset (default: 0) sortOrder?: "asc" | "desc"; // Sort by date (default: "desc") type?: number; // 1=inbox, 2=sent, 3=draft minDate?: number; // After this timestamp (ms) maxDate?: number; // Before this timestamp (ms) } Options for filtering and paginating SMS messages. interface ReadSMSOptions { phoneNumber?: string; // Filter by phone number (partial match) limit?: number; // Max messages to return (default: 20) skip?: number; // Messages to skip (default: 0) sortOrder?: "asc" | "desc"; // Sort by date (default: "desc") type?: number; // SMS type: 1=inbox, 2=sent, 3=draft minDate?: number; // Only after this timestamp (Unix ms) maxDate?: number; // Only before this timestamp (Unix ms) } ReadSMSOptions Properties SmsMessage Represents an SMS message. interface SmsMessage { id: string; // Unique message ID address: string; // Phone number body: string; // Message text content date: number; // Timestamp (Unix ms) type: number; // 1=inbox, 2=sent, 3=draft read: boolean; // Has been read seen: boolean; // Has been seen } SmsMessage Properties warning -------------------------------------------------------------------------------- ## Reference / agent.notifications Path: /docs/automation/reference/agent/notifications Description: Handle system notifications Methods: ### setNotificationCallback() Signature: setNotificationCallback(callback: NotificationCallback | null): void Registers a callback to receive system notifications. Pass null to unregister. ### onProcessed() Signature: onProcessed(notificationId: string, shouldOpenNotification: boolean): void Call this after processing a notification to indicate it has been handled. Optionally open the notification. Content: agent.notifications.setNotificationCallback(null) next setNotificationCallback setNotificationCallback(callback: NotificationCallback | null): void Registers a callback to receive system notifications. Pass null to unregister. callback (id: string, packageName: string, channelId: string, extras: any) => void Callback function or null to unregister agent.notifications.setNotificationCallback((id, packageName, channelId, extras) => { console.log("Notification from:", packageName); console.log("Title:", extras.title); console.log("Text:", extras.text); // Process the notification agent.notifications.onProcessed(id, false); }); // Later, to stop receiving notifications: agent.notifications.setNotificationCallback(null); onProcessed onProcessed(notificationId: string, shouldOpenNotification: boolean): void Call this after processing a notification to indicate it has been handled. Optionally open the notification. notificationId string The notification ID from the callback shouldOpenNotification boolean Whether to open/click the notification Basic notification handling // Set up notification listener agent.notifications.setNotificationCallback((id, packageName, channelId, extras) => { console.log("Notification from:", packageName); console.log("Title:", extras.title); console.log("Text:", extras.text); // Mark as processed without opening agent.notifications.onProcessed(id, false); }); Open specific notifications agent.notifications.setNotificationCallback((id, packageName, channelId, extras) => { // Only open notifications from specific app if (packageName === "com.whatsapp") { console.log("WhatsApp message:", extras.text); agent.notifications.onProcessed(id, true); // Open the notification } else { agent.notifications.onProcessed(id, false); // Just dismiss } }); Extract verification codes agent.notifications.setNotificationCallback((id, packageName, channelId, extras) => { const text = extras.text || ""; // Look for verification codes in notifications const codeMatch = text.match(/\b\d{4,6}\b/); if (codeMatch) { console.log("Found verification code:", codeMatch[0]); // Store or use the code... } agent.notifications.onProcessed(id, false); }); Unique notification identifier packageName App that sent the notification channelId Notification channel ID extras Notification extras (title, text, etc.) // Track notifications for a specific task const receivedCodes = []; // Set up listener agent.notifications.setNotificationCallback((id, packageName, channelId, extras) => { console.log("=== Notification Received ==="); console.log("Package:", packageName); console.log("Channel:", channelId); console.log("Title:", extras.title); console.log("Text:", extras.text); // Look for OTP codes const text = (extras.title || "") + " " + (extras.text || ""); const otpMatch = text.match(/\b\d{6}\b/); if (otpMatch) { receivedCodes.push({ code: otpMatch[0], from: packageName, time: Date.now() }); } // Mark as processed agent.notifications.onProcessed(id, false); }); // Later, when you need the code: if (receivedCodes.length > 0) { const latestCode = receivedCodes[receivedCodes.length - 1]; console.log("Using code:", latestCode.code); } // Clean up when done agent.notifications.setNotificationCallback(null); agent.notifications · Reference agent.notifications Interface Register callbacks to receive and process system notifications. AgentNotifications Interface interface AgentNotifications { setNotificationCallback(callback: NotificationCallback | null): void; onProcessed(notificationId: string, shouldOpenNotification: boolean): void; } Complete Example NotificationCallback type NotificationCallback = ( id: string, packageName: string, channelId: string, extras: { title?: string; text?: string; [key: string]: any; } ) => void; Callback Parameters -------------------------------------------------------------------------------- ## Reference / AndroidNode Path: /docs/automation/reference/android-node Description: Accessibility tree nodes for UI element interaction Sections: - Methods - Action Methods - Common Patterns Methods: ### allNodes() Signature: allNodes(): AndroidNode[] Returns all nodes in the subtree as a flat array, including this node and all descendants. ### findById() Signature: findById(id: string): AndroidNode[] Finds all nodes with the matching viewId. ### findByIdOne() Signature: findByIdOne(id: string): AndroidNode | null Finds the first node with the matching viewId. ### findText() Signature: findText(textOrRegex: string | RegExp): AndroidNode[] Finds all nodes containing the text in description, text, or hintText (case-insensitive). Supports strings and regular expressions. ### findTextOne() Signature: findTextOne(textOrRegex: string | RegExp): AndroidNode | null Finds the first node containing the text (case-insensitive). ### find() Signature: find(predicate: (node: AndroidNode) => boolean): AndroidNode | null Finds the first node matching a custom predicate function. ### filter() Signature: filter(predicate: (node: AndroidNode) => boolean): AndroidNode[] Filters all nodes matching a custom predicate function. ### findAdvanced() Signature: findAdvanced(filterBuilder: (f: AndroidNodeFilter) => AndroidNodeFilter): AndroidNode | null Finds the first node using the AndroidNodeFilter builder pattern. ### filterAdvanced() Signature: filterAdvanced(filterBuilder: (f: AndroidNodeFilter) => AndroidNodeFilter): AndroidNode[] Filters all nodes using the AndroidNodeFilter builder pattern. ### matches() Signature: matches(filterBuilder: (f: AndroidNodeFilter) => AndroidNodeFilter): boolean Checks if this node matches the given filter. ### toJSON() Signature: toJSON(): object Converts the node to a plain JSON object. Useful for debugging or serialization. ### performAction() Signature: performAction(actionInt: number, data?: object, fieldsToIgnore?: string[]): Promise<{ actionPerformed: boolean }> Performs an accessibility action on this node. This is the preferred way to interact with UI elements as it uses the accessibility system. ### adbPerformAction() Signature: adbPerformAction(actionInt: number, data?: object, fieldsToIgnore?: string[]): Promise<{ actionPerformed: boolean }> Performs an accessibility action on this node via ADB. The ADB equivalent of performAction(). Uses UiAutomation through the ADB touch server instead of the accessibility service. ### randomClick() Signature: randomClick(): void Performs a tap at a random position within this node's bounds. Uses boundsInScreen to determine the click area. Useful for more natural-looking automation. ### randomSwipe() Signature: randomSwipe(direction: "up" | "down" | "left" | "right"): void Performs a swipe gesture within this node's bounds. The swipe starts and ends at random positions within the node for more natural-looking automation. ### randomClickAdb() Signature: randomClickAdb(): void Performs a tap at a random position within this node's bounds using ADB shell input instead of the accessibility service. Fire-and-forget like randomClick(). ### randomSwipeAdb() Signature: randomSwipeAdb(direction: "up" | "down" | "left" | "right"): void Performs a swipe gesture within this node's bounds using ADB shell input instead of the accessibility service. Fire-and-forget like randomSwipe(). Content: AndroidNode represents a node in the Android accessibility tree. Each node corresponds to a UI element on screen and contains properties like text, bounds, and state. Use methods like screenContent() to get the root node and traverse the tree. Methods Action Methods These methods allow you to perform actions directly on nodes. Available since app version 2.119. Note: randomClickAdb() randomSwipeAdb() require app version 2.141 (153)+. They use ADB shell commands instead of the accessibility service, which can be more reliable in some scenarios. Common Patterns next falseId string UUID unique per request (changes each time screen content is fetched) viewId Android view ID (e.g., 'com.example:id/button') className string | null Android class name (e.g., 'android.widget.Button') packageName Package name of the app owning this node boundsInScreen {left, top, right, bottom} Screen coordinates of the node's bounding box index number Index within parent's children text Text content of the node description Content description (accessibility label) hintText Hint text for input fields maxTextLength Maximum text length for input fields inputType Input type for text fields isShowingHintText boolean Whether hint text is currently shown tooltipText Tooltip text Android 9+ paneTitle Pane title for accessibility containerTitle Container title Android 14+ isEnabled Whether the node is enabled clickable Whether the node is clickable isLongClickable Whether the node responds to long click isContextClickable Whether the node responds to context click isFocusable isFocused boolean | null isScrollable Whether the node is scrollable isSelected Whether the node is selected isChecked Checked state (for checkboxes, switches) isEditable Whether text can be edited isDismissable Whether the node can be dismissed isPassword Whether this is a password field isMultiLine Whether this is a multi-line text field isVisibleToUser Whether the node is visible on screen isImportantForAccessibility Whether important for accessibility isHeading Whether this node is a heading isTextSelectable Whether text can be selected Android 13+ isTextEntryKey Whether this is a keyboard key Android 10+ collectionInfo object Info when node is a collection (ListView, RecyclerView). Contains rowCount, columnCount, itemCount, hierarchical, selectionMode collectionItemInfo Info when node is a collection item. Contains rowIndex, columnIndex, rowSpan, columnSpan, isSelected children AndroidNode[] Child nodes parent AndroidNode | null Parent node (null for root) actions number[] Array of supported action IDs (use with agent.constants.ACTION_*) actionLabels {id: number, label?: string}[] Action labels with optional custom text allNodes allNodes(): AndroidNode[] Returns all nodes in the subtree as a flat array, including this node and all descendants. Flat array of all nodes const screen = await agent.actions.screenContent(); const allNodes = screen.allNodes(); console.log("Total nodes:", allNodes.length); // Find all visible text const textNodes = allNodes.filter(n => n.text && n.isVisibleToUser); findById findById(id: string): AndroidNode[] Finds all nodes with the matching viewId. View ID to search for (e.g., 'com.example:id/button') Array of matching nodes const buttons = screen.findById("com.example:id/submit_button"); findByIdOne findByIdOne(id: string): AndroidNode | null Finds the first node with the matching viewId. View ID to search for First matching node or null const button = screen.findByIdOne("com.example:id/submit"); if (button) { await button.performAction(agent.constants.ACTION_CLICK); } findText findText(textOrRegex: string | RegExp): AndroidNode[] Finds all nodes containing the text in description, text, or hintText (case-insensitive). Supports strings and regular expressions. textOrRegex string | RegExp Text to search for or regex pattern Find by string const nodes = screen.findText("Submit"); Find by regex const priceNodes = screen.findText(/\$\d+\.\d{2}/); findTextOne findTextOne(textOrRegex: string | RegExp): AndroidNode | null Finds the first node containing the text (case-insensitive). const submit = screen.findTextOne("Submit"); if (submit) { const { left, top, right, bottom } = submit.boundsInScreen; await agent.actions.tap((left + right) / 2, (top + bottom) / 2); } find find(predicate: (node: AndroidNode) => boolean): AndroidNode | null Finds the first node matching a custom predicate function. predicate (node: AndroidNode) => boolean Function to test each node // Find first clickable button with specific text const btn = screen.find(node => node.className === "android.widget.Button" && node.clickable && node.text === "Submit" ); // Find first visible input field const input = screen.find(node => node.isEditable && node.isVisibleToUser ); filter filter(predicate: (node: AndroidNode) => boolean): AndroidNode[] Filters all nodes matching a custom predicate function. // Find all buttons const buttons = screen.filter(node => node.className === "android.widget.Button" ); // Find all clickable, visible nodes const clickables = screen.filter(node => node.clickable && node.isVisibleToUser ); // Find nodes with text containing dollar amounts const prices = screen.filter(node => node.text?.includes("$") ); findAdvanced findAdvanced(filterBuilder: (f: AndroidNodeFilter) => AndroidNodeFilter): AndroidNode | null Finds the first node using the AndroidNodeFilter builder pattern. filterBuilder (f: AndroidNodeFilter) => AndroidNodeFilter Function to build the filter // Find the submit button const submit = screen.findAdvanced(f => f.isButton().hasText("Submit")); // Find first editable text field const input = screen.findAdvanced(f => f.isEditText().isEditable()); // Find by view ID const header = screen.findAdvanced(f => f.hasId("com.example:id/header")); filterAdvanced filterAdvanced(filterBuilder: (f: AndroidNodeFilter) => AndroidNodeFilter): AndroidNode[] Filters all nodes using the AndroidNodeFilter builder pattern. // Find all clickable buttons const buttons = screen.filterAdvanced(f => f.isButton().isClickable()); // Find text nodes with specific text const labels = screen.filterAdvanced(f => f.isText().hasText("Welcome")); // Find nodes in a specific list const listItems = screen.filterAdvanced(f => f.anyParent(p => p.hasId("com.example:id/list")) ); matches matches(filterBuilder: (f: AndroidNodeFilter) => AndroidNodeFilter): boolean Checks if this node matches the given filter. true if node matches the filter // Check if a node is a clickable button if (node.matches(f => f.isButton().isClickable())) { await node.performAction(agent.constants.ACTION_CLICK); } // Filter children manually const clickableChildren = node.children.filter(child => child.matches(f => f.isClickable()) ); toJSON toJSON(): object Converts the node to a plain JSON object. Useful for debugging or serialization. Plain object representation console.log(JSON.stringify(node.toJSON(), null, 2)); performAction performAction(actionInt: number, data?: object, fieldsToIgnore?: string[]): Promise<{ actionPerformed: boolean }> Performs an accessibility action on this node. This is the preferred way to interact with UI elements as it uses the accessibility system. 2.119 actionInt Action constant (use agent.constants.ACTION_*) data Additional action data (used for ACTION_SET_TEXT etc.) fieldsToIgnore string[] Fields to ignore when matching node Promise<{ actionPerformed: boolean }> Whether the action was performed successfully Click a button const button = screen.findTextOne("Submit"); if (button) { await button.performAction(agent.constants.ACTION_CLICK); } Set text on an input field const input = screen.findAdvanced(f => f.isEditText()); if (input) { await input.performAction(agent.constants.ACTION_SET_TEXT, { [agent.constants.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE]: "Hello World" }); } Scroll a list const scrollView = screen.findAdvanced(f => f.isScrollable()); if (scrollView) { await scrollView.performAction(agent.constants.ACTION_SCROLL_FORWARD); } adbPerformAction adbPerformAction(actionInt: number, data?: object, fieldsToIgnore?: string[]): Promise<{ actionPerformed: boolean }> Performs an accessibility action on this node via ADB. The ADB equivalent of performAction(). Uses UiAutomation through the ADB touch server instead of the accessibility service. Whether the action was successfully performed Click a button via ADB const button = screen.findTextOne("Submit"); if (button) { await button.adbPerformAction(agent.constants.ACTION_CLICK); } Set text via ADB const input = screen.findAdvanced(f => f.isEditText()); if (input) { await input.adbPerformAction(agent.constants.ACTION_SET_TEXT, { [agent.constants.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE]: "Hello World" }); } randomClick randomClick(): void Performs a tap at a random position within this node's bounds. Uses boundsInScreen to determine the click area. Useful for more natural-looking automation. const button = screen.findTextOne("Submit"); if (button) { button.randomClick(); } Click multiple items const items = screen.filterAdvanced(f => f.isClickable().hasText(/Item \d+/)); for (const item of items) { item.randomClick(); await sleep(500); } randomSwipe randomSwipe(direction: "up" | "down" | "left" | "right"): void Performs a swipe gesture within this node's bounds. The swipe starts and ends at random positions within the node for more natural-looking automation. direction "up" | "down" | "left" | "right" Direction to swipe Scroll a list down const listView = screen.findAdvanced(f => f.isScrollable()); if (listView) { listView.randomSwipe("up"); // Swipe up to scroll down } Swipe through a carousel const carousel = screen.findAdvanced(f => f.hasId("carousel")); if (carousel) { carousel.randomSwipe("left"); // Swipe left to see next item } randomClickAdb randomClickAdb(): void Performs a tap at a random position within this node's bounds using ADB shell input instead of the accessibility service. Fire-and-forget like randomClick(). 2.141 (153) const button = screen.findTextOne("Submit"); if (button) { button.randomClickAdb(); } randomSwipeAdb randomSwipeAdb(direction: "up" | "down" | "left" | "right"): void Performs a swipe gesture within this node's bounds using ADB shell input instead of the accessibility service. Fire-and-forget like randomSwipe(). Scroll a list down via ADB const listView = screen.findAdvanced(f => f.isScrollable()); if (listView) { listView.randomSwipeAdb("up"); // Swipe up to scroll down } AndroidNode · Reference Interface Accessibility tree nodes for UI element interaction. Getting Screen Content // Get the accessibility tree const screen = await agent.actions.screenContent(); // Find elements const button = screen.findTextOne("Submit"); const allInputs = screen.filterAdvanced(f => f.isEditText()); // Interact with elements if (button) { const { left, top, right, bottom } = button.boundsInScreen; await agent.actions.tap((left + right) / 2, (top + bottom) / 2); } AndroidNodeFilter Learn about the builder pattern for complex node queries /docs/automation/reference/android-node/filter Identification Layout Text Content State Collection Info Hierarchy Actions Click a button (random position) Click using accessibility action const button = screen.findAdvanced(f => f.isButton().hasText("OK")); if (button) { await button.performAction(agent.constants.ACTION_CLICK); } Type in an input field const input = screen.findAdvanced(f => f.isEditText().isEditable()); if (input) { await input.performAction(agent.constants.ACTION_CLICK); await agent.actions.writeText("Hello World"); } const scrollView = screen.findAdvanced(f => f.isScrollable()); if (scrollView) { // Using accessibility action await scrollView.performAction(agent.constants.ACTION_SCROLL_FORWARD); // Or using random swipe for more natural scrolling scrollView.randomSwipe("up"); } -------------------------------------------------------------------------------- ## Reference / AndroidNode.filter Path: /docs/automation/reference/android-node/filter Description: Builder pattern for complex node queries Sections: - Class Matchers - State Matchers - Content Matchers - Identity Matchers - Logical Operators - Hierarchy Matchers - Complex Examples Methods: ### isButton() Signature: isButton(): AndroidNodeFilter Matches nodes with className 'android.widget.Button'. ### isText() Signature: isText(): AndroidNodeFilter Matches nodes with className 'android.widget.TextView'. ### isImage() Signature: isImage(): AndroidNodeFilter Matches nodes with className 'android.widget.ImageView'. ### isEditText() Signature: isEditText(): AndroidNodeFilter Matches nodes with className 'android.widget.EditText' (text input fields). ### isCheckBox() Signature: isCheckBox(): AndroidNodeFilter Matches nodes with className 'android.widget.CheckBox'. ### isRadioButton() Signature: isRadioButton(): AndroidNodeFilter Matches nodes with className 'android.widget.RadioButton'. ### isSwitch() Signature: isSwitch(): AndroidNodeFilter Matches nodes with className 'android.widget.Switch'. ### isSeekBar() Signature: isSeekBar(): AndroidNodeFilter Matches nodes with className 'android.widget.SeekBar'. ### isViewGroup() Signature: isViewGroup(): AndroidNodeFilter Matches nodes with className 'android.view.ViewGroup' (container elements). ### is() Signature: is(className: string): AndroidNodeFilter Matches nodes with a custom className. ### isClickable() Signature: isClickable(): AndroidNodeFilter Matches nodes that are clickable. ### isScrollable() Signature: isScrollable(): AndroidNodeFilter Matches nodes that are scrollable. ### isSelected() Signature: isSelected(): AndroidNodeFilter Matches nodes that are currently selected. ### isEditable() Signature: isEditable(): AndroidNodeFilter Matches nodes that allow text editing. ### hasText() Signature: hasText(text: string): AndroidNodeFilter Matches nodes with exact text content. ### text() Signature: text(condition: (text: string | undefined) => boolean): AndroidNodeFilter Matches nodes based on a custom text condition. ### hasDescription() Signature: hasDescription(description: string): AndroidNodeFilter Matches nodes with exact content description (accessibility label). ### descriptionContains() Signature: descriptionContains(part: string): AndroidNodeFilter Matches nodes whose description contains the given text. ### description() Signature: description(condition: (desc: string | undefined) => boolean): AndroidNodeFilter Matches nodes based on a custom description condition. ### hasChildWithText() Signature: hasChildWithText(text: string): AndroidNodeFilter Matches nodes that have a direct child with the specified text. ### hasId() Signature: hasId(id: string): AndroidNodeFilter Matches nodes with the specified viewId. ### hasPackageName() Signature: hasPackageName(packageName: string): AndroidNodeFilter Matches nodes belonging to the specified package. ### hasChildWithId() Signature: hasChildWithId(id: string): AndroidNodeFilter Matches nodes that have a direct child with the specified viewId. ### and() Signature: and(filterBuilder: (f: AndroidNodeFilter) => void): AndroidNodeFilter Combines with another filter using AND logic. The node must match both filters. ### or() Signature: or(filterBuilder: (f: AndroidNodeFilter) => void): AndroidNodeFilter Combines with another filter using OR logic. The node must match either filter. ### parent() Signature: parent(filterBuilder: (f: AndroidNodeFilter) => void): AndroidNodeFilter Matches nodes whose immediate parent matches the given filter. ### anyParent() Signature: anyParent(filterBuilder: (f: AndroidNodeFilter) => void): AndroidNodeFilter Matches nodes that have any ancestor matching the given filter. Content: AndroidNodeFilter provides a fluent builder interface for constructing complex queries to find nodes in the accessibility tree. All methods are chainable and return the filter instance. Class Matchers Match nodes based on their Android class type. State Matchers Match nodes based on their current state. Content Matchers Match nodes based on their text content. Identity Matchers Match nodes based on their identity attributes. Logical Operators Combine multiple conditions using logical operators. Hierarchy Matchers Match nodes based on their position in the node hierarchy. Complex Examples next isButton isButton(): AndroidNodeFilter Matches nodes with className 'android.widget.Button'. const buttons = screen.filterAdvanced(f => f.isButton()); isText isText(): AndroidNodeFilter Matches nodes with className 'android.widget.TextView'. const labels = screen.filterAdvanced(f => f.isText()); isImage isImage(): AndroidNodeFilter Matches nodes with className 'android.widget.ImageView'. const images = screen.filterAdvanced(f => f.isImage()); isEditText isEditText(): AndroidNodeFilter Matches nodes with className 'android.widget.EditText' (text input fields). const inputs = screen.filterAdvanced(f => f.isEditText()); isCheckBox isCheckBox(): AndroidNodeFilter Matches nodes with className 'android.widget.CheckBox'. const checkboxes = screen.filterAdvanced(f => f.isCheckBox()); isRadioButton isRadioButton(): AndroidNodeFilter Matches nodes with className 'android.widget.RadioButton'. const radios = screen.filterAdvanced(f => f.isRadioButton()); isSwitch isSwitch(): AndroidNodeFilter Matches nodes with className 'android.widget.Switch'. const switches = screen.filterAdvanced(f => f.isSwitch()); isSeekBar isSeekBar(): AndroidNodeFilter Matches nodes with className 'android.widget.SeekBar'. const sliders = screen.filterAdvanced(f => f.isSeekBar()); isViewGroup isViewGroup(): AndroidNodeFilter Matches nodes with className 'android.view.ViewGroup' (container elements). const containers = screen.filterAdvanced(f => f.isViewGroup()); is(className: string): AndroidNodeFilter Matches nodes with a custom className. className string Full Android class name to match const recyclers = screen.filterAdvanced(f => f.is("androidx.recyclerview.widget.RecyclerView")); isClickable isClickable(): AndroidNodeFilter Matches nodes that are clickable. const clickables = screen.filterAdvanced(f => f.isClickable()); isScrollable isScrollable(): AndroidNodeFilter Matches nodes that are scrollable. const scrollView = screen.findAdvanced(f => f.isScrollable()); if (scrollView) { await scrollView.performAction(agent.constants.ACTION_SCROLL_DOWN); } isSelected isSelected(): AndroidNodeFilter Matches nodes that are currently selected. const selectedItem = screen.findAdvanced(f => f.isSelected()); isEditable isEditable(): AndroidNodeFilter Matches nodes that allow text editing. const editableInputs = screen.filterAdvanced(f => f.isEditText().isEditable()); hasText hasText(text: string): AndroidNodeFilter Matches nodes with exact text content. text Exact text to match const submitBtn = screen.findAdvanced(f => f.isButton().hasText("Submit")); text(condition: (text: string | undefined) => boolean): AndroidNodeFilter Matches nodes based on a custom text condition. condition (text: string | undefined) => boolean Function to test the text Contains const priceNodes = screen.filterAdvanced(f => f.text(t => t?.includes("$") ?? false)); Starts with const items = screen.filterAdvanced(f => f.text(t => t?.startsWith("Item") ?? false)); hasDescription hasDescription(description: string): AndroidNodeFilter Matches nodes with exact content description (accessibility label). description Exact description to match const closeBtn = screen.findAdvanced(f => f.hasDescription("Close")); descriptionContains descriptionContains(part: string): AndroidNodeFilter Matches nodes whose description contains the given text. part Text to search for in description const icons = screen.filterAdvanced(f => f.isImage().descriptionContains("icon")); description(condition: (desc: string | undefined) => boolean): AndroidNodeFilter Matches nodes based on a custom description condition. (desc: string | undefined) => boolean Function to test the description const nodes = screen.filterAdvanced(f => f.description(d => d?.toLowerCase().includes("button") ?? false) ); hasChildWithText hasChildWithText(text: string): AndroidNodeFilter Matches nodes that have a direct child with the specified text. Text to find in children // Find containers that have a "Settings" label inside const settingsSection = screen.findAdvanced(f => f.isViewGroup().hasChildWithText("Settings") ); hasId hasId(id: string): AndroidNodeFilter Matches nodes with the specified viewId. View ID to match (e.g., 'com.example:id/button') const header = screen.findAdvanced(f => f.hasId("com.example:id/header_title")); hasPackageName hasPackageName(packageName: string): AndroidNodeFilter Matches nodes belonging to the specified package. packageName Package name to match // Only match nodes from a specific app const appNodes = screen.filterAdvanced(f => f.hasPackageName("com.example.myapp") ); hasChildWithId hasChildWithId(id: string): AndroidNodeFilter Matches nodes that have a direct child with the specified viewId. View ID to find in children // Find containers with specific child elements const container = screen.findAdvanced(f => f.hasChildWithId("com.example:id/item_icon") ); and(filterBuilder: (f: AndroidNodeFilter) => void): AndroidNodeFilter Combines with another filter using AND logic. The node must match both filters. filterBuilder (f: AndroidNodeFilter) => void Function to build the additional filter // Find clickable buttons with specific text const btn = screen.findAdvanced(f => f.isButton().and(a => a.isClickable().hasText("Submit")) ); or(filterBuilder: (f: AndroidNodeFilter) => void): AndroidNodeFilter Combines with another filter using OR logic. The node must match either filter. Function to build the alternative filter Match buttons OR clickable images const clickables = screen.filterAdvanced(f => f.isButton().or(o => o.isImage().isClickable()) ); Match multiple text options const node = screen.findAdvanced(f => f.hasText("OK").or(o => o.hasText("Confirm")).or(o => o.hasText("Yes")) ); parent parent(filterBuilder: (f: AndroidNodeFilter) => void): AndroidNodeFilter Matches nodes whose immediate parent matches the given filter. Function to build the parent filter // Find text views inside a specific container const labels = screen.filterAdvanced(f => f.isText().parent(p => p.hasId("com.example:id/header")) ); anyParent anyParent(filterBuilder: (f: AndroidNodeFilter) => void): AndroidNodeFilter Matches nodes that have any ancestor matching the given filter. Function to build the ancestor filter Find items inside a specific list const listItems = screen.filterAdvanced(f => f.isText().anyParent(p => p.hasId("com.example:id/user_list")) ); Find clickables inside a dialog const dialogButtons = screen.filterAdvanced(f => f.isClickable().anyParent(p => p.is("android.app.Dialog")) ); AndroidNode.filter · Reference Interface Builder pattern for complex node queries. Basic Usage // Find a submit button const button = screen.findAdvanced(f => f.isButton().hasText("Submit")); // Find all editable text fields const inputs = screen.filterAdvanced(f => f.isEditText().isEditable()); // Complex query with multiple conditions const node = screen.findAdvanced(f => f.isClickable() .hasPackageName("com.example.app") .anyParent(p => p.hasId("com.example:id/main_container")) ); Find login button in a specific form const loginBtn = screen.findAdvanced(f => f.isButton() .hasText("Login") .isClickable() .anyParent(p => p.hasId("com.example:id/login_form")) ); Find all product prices const prices = screen.filterAdvanced(f => f.isText() .text(t => /^\$\d+/.test(t || "")) .anyParent(p => p.hasId("com.example:id/product_card")) ); for (const price of prices) { console.log("Price:", price.text); } Find first empty input field const emptyInput = screen.findAdvanced(f => f.isEditText() .isEditable() .text(t => !t || t.length === 0) ); if (emptyInput) { await emptyInput.performAction(agent.constants.ACTION_CLICK); await agent.actions.writeText("Hello"); } Find any confirmation button const confirmBtn = screen.findAdvanced(f => f.isButton() .isClickable() .or(o => o.hasText("OK")) .or(o => o.hasText("Confirm")) .or(o => o.hasText("Yes")) .or(o => o.hasText("Accept")) ); -------------------------------------------------------------------------------- ## Reference / Types Path: /docs/automation/reference/types Description: All supporting types used throughout the Automation API Sections: - File Types - Info Types - Email Type - OCR Types - Callback Types - Other Types - Usage Examples Content: This page documents all supporting types, interfaces, and type aliases used by the Automation API. File Types Types used by agent.utils.files methods. Info Types agent.info Email Type OCR Types agent.actions.recognizeText() . The structure follows: TextJSON → TextBlockJSON → LineJSON → ElementJSON → SymbolJSON. Callback Types agent.utils Other Types Usage Examples next Types · Reference Types Reference All supporting types used throughout the Automation API FileInfo Basic file or directory information returned by list(). DirectoryInfo Detailed information about a directory. interface DirectoryInfo { exists: true; path: string; name: string; isDirectory: true; isFile: false; lastModified: number; canRead: boolean; canWrite: boolean; fileCount: number; // Number of files directoryCount: number; // Number of subdirectories totalItems: number; // Total items (files + directories) } FilePathInfo Detailed information about a file. interface FilePathInfo { exists: true; path: string; name: string; isDirectory: false; isFile: true; lastModified: number; canRead: boolean; canWrite: boolean; size: number; } PathNotFoundInfo Returned when a path doesn't exist. interface PathNotFoundInfo { exists: false; error?: string; // Optional error message } FileHashes Hash values for a file. interface FileHashes { md5: string; // MD5 hash sha1: string; // SHA-1 hash sha256: string; // SHA-256 hash size: number; // File size } FileHashError Error returned when hashing fails. interface FileHashError { error: string; } AutomationInfo Metadata about the running automation. interface AutomationInfo { name: string; // Automation name description: string; // Automation description launchId: string; // Unique ID for this execution agent?: { // Optional agent details id: string; commitId: string; token: string; jobTaskId: string; }; serverBaseUrl: string; // Server URL for API calls timeout?: number; // Optional timeout in minutes available since app v2.123 (135) } DeviceInfo Hardware and configuration info about the device. interface DeviceInfo { id: string; // Unique device ID brand: string; // Device brand (e.g., "Samsung") model: string; // Device model sdkVersion: number; // Android SDK version processor: string; // CPU info numberOfCores: number; // CPU cores ramMb: number; // RAM in MB country: string; // Device country isEmulator: boolean; // true if emulator width: number; // Screen width in pixels height: number; // Screen height in pixels } Email Email message returned by readIMAPEmails(). interface Email { id: string; // Unique email ID subject: string; // Email subject from: string; // Sender email address fromName: string; // Sender display name to: string[]; // Recipients cc: string[]; // CC recipients bcc: string[]; // BCC recipients date: number; // Timestamp (Unix ms) body: string; // Email body content isHtml: boolean; // true if body is HTML isRead: boolean; // Read status hasAttachments: boolean; // Has attachments attachmentNames: string[]; // Attachment file names } TextJSON Root OCR result containing all recognized text. interface TextJSON { text: string; // Complete recognized text textBlocks: TextBlockJSON[]; // Array of text blocks } TextBlockJSON A block of text (paragraph or distinct region). interface TextBlockJSON { text: string; // Text content of the block recognizedLanguage: string; // Detected language boundingBox?: RectJSON; // Bounding box on screen lines: LineJSON[]; // Lines within the block } LineJSON A single line of text. interface LineJSON { text: string; // Text content of the line angle: number; // Rotation angle confidence: number; // Recognition confidence (0-1) recognizedLanguage: string; // Detected language boundingBox?: RectJSON; // Bounding box on screen elements: ElementJSON[]; // Words/elements in the line } ElementJSON A word or element within a line. interface ElementJSON { text: string; // Text content (usually a word) angle: number; // Rotation angle confidence: number; // Recognition confidence (0-1) recognizedLanguage: string; // Detected language boundingBox?: RectJSON; // Bounding box on screen symbols: SymbolJSON[]; // Individual characters } SymbolJSON A single character. interface SymbolJSON { text: string; // Single character angle: number; // Rotation angle confidence: number; // Recognition confidence (0-1) recognizedLanguage: string; // Detected language boundingBox?: RectJSON; // Bounding box on screen } RectJSON Bounding box coordinates. interface RectJSON { left: number; // Left edge top: number; // Top edge right: number; // Right edge bottom: number; // Bottom edge } NotificationCallback Callback for receiving system notifications. type NotificationCallback = ( id: string, // Notification ID packageName: string, // Source app package channelId: string, // Notification channel extras: any // Notification data (title, text, etc.) ) => void; NetworkCallback Callback for network state changes. type NetworkCallback = ( networkAvailable: boolean // true if network is available ) => void; ToastCallback Callback for receiving toast messages. type ToastCallback = ( packageName: string, // Source app package data: { message: string } // Toast message ) => void; MultiTapSequenceItem Item in a multi-tap sequence for agent.actions.multiTap(). interface MultiTapSequenceItem { x: number; // X coordinate to tap y: number; // Y coordinate to tap delay: number; // Delay in ms after this tap } Working with file types const info = agent.utils.files.getPathInfo("/sdcard/Download"); if (info.exists) { if (info.isDirectory) { // TypeScript knows this is DirectoryInfo console.log("Contains", info.totalItems, "items"); } else { // TypeScript knows this is FilePathInfo console.log("File size:", info.size); } } else { // TypeScript knows this is PathNotFoundInfo console.log("Path not found:", info.error); } Working with OCR results const { screenshot } = await agent.actions.screenshot(1080, 1920, 90); const result = await agent.actions.recognizeText(screenshot); // Access full text console.log("Full text:", result.text); // Iterate through hierarchy for (const block of result.textBlocks) { console.log("Block:", block.text); for (const line of block.lines) { console.log(" Line:", line.text, "confidence:", line.confidence); // Get bounding box for the line if (line.boundingBox) { const { left, top, right, bottom } = line.boundingBox; console.log(" Position:", left, top, "to", right, bottom); } } } -------------------------------------------------------------------------------- ## Reference / Helper Functions Path: /docs/automation/reference/helpers Description: Standalone utility functions for working with AndroidNode trees Sections: - Functions - When to Use Helper Functions vs Methods Methods: ### getAllNodes() Signature: getAllNodes(rootNode: AndroidNode | null | undefined): AndroidNode[] Returns all nodes in the tree as a flat array. Equivalent to calling rootNode.allNodes() but handles null/undefined input safely. ### buildNodePath() Signature: buildNodePath(rootNode: AndroidNode | null | undefined, targetNode: AndroidNode | null | undefined): AndroidNode[] Builds the path from the root node to the target node as an array of nodes. Useful for understanding the hierarchy or debugging. ### findNodesById() Signature: findNodesById(rootNode: AndroidNode | null | undefined, id: string): AndroidNode[] Finds all nodes with the matching viewId. Equivalent to rootNode.findById(id) but handles null/undefined input. ### findNodesByText() Signature: findNodesByText(rootNode: AndroidNode | null | undefined, text: string): AndroidNode[] Finds all nodes containing the specified text in their text, description, or hintText properties. Case-insensitive search. ### findParentOf() Signature: findParentOf(rootNode: AndroidNode | null | undefined, targetNode: AndroidNode | null | undefined): AndroidNode | null Finds the parent of the target node within the tree. Alternative to accessing targetNode.parent directly, with null-safe handling. Content: These global helper functions provide additional ways to work with the accessibility tree. They are available alongside the methods on AndroidNode instances. Functions When to Use Helper Functions vs Methods The root node might be null or undefined You want consistent null-safe behavior You prefer a functional programming style Working with nodes from potentially failed operations You have a guaranteed non-null AndroidNode You're chaining operations fluently You need the advanced filter builder pattern Writing more concise code next getAllNodes getAllNodes(rootNode: AndroidNode | null | undefined): AndroidNode[] Returns all nodes in the tree as a flat array. Equivalent to calling rootNode.allNodes() but handles null/undefined input safely. rootNode AndroidNode | null | undefined Root node of the tree to flatten AndroidNode[] Flat array of all nodes in the tree, or empty array if input is null/undefined const screen = await agent.actions.screenContent(); const allNodes = getAllNodes(screen); // Count nodes by type const buttonCount = allNodes.filter(n => n.className === "android.widget.Button" ).length; console.log("Total nodes:", allNodes.length); console.log("Buttons:", buttonCount); Safe handling of null // Returns empty array instead of throwing const nodes = getAllNodes(null); // [] buildNodePath buildNodePath(rootNode: AndroidNode | null | undefined, targetNode: AndroidNode | null | undefined): AndroidNode[] Builds the path from the root node to the target node as an array of nodes. Useful for understanding the hierarchy or debugging. Root of the tree targetNode Target node to find path to Array of nodes from root to target, or empty array if path not found const screen = await agent.actions.screenContent(); const button = screen.findTextOne("Submit"); if (button) { const path = buildNodePath(screen, button); // Print the hierarchy console.log("Path to button:"); path.forEach((node, i) => { const indent = " ".repeat(i); console.log(indent + (node.className || "unknown")); }); } Debug node location const target = screen.findByIdOne("com.example:id/hidden_button"); if (target) { const path = buildNodePath(screen, target); console.log("Button is", path.length, "levels deep"); console.log("Parent classes:", path.map(n => n.className).join(" > ")); } findNodesById findNodesById(rootNode: AndroidNode | null | undefined, id: string): AndroidNode[] Finds all nodes with the matching viewId. Equivalent to rootNode.findById(id) but handles null/undefined input. Root of the tree to search string View ID to search for (e.g., 'com.example:id/button') Array of matching nodes, or empty array if none found const screen = await agent.actions.screenContent(); const buttons = findNodesById(screen, "com.example:id/action_button"); console.log("Found", buttons.length, "action buttons"); for (const button of buttons) { console.log("Button text:", button.text); } findNodesByText findNodesByText(rootNode: AndroidNode | null | undefined, text: string): AndroidNode[] Finds all nodes containing the specified text in their text, description, or hintText properties. Case-insensitive search. text Text to search for const screen = await agent.actions.screenContent(); const submitNodes = findNodesByText(screen, "Submit"); // Find all nodes mentioning "error" const errorNodes = findNodesByText(screen, "error"); if (errorNodes.length > 0) { console.log("Found error message:", errorNodes[0].text); } findParentOf findParentOf(rootNode: AndroidNode | null | undefined, targetNode: AndroidNode | null | undefined): AndroidNode | null Finds the parent of the target node within the tree. Alternative to accessing targetNode.parent directly, with null-safe handling. Node to find parent of AndroidNode | null Parent node, or null if not found or target is root const screen = await agent.actions.screenContent(); const button = screen.findTextOne("Submit"); if (button) { const parent = findParentOf(screen, button); if (parent) { console.log("Parent class:", parent.className); console.log("Parent has", parent.children.length, "children"); } } Find containing form const input = screen.findAdvanced(f => f.isEditText()); if (input) { let current = input; let parent = findParentOf(screen, current); // Walk up to find a form container while (parent) { if (parent.viewId?.includes("form")) { console.log("Found form container:", parent.viewId); break; } current = parent; parent = findParentOf(screen, current); } } Helper Functions · Reference Helper Functions Standalone utility functions for working with AndroidNode trees Overview // All helper functions are globally available declare function getAllNodes(rootNode: AndroidNode | null | undefined): AndroidNode[]; declare function buildNodePath(rootNode: AndroidNode | null | undefined, targetNode: AndroidNode | null | undefined): AndroidNode[]; declare function findNodesById(rootNode: AndroidNode | null | undefined, id: string): AndroidNode[]; declare function findNodesByText(rootNode: AndroidNode | null | undefined, text: string): AndroidNode[]; declare function findParentOf(rootNode: AndroidNode | null | undefined, targetNode: AndroidNode | null | undefined): AndroidNode | null; Use Helper Functions When: Use Instance Methods When: Comparison // Using helper function (null-safe) const allNodes = getAllNodes(screen); const byId = findNodesById(screen, "com.example:id/btn"); const byText = findNodesByText(screen, "Submit"); // Using instance methods (requires non-null node) const allNodes2 = screen.allNodes(); const byId2 = screen.findById("com.example:id/btn"); const byText2 = screen.findText("Submit"); // Instance methods also provide advanced filtering const filtered = screen.filterAdvanced(f => f.isButton().isClickable()); -------------------------------------------------------------------------------- ## Reference / API Path: /docs/automation/reference/api Description: REST API endpoints for automation control Sections: - Responses - Log Levels - Usage Example - Error Handling Content: Responses Log Levels Level Description These API endpoints allow you to programmatically control automations on devices. All endpoints require authentication via Bearer token in the Authorization header. All API requests must include an Authorization header with a valid Bearer token: Authorization: Bearer your_api_token Launches a saved automation project on one or more devices. Use this to run automations that have been created and saved in the IDE. Launches automation code directly on a device without saving it as a project. You provide a unique launch ID (UUID) to track the automation and retrieve logs. The launch_id automationLogs to retrieve logs for this automation. Stops all running automations on the specified device. Invalid device_id or device not found Retrieves console logs from running automations. Only returns logs from the last 10 minutes. Use the from parameter for pagination to get logs after a specific log ID. When using launchDirectAutomation you provided in the automation_ids Creates or updates a file within an automation project. For TypeScript files, the content is automatically compiled to JavaScript. Automation project not found or access denied When writing .ts files, the TypeScript is automatically compiled and both the source ( ) and compiled ( .js files that have a paired source. Reads the content of a file within an automation project. File or automation project not found Creates a new commit with all current changes in the automation project. The project must have uncommitted changes for this to succeed. Each automation project has built-in version control. After making changes with the Write Project File API, use this endpoint to commit those changes. The commit hash can be used to track versions and revert if needed. Retrieves out of steps records for automations where you are the owner or a shared editor. These records contain screenshots and UI data captured when an automation encounters an unknown screen or runs out of steps. Returns one record at a time with cursor-based pagination. uiUrl - Full URL to the UI JSON file containing the accessibility tree screenshot.screenshotUrl - Full URL to the screenshot JPEG image nextStage - The stage the automation was attempting to reach screenState - The identified screen state (or "unknown") maxSteps - Remaining steps when captured (0 = out of steps) Mark an out of steps record as solved or skipped. Only automation editors (owner or shared_editor) can mark status. Once marked, status cannot be changed. You don't have permission to mark this out of steps record Once a status is set to solved skipped , it cannot be changed. Make sure you've addressed the issue before marking it. Usage Example Here's a complete example of launching a direct automation and polling for logs: Error Handling The API token is missing, invalid, or expired. Ensure you're passing a valid Bearer token in the Authorization header. The request parameters are invalid or missing. Check that all required fields are provided with correct types. The target device is not connected. Ensure the device is online and has the Xgodo app running before launching automations. next API · Reference json Standard console.log output warn Warning messages from console.warn error Error messages from console.error API Reference REST API endpoints for automation control Authentication POST /api/v2/devices/launchAutomation Launch Automation Request Parameters device_ids string[] Array of device IDs to run the automation on automationId string The ID of the saved automation project command Command to execute: "start" or "stop" automationParameters object Parameters defined in the automation schema jobVariables Job-specific variables for the automation Example Request { "device_ids": ["device_123456", "device_789012"], "automationId": "83ba7359-d8eb-4374-8ecb-055af01fddfe", "command": "start", "automationParameters": { "maxRetries": 3, "timeout": 30000 }, "jobVariables": { "email": "user@example.com", "password": "secret123" } } 200 Success { "success": true, "message": "Action performed successfully" } 400 Bad Request { "success": false, "message": "Some devices offline or not found." } /api/v2/devices/launchDirectAutomation Launch Direct Automation device_id The ID of the device to run automation on string (UUID) Unique identifier for this automation launch (generate a UUID) code JavaScript/TypeScript code to execute on the device Parameters accessible via agent.arguments.automationParameters Variables accessible via agent.arguments.jobVariables { "device_id": "device_123456", "launch_id": "550e8400-e29b-41d4-a716-446655440000", "command": "start", "code": "console.log('Hello from automation!');", "automationParameters": { "maxRetries": 3 }, "jobVariables": { "targetUrl": "https://example.com" } } warning Using the launch_id /api/v2/devices/stopAllAutomations Stop All Automations The ID of the device to stop automations on { "device_id": "device_123456" } { "success": true, "message": "All automations stopped successfully" } /api/v2/devices/automationLogs Get Automation Logs The ID of the device to get logs from Array of automation IDs or launch IDs (for direct automations) to get logs for Log ID to get logs after (for pagination) automation_ids for Direct Automations { "device_id": "device_123456", "automation_ids": ["550e8400-e29b-41d4-a716-446655440000"], "from": "e0e1a3ef-7b2a-42b5-8f21-b845fa41c8b9" } { "logs": [ { "id": "e0e1a3ef-7b2a-42b5-8f21-b845fa41c8b9", "automationId": "550e8400-e29b-41d4-a716-446655440000", "message": "Processing screen content...", "lineNumber": 42, "messageLevel": "log", "timestamp": 1755811585870 }, { "id": "f1f2b4ff-8c3b-53c6-9g32-c956gb52d9ca", "automationId": "550e8400-e29b-41d4-a716-446655440000", "message": "Button clicked successfully", "lineNumber": 45, "messageLevel": "log", "timestamp": 1755811586120 } ] } PUT /api/v2/automation/file/:id Write Project File URL Parameters The automation project ID Request Body path Relative file path within the project (e.g., main.ts, utils/helpers.ts) content The file content (max ~50MB) { "path": "main.ts", "content": "async function main() {\n console.log('Hello!');\n stopCurrentAutomation();\n}\n\nmain();" } { "success": true, "path": "main.ts", "compiledPath": "main.js", "compiled": true } TypeScript Compilation Error { "success": false, "message": "TypeScript compilation failed", "errors": ["Error at 5:10: Property 'foo' does not exist on type 'string'."] } 404 Not Found TypeScript Compilation GET Read Project File Query Parameters Relative file path within the project (e.g., main.ts) bash curl "https://xgodo.com/api/v2/automation/file/83ba7359-d8eb-4374-8ecb-055af01fddfe?path=main.ts" \ -H "Authorization: Bearer your_api_token" { "success": true, "content": "async function main() {\n console.log('Hello!');\n stopCurrentAutomation();\n}\n\nmain();", "path": "main.ts" } /api/v2/automation/commit/:id Commit Project Changes message The commit message describing the changes (1-1000 characters) { "message": "Add error handling to main automation flow" } 201 Created { "success": true, "commit": { "hash": "a1b2c3d4e5f6789012345678901234567890abcd", "message": "Add error handling to main automation flow" } } No Changes { "success": false, "message": "No changes to commit" } Version Control /api/v2/automation-project/out-of-steps Get Out of Steps Records cursor Direct link to a specific record by ID before_cursor Get records older than this cursor (for "next" pagination) after_cursor Get records newer than this cursor (for "prev" pagination) automation_id Filter by automation ID type Filter by type: outOfSteps, timeout, debug, crash status Filter by status: pending, solved, skipped partial Filter by partial: true or false { "success": true, "result": { "device_info": { "brand": "Samsung", "model": "Galaxy S21", ... }, "automation_name": "My Automation", "out_of_steps_id": "507f1f77bcf86cd799439011", "automation_id": "507f1f77bcf86cd799439012", "type": "outOfSteps", "status": "pending", "screens": [ { "_id": "507f1f77bcf86cd799439013", "uiUrl": "https://server.com/outofsteps/2024/01/15/.../ui_....json", "screenshot": { "screenshotUrl": "https://server.com/outofsteps/2024/01/15/.../screenshot_....jpeg", "compressedWidth": 540, "compressedHeight": 1200, "originalWidth": 1080, "originalHeight": 2400 }, "nextStage": "main", "screenState": "unknown", "maxSteps": 0, "timestamp": 1705334400000 } ], "added": "2024-01-15T12:00:00.000Z" }, "total": 100, "currentPage": 1, "currentCursor": "507f1f77bcf86cd799439011", "hasNext": true, "hasPrev": false } Screen Object Fields PATCH /api/v2/automation-project/out-of-steps/:id/status Mark Out of Steps Status The out of steps record ID The status to set: "solved" or "skipped" { "status": "solved" } { "success": true, "message": "Out of steps marked as solved", "status": "solved" } Already Marked { "success": false, "message": "Status already marked as \"solved\". Cannot change status once marked." } 403 Forbidden Status is Immutable Example: Launch and Monitor Direct Automation const API_BASE = "https://xgodo.com"; const TOKEN = "your_api_token"; // Generate a unique launch ID function generateUUID(): string { return crypto.randomUUID(); } async function runDirectAutomation(deviceId: string, code: string) { const launchId = generateUUID(); // Launch the automation const launchRes = await fetch(`${API_BASE}/api/v2/devices/launchDirectAutomation`, { method: "POST", headers: { "Content-Type": "application/json", "Authorization": `Bearer ${TOKEN}` }, body: JSON.stringify({ device_id: deviceId, launch_id: launchId, command: "start", code: code, automationParameters: { maxRetries: 3 }, jobVariables: { targetUrl: "https://example.com" } }) }); const launchData = await launchRes.json(); if (!launchData.success) { throw new Error(launchData.message); } console.log("Automation started with launch ID:", launchId); // Poll for logs using the launch_id let lastLogId: string | undefined; const pollLogs = async () => { const logsRes = await fetch(`${API_BASE}/api/v2/devices/automationLogs`, { method: "POST", headers: { "Content-Type": "application/json", "Authorization": `Bearer ${TOKEN}` }, body: JSON.stringify({ device_id: deviceId, automation_ids: [launchId], // Use launch_id here from: lastLogId }) }); const logsData = await logsRes.json(); for (const log of logsData.logs) { console.log(`[${log.messageLevel}] ${log.message}`); lastLogId = log.id; } }; // Poll every 2 seconds const interval = setInterval(pollLogs, 2000); // Stop after 60 seconds setTimeout(async () => { clearInterval(interval); // Stop the automation await fetch(`${API_BASE}/api/v2/devices/stopAllAutomations`, { method: "POST", headers: { "Content-Type": "application/json", "Authorization": `Bearer ${TOKEN}` }, body: JSON.stringify({ device_id: deviceId }) }); console.log("Automation stopped"); }, 60000); } // Usage runDirectAutomation("device_123456", ` const params = agent.arguments.automationParameters; const vars = agent.arguments.jobVariables; console.log("Max retries:", params.maxRetries); console.log("Target URL:", vars.targetUrl); await agent.actions.goHome(); console.log("Done!"); stopCurrentAutomation(); `); 401 Unauthorized 400 Bad Request Device Offline -------------------------------------------------------------------------------- ## Reference / agent.actions.adb Path: /docs/automation/reference/agent/actions/adb Description: ADB shell-based device actions that bypass the accessibility service. Methods: ### tap() Signature: tap(x: number, y: number): Promise Performs a single tap at the specified screen coordinates using ADB shell input. ### hold() Signature: hold(x: number, y: number, duration: number): Promise Performs a long press at the specified coordinates using ADB shell input. ### swipe() Signature: swipe(x1: number, y1: number, x2: number, y2: number, duration: number): Promise Performs a swipe gesture from one point to another using ADB shell input. ### swipePoly() Signature: swipePoly(startX: number, startY: number, sequence: {x: number, y: number, duration?: number}[], duration: number): Promise Swipes through multiple points sequentially using ADB shell input. Each segment is executed as a separate ADB swipe command. Optionally supports per-segment duration. ### doubleTap() Signature: doubleTap(x: number, y: number, interval?: number): Promise Performs a double tap at the specified coordinates using ADB shell input. ### goHome() Signature: goHome(): Promise Returns to the home screen using ADB keyevent. ### goBack() Signature: goBack(): Promise Presses the system back button using ADB keyevent. ### recents() Signature: recents(): Promise Opens the recent apps screen using ADB keyevent. ### dpad() Signature: dpad(direction: "up" | "down" | "left" | "right" | "center"): Promise Sends a D-pad navigation event using ADB keyevent. ### inputKey() Signature: inputKey(keyCode: number, duration?: number): Promise Sends a key input event using ADB keyevent. Supports long press via duration parameter. ### writeText() Signature: writeText(text: string): Promise Types text using ADB shell input. Special characters are automatically escaped. ### launchApp() Signature: launchApp(packageName: string): Promise Launches an app by its package name using ADB monkey command. ### toggleScreenLock() Signature: toggleScreenLock(): Promise Toggles screen lock by pressing the power button using ADB keyevent 26 (KEYCODE_POWER). Turns the screen off if on, or on if off. ### isDeviceLocked() Signature: isDeviceLocked(): Promise Checks if the device is locked. Returns true if the screen is off or the keyguard (lock screen) is showing. ### unlockDevice() Signature: unlockDevice(): Promise Attempts to unlock the device by pressing the power button. Handles the case where the screen is on but locked (keyguard showing) — pressing power would turn the screen off, so it detects this and presses power again. Returns true if the device is now unlocked. Does not handle PIN/pattern entry. ### paste() Signature: paste(): Promise Pastes clipboard contents using ADB keyevent 279 (KEYCODE_PASTE). ### screenContent() Signature: screenContent(): Promise Gets the current screen content (UI hierarchy) via ADB uiautomator dump. Returns the same AndroidNode structure as the regular screenContent() method. ### listApps() Signature: listApps(): Promise Gets the list of installed package names via ADB pm list packages. ### screenshot() Signature: screenshot(maxWidth: number, maxHeight: number, quality: number, cropX1?: number, cropY1?: number, cropX2?: number, cropY2?: number): Promise<{...}> Takes a screenshot via ADB screencap. Returns the image as a base64-encoded JPEG string along with dimension metadata. ### allScreensContent() Signature: allScreensContent(): Promise Gets the UI hierarchy from all screens/windows via ADB. Returns an array of AndroidNode objects, one per window. ### nodeAction() Signature: nodeAction(node: AndroidNode | object, actionInt: number, data?: object, fieldsToIgnore?: string[]): Promise<{actionPerformed: boolean}> Performs an accessibility action on a node via ADB. This is the ADB equivalent of agent.actions.nodeAction(). Uses UiAutomation through the ADB touch server instead of the accessibility service. Also available as node.adbPerformAction(). ### logcat() Signature: logcat(options?: { args?: string; packageName?: string; filter?: string; priority?: string; regex?: string; buffer?: string; limit?: number; order?: 'asc' | 'desc' }): Promise<{ logs: string[]; count: number; command: string | null }> Reads device logs via `adb shell logcat -d` (dump and exit). Two usage modes are supported: pass an `args` string for full control over logcat flags (escape hatch for power users), or use the structured filter options (packageName, filter, priority, etc.) and let the platform build the command. The `limit` is always enforced — even when `args` is supplied, `-t ` is auto-appended unless `args` already contains `-t` or `-T`. Content: agent.actions.adb . These actions use ADB shell commands instead of the accessibility service, providing an alternative when the accessibility service is unavailable or unreliable. Available since: App version 2.141 (153) ", description: "Reads device logs via `adb shell logcat -d` (dump and exit). Two usage modes are supported: pass an `args` string for full control over logcat flags (escape hatch for power users), or use the structured filter options (packageName, filter, priority, etc.) and let the platform build the command. The `limit` is always enforced — even when `args` is supplied, `-t next agent.actions.adb · Reference ADB Actions Actions ADB shell-based device actions that bypass the accessibility service info tap(x: number, y: number): Promise Performs a single tap at the specified screen coordinates using ADB shell input. number X coordinate on screen Y coordinate on screen Promise Resolves when tap is complete Simple tap await agent.actions.adb.tap(100, 200); hold hold(x: number, y: number, duration: number): Promise Performs a long press at the specified coordinates using ADB shell input. duration Hold duration in milliseconds Resolves when hold is complete Long press for 1 second await agent.actions.adb.hold(500, 500, 1000); swipe swipe(x1: number, y1: number, x2: number, y2: number, duration: number): Promise Performs a swipe gesture from one point to another using ADB shell input. x1 Starting X coordinate y1 Starting Y coordinate x2 Ending X coordinate y2 Ending Y coordinate Duration in milliseconds Resolves when swipe is complete Swipe up await agent.actions.adb.swipe(500, 1500, 500, 500, 300); swipePoly swipePoly(startX: number, startY: number, sequence: {x: number, y: number, duration?: number}[], duration: number): Promise Swipes through multiple points sequentially using ADB shell input. Each segment is executed as a separate ADB swipe command. Optionally supports per-segment duration. startX startY sequence {x, y, duration?}[] Array of points to swipe through. Each point can optionally specify its own duration in ms. Total duration in ms. Divided equally among segments if per-point durations are not specified. Use 0 for default (~500ms total). Resolves when all swipe segments are complete Draw an L shape await agent.actions.adb.swipePoly(100, 100, [ { x: 100, y: 500 }, { x: 400, y: 500 } ], 600); With per-segment duration await agent.actions.adb.swipePoly(100, 100, [ { x: 100, y: 500, duration: 200 }, { x: 400, y: 500, duration: 300 } ], 0); doubleTap doubleTap(x: number, y: number, interval?: number): Promise Performs a double tap at the specified coordinates using ADB shell input. interval Interval between taps in ms (default: 100) Resolves when double tap is complete Double tap await agent.actions.adb.doubleTap(500, 500); Double tap with custom interval await agent.actions.adb.doubleTap(500, 500, 200); goHome goHome(): Promise Returns to the home screen using ADB keyevent. Resolves when navigation is complete await agent.actions.adb.goHome(); goBack goBack(): Promise Presses the system back button using ADB keyevent. Resolves when back action is complete await agent.actions.adb.goBack(); recents recents(): Promise Opens the recent apps screen using ADB keyevent. Resolves when recent apps screen is shown await agent.actions.adb.recents(); dpad dpad(direction: "up" | "down" | "left" | "right" | "center"): Promise Sends a D-pad navigation event using ADB keyevent. direction "up" | "down" | "left" | "right" | "center" Direction to navigate await agent.actions.adb.dpad("down"); await agent.actions.adb.dpad("center"); // Select/Enter inputKey inputKey(keyCode: number, duration?: number): Promise Sends a key input event using ADB keyevent. Supports long press via duration parameter. keyCode Android KeyEvent code (e.g., 66 for Enter, 67 for Backspace) Press duration in ms. 0 for normal press, >0 for long press (default: 0) Resolves when key event is complete Press Enter await agent.actions.adb.inputKey(66); Long press a key await agent.actions.adb.inputKey(66, 1000); writeText writeText(text: string): Promise Types text using ADB shell input. Special characters are automatically escaped. text string Text to type Resolves when text is typed await agent.actions.adb.writeText("Hello World"); launchApp launchApp(packageName: string): Promise Launches an app by its package name using ADB monkey command. packageName The app's package name (e.g., com.android.chrome) Resolves when app launch command is sent await agent.actions.adb.launchApp("com.android.chrome"); toggleScreenLock toggleScreenLock(): Promise Toggles screen lock by pressing the power button using ADB keyevent 26 (KEYCODE_POWER). Turns the screen off if on, or on if off. Turn screen off await agent.actions.adb.toggleScreenLock(); isDeviceLocked isDeviceLocked(): Promise Checks if the device is locked. Returns true if the screen is off or the keyguard (lock screen) is showing. Promise true if device is locked, false if unlocked Check lock state const locked = await agent.actions.adb.isDeviceLocked(); console.log("Device locked:", locked); unlockDevice unlockDevice(): Promise Attempts to unlock the device by pressing the power button. Handles the case where the screen is on but locked (keyguard showing) — pressing power would turn the screen off, so it detects this and presses power again. Returns true if the device is now unlocked. Does not handle PIN/pattern entry. true if device is now unlocked, false if unlock failed Unlock before automation const unlocked = await agent.actions.adb.unlockDevice(); if (!unlocked) { console.log("Could not unlock device"); return; } paste paste(): Promise Pastes clipboard contents using ADB keyevent 279 (KEYCODE_PASTE). Resolves when paste is complete Copy text then paste via ADB await agent.actions.copyText("Hello World"); await agent.actions.adb.paste(); screenContent screenContent(): Promise Gets the current screen content (UI hierarchy) via ADB uiautomator dump. Returns the same AndroidNode structure as the regular screenContent() method. Promise Root node of the accessibility tree Get screen content via ADB const screen = await agent.actions.adb.screenContent(); const button = screen.findTextOne("Submit"); listApps listApps(): Promise Gets the list of installed package names via ADB pm list packages. Promise Array of installed package names List installed apps const apps = await agent.actions.adb.listApps(); console.log(apps); // ["com.android.chrome", "com.google.android.youtube", ...] screenshot screenshot(maxWidth: number, maxHeight: number, quality: number, cropX1?: number, cropY1?: number, cropX2?: number, cropY2?: number): Promise<{...}> Takes a screenshot via ADB screencap. Returns the image as a base64-encoded JPEG string along with dimension metadata. maxWidth Maximum width to scale down to maxHeight Maximum height to scale down to quality JPEG quality (1-100) cropX1 Crop left coordinate cropY1 Crop top coordinate cropX2 Crop right coordinate cropY2 Crop bottom coordinate Promise<{ screenshot, compressedWidth, compressedHeight, originalWidth, originalHeight }> Screenshot data with base64 image and dimensions Take a screenshot via ADB const result = await agent.actions.adb.screenshot(720, 1280, 80); console.log(result.originalWidth, result.originalHeight); Take a cropped screenshot const result = await agent.actions.adb.screenshot(720, 1280, 80, 0, 0, 360, 640); allScreensContent allScreensContent(): Promise Gets the UI hierarchy from all screens/windows via ADB. Returns an array of AndroidNode objects, one per window. Promise Array of root nodes, one per screen/window Get all screens content via ADB const screens = await agent.actions.adb.allScreensContent(); for (const screen of screens) { const buttons = screen.findText("OK"); console.log(buttons.length); } nodeAction nodeAction(node: AndroidNode | object, actionInt: number, data?: object, fieldsToIgnore?: string[]): Promise<{actionPerformed: boolean}> Performs an accessibility action on a node via ADB. This is the ADB equivalent of agent.actions.nodeAction(). Uses UiAutomation through the ADB touch server instead of the accessibility service. Also available as node.adbPerformAction(). node AndroidNode | object The node to perform action on actionInt Action constant (use agent.constants.ACTION_*) data object Additional action data fieldsToIgnore string[] Fields to ignore when matching node Promise<{ actionPerformed: boolean }> Whether the action was successfully performed Click a button via ADB const screen = await agent.actions.adb.screenContent(); const button = screen.findTextOne("Submit"); if (button) { await agent.actions.adb.nodeAction(button, agent.constants.ACTION_CLICK); } Using adbPerformAction on a node const screen = await agent.actions.adb.screenContent(); const button = screen.findTextOne("Submit"); if (button) { await button.adbPerformAction(agent.constants.ACTION_CLICK); } logcat logcat(options?: { args?: string; packageName?: string; filter?: string; priority?: string; regex?: string; buffer?: string; limit?: number; order?: 'asc' | 'desc' }): Promise<{ logs: string[]; count: number; command: string | null }> Reads device logs via `adb shell logcat -d` (dump and exit). Two usage modes are supported: pass an `args` string for full control over logcat flags (escape hatch for power users), or use the structured filter options (packageName, filter, priority, etc.) and let the platform build the command. The `limit` is always enforced — even when `args` is supplied, `-t ` is auto-appended unless `args` already contains `-t` or `-T`. options.args Raw logcat argument string (e.g. "-b crash *:E"). If provided, structured filter options are ignored. options.packageName Filter to a specific app — uses `--pid=$(pidof -s )`, so the app must be currently running. options.filter Tag filter spec, e.g. "MyTag:V *:S". Takes precedence over `priority`. options.priority "V" | "D" | "I" | "W" | "E" | "F" | "S" Minimum log priority (Verbose, Debug, Info, Warn, Error, Fatal, Silent). Used only if `filter` is not set. options.regex Only return lines matching this regex (passed as logcat `-e`). options.buffer "main" | "system" | "radio" | "events" | "crash" | "all" Log buffer to read from. options.limit Maximum number of log lines to return. Default 100, max 5000. options.order "asc" | "desc" Time ordering: "asc" = oldest first, "desc" = newest first. Default "desc". Promise<{ logs: string[]; count: number; command: string | null }> Object containing the log lines array, count, and the actual logcat command that was executed (useful for debugging). Get latest 50 errors const result = await agent.actions.adb.logcat({ priority: "E", limit: 50 }); console.log(result.command); // "logcat -d -v threadtime -t 50 *:E 2>&1" result.logs.forEach(line => console.log(line)); Filter by package (app must be running) const result = await agent.actions.adb.logcat({ packageName: "com.android.chrome", limit: 200, order: "asc", }); Read crash buffer const result = await agent.actions.adb.logcat({ buffer: "crash", limit: 100 }); Search by regex const result = await agent.actions.adb.logcat({ regex: "ANR|FATAL", limit: 100 }); Tag filter spec const result = await agent.actions.adb.logcat({ filter: "ActivityManager:I *:S", limit: 100 }); Raw args (escape hatch) // Limit is auto-appended as "-t 30" since args has no -t. const result = await agent.actions.adb.logcat({ args: "-b system *:W", limit: 30, }); --------------------------------------------------------------------------------