TEST AUTHORING
Harness AI

Best Practices: AI Based Commands

9min

This page outlines the four primary AI command types used in our testing framework, along with detailed explanations and best practices for each type.

AI Assertion

AI Assertions are verification commands that check if specific conditions are true in the application's current state. They function as test validation points and always return a boolean result (true/false).

Purpose: To verify that the application is in the expected state at specific points in test execution.

Example:

  • "Am I on the home page?"
  • "Is the account balance displayed as $1,250.00?"
  • "Does the error message contain the text 'Invalid credentials'?"
  • "Are there exactly 5 items in the shopping cart?"

When to use:

  • After navigation actions to confirm the correct page loaded
  • Before critical actions to ensure preconditions are met
  • After data operations to verify changes were applied correctly
  • To check for the presence or absence of UI elements

Best Practices:

  • Always end your question with a question mark ("?") and use consistent formats like "Is [expected condition] true?" for clearer assertions
  • Formulate simple, binary (yes/no) questions that the AI can evaluate based solely on the information displayed
  • Since the AI lacks context from previous steps, include specific reference points (e.g., "In the confirmation dialog, is the deposit amount $100?")
  • Include both positive and negative assertions to ensure comprehensive test coverage (e.g., "Is the error message absent from the screen?")
  • Create atomic assertions that test only one condition per question for better error isolation
  • Avoid compound questions that check multiple conditions (use separate assertions instead)
  • Be specific about the expected values or states you're verifying

AI Command

AI Commands instruct the AI to perform specific, discrete actions within the application interface, similar to how a user would interact with UI elements.

Purpose: To manipulate the application interface by interacting with specific elements or controls.

Example:

  • "Pick a date two days in the future from the Calendar"
  • "Click on the 'Submit' button at the bottom of the form"
  • "Enter '[email protected]' in the email field"
  • "Select 'California' from the state dropdown menu"
  • "Scroll down until the 'Load More' button is visible"

When to use:

  • For basic UI interactions like clicking, typing, selecting, and scrolling
  • When you need precise control over individual steps in a workflow
  • For test setup operations that prepare the application state
  • To interact with specific UI components that require direct manipulation

Best Practices:

  • Prepare for dynamic scenarios by using descriptive references rather than fixed positions (e.g., "Click on the row where the status first appears as 'completed'" instead of "Click on the third row")
  • Make commands precise and specific, including both the action and the expected target element with distinguishing characteristics (e.g., "Click the 'Submit' button with the green background in the payment form")
  • Since commands lack temporal awareness, include sequencing hints or waiting conditions when needed (e.g., "After the loading indicator disappears, click the 'Continue' button")
  • Use semantic identifiers instead of technical selectors when possible to improve test readability and maintenance (e.g., "Click on the shopping cart icon" rather than referencing element IDs)
  • Consider edge cases where elements might be conditionally present and provide fallback actions
  • For input fields, specify the exact format of data to be entered, including any special characters or formatting
  • When dealing with tables or lists, describe how to identify the target row or item based on its content rather than position

AI Task

AI Tasks represent high-level business operations that may involve multiple steps and decisions. They operate at a higher abstraction level than individual commands.

Purpose: To execute complete user journeys or business workflows with a single instruction.

Example:

  • "Deposit $100 into checking account"
  • "Purchase the cheapest available flight from New York to London for next Friday"
  • "Register a new user account with random valid information"
  • "Add three items to the cart and complete checkout with standard shipping"
  • "Search for and apply to jobs matching 'software engineer' with salary above $100k"

When to use:

  • For end-to-end workflow testing that mimics real user journeys
  • When you want to abstract away implementation details
  • For data setup that requires complex business operations
  • To test complete business processes rather than individual UI interactions

Best Practices:

  • Author an intent-based action which clearly represents a complete business workflow with specific values (e.g., "Deposit $100 into Checking Account" instead of "Make a deposit")
  • Make your actions specific while including contextual details that reduce ambiguity, such as preconditions, account types, or expected outcomes (e.g., "Deposit $100 into Checking Account and verify the updated balance reflects the deposit")
  • While prompts don't need to follow a specific format, structure complex workflows using Gherkin syntax (Given/When/Then) to clarify preconditions, actions, and expected outcomes
  • Break down complex business processes into a sequence of smaller, focused tasks that can be more easily understood and executed by the AI
  • Include relevant business rules or validation criteria that should be considered during task execution (e.g., "Transfer $500 from Savings to Checking, ensuring sufficient funds are available in Savings")
  • Consider providing examples of expected outputs to guide the AI's understanding
  • Specify how to handle expected variations or decision points in the workflow
  • Include success criteria so the AI knows when the task is complete
  • For data-intensive tasks, clearly specify what data should be used or how it should be generated

AI Extract Data

AI Extract Data commands capture specific information from the application for storage and later use in test execution.

Purpose: To dynamically retrieve and store data from the application interface for verification or subsequent operations.

Example:

  • "Create parameter NAME and assign the name of the user on the welcome message"
  • "Extract the order number from the confirmation page and store it as ORDER_ID"
  • "Capture all product prices on the search results page and store as PRICE_LIST"
  • "Read the error message text and save it as ERROR_MESSAGE"
  • "Get the total amount from the invoice and store it as INVOICE_TOTAL"

When to use:

  • When you need to capture dynamic data generated during test execution
  • For retrieving values that need to be verified later in the test
  • To gather data needed for subsequent test steps
  • For comparing values across different parts of the application
  • To extract information for reporting or debugging purposes

Best Practices:

  • Make commands precise and specific to the data you want the AI to extract
  • Make sure the data is in the viewport and legible to human eyes
  • When extracting from structured documents, mention the relevant headers or sections to help the AI focus on the right areas
  • Specify the format you expect the extracted data to be in (text, number, date, etc.)
  • Use descriptive parameter names that clearly indicate what data they contain
  • For tables or lists, clarify whether you want to extract a single value or multiple values
  • When extracting dates, numbers, or currency values, indicate if any formatting should be preserved or removed
  • Provide context on how to identify the correct data when similar items appear multiple times
  • Specify fallback behavior if the expected data cannot be found

๏ปฟ