Skip to main content

Set up Katalon AI Agent

This document shows you how to set up Katalon AI Agent for autonomous test generation with TrueTest.

Important:

Add application under test​

To configure autonomous test generation, first you need to define an application under test (AUT). The AUT specifies the production environment where Katalon AI can track all user interactions and leverage them in generating test cases.

Follow these steps:

  1. Log in to TestOps Homepage.
  2. On the top right corner, select Settings > TrueTest.
    TestOps Settings > TrueTest
    The TrueTest Configuration menu displays as below.autonomous testing menu
  3. Click Add Application Under Test. In the displayed dialog, provide the information of your AUT.
    Add application under test dialog

    • Name: The displayed name of your application.

    • Application Domain: The domain of your deployed applications. A user journey can involve multiple domains. This domain must not include: protocol name, "www" prefix, or trailing slash.

      For example, valid domains can be *.katalon.com or katalon.com.

    • Description: The description for your application.

  4. Click Add.
Your application under test is successfully added.

Configure the application under test​

After the AUT is defined, you need to install the event tracking agent in your application, provide the development environment and link with the test project.

Select the AUT you want to configure. The configuration menu appears with three main components:
  • Install TrueTest on your application. You can manually add the tracking snippet in the AUT source code or use Google Tag Manager.
  • Add Test Environment(s).
  • Link Project(s).

After configuring the three components, navigate back to the Application Under Test list and check the Activation Date, Last Data Received Date data columns, and the Active status. These items signify that the Katalon AI Agent has successfully tracked data from the AUT.

Install TrueTest snippet​

A developer or someone with access to the codebase of the AUT must add the tracking code snippet provided by Katalon. The code snippet should be pasted into the <HEAD> element of the HTML code of the AUT. Once the code snippet is added, the AUT should be saved, built, and deployed to the production environment.

TrueTest also provides the option to install with Google Tag Manager.

Follow one of the two steps below to install the code snippet:

  • Install the code snippet manually. In the Install your TrueTest snippet section, copy the displayed code snippet and paste into the <HEAD> element of your application under test.
    <!DOCTYPE html>
    <html>
      <head>
        <!-- Paste the Katalon AI code snippet here -->
      </head>
      <body></body>
    </html>
  • If you've already implemented Google Tag Manager as the tag management system, you can dynamically set up code snippet.
    1. Log in to Google Tag Manager.
    2. In the container that is connected to the application under test, create a new tag.
    3. Name and edit the tag. Here we define a custom HTML tag.
      GTM - Add custom HTML tag
    4. Paste the provided snippet into HTML block. Here, the CLIENT-CODE is the code displayed in the configuration UI.
      GTM - paste code snippet
    5. Add a triggering event and select the All pages trigger to ensure that the TrueTest code snippet is enabled on all pages where the GTM is enabled.
      GTM - add trigger
Once saved, the TrueTest tracking snippet will be added to the HTML header when a user accesses the application.

Add test environment​

To generate test cases with traffic collected from the production site, Katalon AI Agent requires a test environment, which functions similarly to a profile in Katalon Studio for managing configurations and scripts. Additionally, you will need to provide a login script within the test environment if the AUT requires login to complete any user flows. This will assist TrueTest in injecting authentication steps into generated test scripts

To add a test environment, follow these steps:

Click on the Add Test Environment button. In the opened dialog, you need to provide the test environment for the test execution. Provide the name and the URL for the environment. You can also provide login credentials for accessing that environment if necessary.
Add test environment dialog

Environment URL should follow the correct pattern with the appropriate protocol and should not contain unnecessary trailing slashes. For example, a valid domain can be https://qe.your-app.com.

For login method, there are two options: No Authentication or Others for custom login script.TrueTest - Login Method - others

Note:

Before generating TrueTest test cases, make sure that you've committed the changes you made, on login scripts and any other related test objects, to the Git repository: Link project.

You can use the record and playback feature in Katalon Studio to record the login actions on your test environment. This will help generate a corresponding login script. See: Record and playback in Katalon Studio.

There are three ways to use the custom script.

  1. Copy the entire login script and paste it into the script editor area.
  2. Use the callTestCase() keyword.
    You can package the login script as a test case and call it with the keyword.
  3. Use a custom keyword.
    You can package your login script into a custom keyword and use it the script editor.

Once test cases are generated, TrueTest needs to store the tests in a repository.

Note:

The Git script repository must be configured with Write permission.

Click on the Link Project button to specify the test project and script repository. The test cases will be committed and pushed to the script repository, which can then be executed as Katalon test scripts.
Link project dialog

Data privacy configuration (optional)​

TrueTest has a filtering mechanism to exclude sensitive information from user's application under test, see: Protect user data privacy.

Additionally, users have the option to manually specify which fields or elements should be excluded while TrueTest is collecting data. To do so, simply add CSS class katalon-excluded to the UI elements that you do not want their values to be captured.

katalon-excluded examples

Advanced settings​

The advanced settings section allows you to further customize the behavior of Katalon AI agent while generating test cases for your AUT.

To access the settings, in the Application Under Test list, click on the desired AUT and switch to the Advanced Settings tab.

Default Selection Method: You can specify the main selection method that Katalon Studio should use when running the generated test cases.

Common Functions: When collecting data among multiple user flows, TrueTest can detect which group of steps are common across the flows and organize these steps into common functions. For example, in the checkout flow of an online shopping website, users may need to fill in billing addresses, shipping addresses, payment information, etc. These steps can be organized into a function. The Minimum number steps specify the minimum number of steps that TrueTest can include in a function.

Data Tracking: Enabling more data tracking can enhance the accuracy of generated test cases.