Fix problems found while testing your AI agent

Flag test replies that went wrong, find out why, and apply and retest a fix from your AI agent's Issues tab.

Written By Frieda Yip (Super Administrator)

Updated at November 18th, 2026

When you test your AI agent in "Preview", you can flag replies that went wrong. Each flagged reply is checked in the background and grouped into an issue on the agent's "Issues" tab. An issue shows what went wrong, the likely cause, and a suggested fix you can apply and then retest.

Use Issues to:

  • Find out why a test reply went wrong
  • Apply a suggested change to the Playbook or Knowledge
  • Add what the reply should have said when the agent doesn't know
  • Retest the same replies after a fix
  • Undo a Playbook fix that made things worse

Issues are saved per agent. Everyone who edits the agent sees the same issues.

Note: Issues only covers test replies from "Preview". Replies reported from Inbox go to Optimization. To learn more, refer to Review and improve AI agent responses with Optimization.

 

 

Flag test replies

You can flag a reply while testing in "Chat", or rate a question Poor in "Batch test". Both become issues on the "Issues" tab.

 

Flag a reply in Chat

  1. In the left navigation bar, click the icon to go to the "SleekFlow AI" page.
  2. Select the "AgentFlow" tab.
  3. Click the name of the AI agent you want to test. The agent builder opens.
  4. Click the "Preview" tab at the top of the page, then select the "Chat" tab.
  5. Send a test message to your agent.
  6. Click "Flag".
  7. Under "What went wrong?", describe the problem. This field is required.
  8. Click "Flag reply".

You can keep testing right away. The check runs in the background and doesn't block the conversation. The reply shows that it's been flagged, along with the issue's current state. Click the state to open the issue.

If the flag couldn't be sent or checked, the reply says so. Click "Retry" to send it again.

Warning: A submitted flag is final. You can't edit its note or remove the flag.

 

 

Rate a Batch test question Poor

  1. In the left navigation bar, click the icon to go to the "SleekFlow AI" page.
  2. Select the "AgentFlow" tab.
  3. Click the name of the AI agent you want to test. The agent builder opens.
  4. Click the "Preview" tab at the top of the page, then select the "Batch test" tab.
  5. Run your questions. To learn more, refer to Test and deploy your AI agent.
  6. Click an answered question to open its panel.
  7. Under "Rate the response", select "Poor".
  8. Optionally, describe the problem under "What went wrong? (optional)".
  9. Leave the question. You can close its panel, open another question, switch to "Chat", or leave "Preview".

The issue is created when you leave the question, not when you select "Poor". After that, the question's row and panel show the issue's state. Click the state to open the issue.

  • A question rated Poor with no note still becomes an issue. Without a note, the check may need you to say what the reply should have said.
  • Once the question becomes an issue, its rating stays "Poor" until you run the question again. Running it again doesn't remove the issue.
  • "Reset results" clears answers, ratings and notes from Batch test. Issues stay on the "Issues" tab.

 

Review issues on the Issues tab

Open the Issues tab

  1. In the left navigation bar, click "SleekFlow AI".
  2. Select the "AgentFlow" tab.
  3. Click the name of your AI agent. The agent builder opens.
  4. Click the "Issues" tab at the top of the page, next to "Edit" and "Preview".

"Deployments" also appears in this tab row once the agent has a deployment.

The "Issues" tab shows how many issues are waiting on you: those in "Fix ready", "Needs your input" or "No change needed". When nothing is waiting, no count is shown. While flagged replies are still being checked, the list shows how many are being checked.

 

Understand the issues list

Each issue in the list shows its title, its state, how many replies it holds, where they came from ("Chat" or "Batch test"), and when a reply was last flagged.

Issues are grouped into "Open" and "Done":

  • "Open" lists issues waiting on you or still being checked, in this order: "Fix ready", "Needs your input", "No change needed", "Checking". Within each state, the most recently flagged comes first.
  • "Done" lists applied and dismissed issues, most recently flagged first. It's collapsed by default.

A new flag joins an open issue that has the same cause. Otherwise, it starts a new issue. A new flag never reopens an issue in "Done".

 

Filter issues

To narrow the list, use the filters:

  • “Status is”:
    • Fix ready
    • Needs your input
    • No change needed
    • Checking
    • Applied
    • Dismissed
  • “Last flagged”:
    • Any time
    • Today
    • Last 7 days
    • Last 30 days
    • Custom range

Click “Clear filters” to reset both.

 

Understand issue states

State

What it means

What you can do

Checking

The flagged reply is being checked.

Keep testing. Nothing is needed from you.

Fix ready

The check found a likely cause and suggests a change.

Review the change, then click "Apply fix" or "Dismiss issue".

Needs your input

The check needs to know what the reply should have said.

Enter the expected reply and click "Check again", or click "Dismiss issue".

No change needed

The reply matches how the agent is set up.

Click "Mark as resolved", or "Mark as still wrong" if you disagree.

Applied

The suggested fix was applied to the agent.

Click "Retest" or "Undo fix".

Dismissed

You dismissed the issue or marked it as resolved.

Nothing. It stays in "Done".

An issue only moves to "Done" when you apply, dismiss, or mark it as resolved. It never moves there on its own.

 

Review an issue

  1. Open the "Issues" tab, as described in "Open the Issues tab".
  2. Click an issue to open it.

 

From top to bottom, the issue shows:

  • Conversation examples: each flagged conversation, with the flagged reply marked and your note under it. Earlier messages in the conversation are collapsed. Click to show them.
  • "Likely cause": why the reply went wrong, with the exact rule, procedure step or Knowledge article quoted. Click "View in playbook" to open that section in "Edit", or "View article" to open the Knowledge article. If two articles disagree, both are shown.
  • The section for the issue's current state, described in "Fix an issue".

If you opened the issue from a reply, click "Back to Chat" or "Back to Batch test case" to return to it.

Note: Missing Knowledge is judged only from the Knowledge the agent retrieved for that reply. If the information is in your Knowledge base but the agent didn't find it, the issue may still suggest adding it. Check your Knowledge sources before adding the same information again.

 

 

When an issue is out of date

An issue in "Fix ready", "Needs your input" or "No change needed" is out of date when the Playbook or Knowledge changed after it was checked or after its fix was suggested. The issue says what changed.

An out-of-date issue offers only "Check if resolved" and "Dismiss issue". You can't apply the old suggestion.

 

Fix an issue

Apply a suggested fix

When an issue is in "Fix ready", "Suggested fix" shows the change as it will look in "Edit":

  • For a Rules field ("Tone of voice", "Guardrails" or "About your business"), the field with the changed words marked.
  • For a procedure, the changed steps marked. Click "Show all" to see the unchanged steps too.
  • For a new scenario, the whole scenario.
  • For an agent with a free-text Playbook, the changed text.

Use the link under the change to view that section in "Edit", or click "Edit procedure instead" to make the change yourself.

To apply the fix:

  1. On the "Issues" tab, open an issue in "Fix ready".
  2. Review the change under "Suggested fix".
  3. Click "Apply fix". The change is added to "Edit" as an unsaved change, and the issue shows as "Applied".
  4. Click "Save" in the agent builder to keep it.

If you leave the agent builder without saving, the change is dropped and the issue goes back to "Fix ready". The change affects Preview replies right away, before and after you save.

 

When you can't apply a fix

"Apply fix" is unavailable when the change would take a field past its character limit or the Playbook past its maximum number of scenarios. The issue explains why.

If you changed the same section after the fix was suggested, applying it replaces your changes. In that case, the issue also offers "Check if resolved".

 

Add missing or corrected Knowledge

When the cause is wrong, conflicting or missing Knowledge, "Fix ready" shows "Suggested knowledge" instead of a Playbook change.

  1. On the "Issues" tab, open the issue.
  2. Review or edit the text under "Knowledge to add". It needs at least 20 characters. If the check doesn't know the facts, the field starts empty, so write what the agent should say.
  3. Click "Apply fix".

To add a file, website or other source instead, click "Add a source instead". It opens the same options as adding a source in Knowledge.

When you apply the fix, the Knowledge is added right away as a custom answer. It doesn't wait for "Save". The issue shows while the Knowledge is processing and when it's ready for the agent to use. Click "View in knowledge base" to open it.

A fix can change both the Playbook and Knowledge. The Knowledge part is added right away. The Playbook part needs "Save", as described in "Apply a suggested fix".

Warning: Knowledge changes can't be undone from an issue. To change Knowledge added from an issue, edit or remove it in your Knowledge base.

 

 

Say what the reply should have said

An issue is in "Needs your input" when the check can't tell what the right reply was. This can happen when a Batch test question is rated Poor with no note.

  1. On the "Issues" tab, open an issue in "Needs your input".
  2. Under "Expected reply", write the reply you expected.
  3. Click "Check again".

The issue goes back to "Checking", then moves to a new state. If your answer couldn't be sent, it stays in the field so you can try again.

 

When the reply was correct

"No change needed" means the reply matches how the agent is set up. The issue quotes the Playbook or Knowledge that supports the reply.

On the "Issues" tab, open an issue in "No change needed", then:

  • Click "Mark as resolved" to move the issue to "Done". To reverse it, click "Undo" in the message that appears.
  • Click "Reopen issue" if you disagree. Write the expected reply, then click "Check again".

 

Dismiss an issue

"Dismiss issue" is available on issues in "Fix ready" or "Needs your input", and on any issue that's out of date. To close an issue in "No change needed", use "Mark as resolved" instead.

  1. On the "Issues" tab, open the issue.
  2. Click "Dismiss issue".

The issue moves to "Done" as "Dismissed". To reverse it, click "Undo" in the message that appears. After the message closes, you can't reopen the issue. A new flag with the same cause starts a new issue.

 

Check and undo a fix

Retest after a fix

"Retest" is available on an applied fix, whether or not you've saved it. It's also available on a flagged reply in "Chat".

  1. On the "Issues" tab, open an issue in "Applied".
  2. Click "Retest". "Preview" opens on the tab each reply came from.

How each reply is retested:

  • Chat replies replay in a new Chat conversation. If the issue has several, click "Next reply" to move to the next one.
  • Batch test replies run again as Batch test questions. A question you deleted from Batch test is added back. Chat replies are never added to Batch test.

While you retest, a bar at the top shows which issue you're retesting. Click "Back to issue" to return to it, or "Stop retesting" to end.

 

Review retest results

Each retest reply appears under its example in the issue, with when it ran and whether it ran on unsaved changes. Retesting doesn't change the issue's state and doesn't give a pass or fail result. Compare the replies and decide.

If the fix added Knowledge, retest waits until the Knowledge is ready.

Note: Batch test replies can't run again while another Batch test run is in progress. Retest them after the run ends.

 

Check whether an issue is resolved

"Check if resolved" replays the issue's examples on the agent as it is now and checks the replies again. It only runs when you click it.

  1. On the "Issues" tab, open the issue.
  2. Click "Check if resolved".

What happens next:

  • It checks the saved agent. Save first to include your unsaved changes.
  • Each example shows when it was checked and whether the problem is fixed.
  • If every reply is fine, the issue moves to "No change needed".
  • If the problem still happens, the check may suggest a new fix.
  • If the check doesn't finish, click "Retry".

 

Undo a Playbook fix

"Undo fix" appears on an applied issue whose fix changed the Playbook.

  1. On the "Issues" tab, open an issue in "Applied".
  2. Click "Undo fix".
  3. Review the confirmation, which lists the sections that will change.
  4. Click "Undo fix" to confirm, or "Cancel".
  5. Click "Save" in the agent builder to keep the undo.

 

What undo changes

  • Each section the fix changed (a Rules field, a scenario, or a free-text section) goes back to how it was just before the fix. Other sections aren't changed.
  • If you edited those sections after the fix, the confirmation warns you and lists the edits that will be lost. You can't choose which edits to keep.
  • Other applied fixes that changed the same sections afterward reopen, and the confirmation names them.
  • After you save, the issue goes back to "Fix ready", and you can apply the fix again later. If the fix was never saved, undoing it removes the unsaved change, and there's nothing to save.
  • If the fix also added Knowledge, only the Playbook part is undone. The Knowledge stays.

 

Frequently asked questions

Why doesn’t my flag show up as an issue right away?

Each flagged reply is checked in the background before it becomes an issue, so testing isn’t blocked. While the check runs, the reply and the issue show “Checking”. This can take a little while. You can keep testing in the meantime. When the check finishes, the issue moves to “Fix ready”, “Needs your input” or “No change needed”.

If the check doesn’t finish, the reply says so. Click “Retry” on the reply to send the flag again.

 

Why didn’t my Poor rating create an issue?

A Batch test question rated Poor becomes an issue only when you leave it. That means closing its panel, opening another question, switching to “Chat”, or leaving “Preview”. Until then, you can still change the rating or add a note.

Leave the question, then check its row: it shows the issue’s state once the issue is created. If the row says the flag failed, click “Retry”.

 

Why does the issue say Applied when I haven’t saved?

“Apply fix” adds the change to “Edit” right away, so you can test it in “Preview” before you decide to keep it. The issue shows as “Applied” from that point. The change isn’t saved to the agent yet.

  • To keep the change, click “Save” in the agent builder.
  • If you leave the agent builder without saving, the change is dropped and the issue goes back to “Fix ready”.

To learn more, refer to “Apply a suggested fix” section above.

 

Why did the issue suggest adding Knowledge I already have?

The check only looks at the Knowledge the agent retrieved for that reply. If the information is in your Knowledge base but the agent didn’t retrieve it, the check treats it as missing.

Before you apply the fix:

  1. Search your Knowledge base for the information.
  2. If it’s there, check whether the source is ready. To learn more, refer to Understand AI Knowledge source statuses and fix issues.
  3. If the source is ready, dismiss the issue instead of adding the same information again.

 

Can I undo Knowledge I added from an issue?

No. Knowledge from an issue is added as a custom answer as soon as you click “Apply fix”, and “Undo fix” only reverses Playbook changes. If the fix changed both, undo reverses the Playbook part and keeps the Knowledge.

To find the custom answer, click “View in knowledge base” on the issue. To stop the agent from using it, remove it from the agent’s Knowledge Base. To learn more, refer to Assign and manage sources in your SleekFlow AI Knowledge Base.

 

Why can’t I apply the suggestion anymore?

The issue is out of date. The Playbook or Knowledge changed after the issue was checked or after its fix was suggested, so the suggestion may no longer fit. The issue says what changed, and “Apply fix” is replaced by two options:

  • “Check if resolved” replays the examples on your saved agent. If the problem is gone, the issue moves to “No change needed”. If it still happens, the check may suggest a new fix.
  • “Dismiss issue” closes the issue if you no longer need it.

 

Why don’t I see replies reported from Inbox here?

The “Issues” tab only covers test replies flagged in “Preview”. Replies your team reports from AI conversations in Inbox go to Optimization under SleekFlow AI. To learn more, refer to Review and improve AI agent responses with Optimization.

 

Limitations

  • There's no list of issues across agents. Each agent has its own "Issues" tab.
  • You can only reopen a dismissed issue from the "Undo" in the message that appears when you dismiss it.
  • Retest shows the new replies without a pass or fail result.
  • If the needed Knowledge exists but the agent didn't retrieve it for the reply, the issue won't identify that as the cause.