Smart Listen and Smart Reply: let the AI pick the path

Without Smart Listen, a Listen node matches the caller’s words against your keywords. With it, an AI reads the words and chooses the path it judges the caller meant, so “born in 1940” is read as an age, not as “40”. Your keywords become examples, not rules, and the AI sees up to 12 keywords for each path.

Smart Reply is a variant of the Play Audio node that plays a recorded answer to an off-script question. Smart Listen can answer such questions by itself, so use Smart Reply when the answer should lead to its own paths, or after an ordinary keyword Listen node.

What happens on one turn

1. The caller answers, and your Recognition Engine (ASR) turns the speech into text.
2. The AI gets the question the bot just asked, your paths with their keywords and hints, About this call and the caller’s words, but no lead details. It picks a path and, if they fit, a recorded reply, a short response, a rebuttal and the answer’s value, such as an age.
3. The system checks the pick: a number the caller said must fit the path’s number rule, and the AI is told not to choose a transfer or qualified path unless the caller clearly qualifies.
4. The bot plays any short response and reply, then follows the path or Not understood. If the AI does not answer in time, or nothing was heard, keywords route the turn.

Smart Listen is not self-serve: the VoiceOI team switches it on for an organization. Until then its controls do not appear and every Listen node routes on keywords.

The one on the LLM provider tile of Integrations, which your AI Voice Agent uses too. Calls use an engine the VoiceOI team deployed unless someone saved that tile (In effect now shows which), and saving your own key, model or endpoint changes it for every AI script in your organization, so ask the team first. See the AI model and the LLM provider tile; a provider of your own charges its own rates. With no engine, the builder warns “Your organization has no AI engine set up. Add it under Integrations > AI Voice Engine; until then this node routes on keywords and plays no replies” (that tile is now called LLM provider). Ask the team to set one up.

Java, an optional second engine

The Java tile on Integrations appears only when Smart Listen is on. Paste your Tavora API key, press Connect and leave Use Java for Smart Listen on, and Smart Listen decisions run on your own Tavora key. When you connect your own key, Tavora bills that key, not VoiceOI; ask the team how to get one. While Java is on for your organization, whether you switched it on or the team set it, callers’ words are also sent to Tavora. When Java is unsure, slow or down, the turn goes to your LLM provider or to keywords, and with the switch off turns go to your LLM provider unless the team has set otherwise. For the statuses, see Integrations and carriers.

1. 1

   Get Smart Listen switched on and check the engine

   Both

   Ask the team to switch it on (and reply libraries, if you want them), then press Test connection on the LLM provider tile.

2. 2

   Switch Smart Listen on for a Listen node

   You

   Select a Listen node and switch on Smart Listen. That sets Replies per call to 2, turns on Answer the caller’s questions and makes Any Other Input Not understood.

3. 3

   Tell the AI what the question is for

   You

   Fill in About this call (optional, read by the AI) with a sentence about the offer and who may qualify, and give nearby nodes a For the AI: what this node says or does line (up to 200 characters). Without a transcription or that line, the AI knows a node by name only, and the builder warns you which.

4. 4

   Set up the paths

   You

   Keep keywords on each path as examples, and add a hint under What this path means (optional) where the name alone is unclear. Add AI path makes a path with no keywords (an AI only badge in the panel): when the AI cannot answer, keywords route the turn and that path is skipped.

   To capture the answer as a value, set Capture the answer as. Connect Not understood to where the call should go when nothing fits; the builder warns that with nothing connected those calls end.

5. 5

   Check the voices

   Both

   Link each voice to the voice roster under Voices > Manage. On a script with no voices defined, pick a Speaker instead. A call in an unlinked voice, or with Speaker set to Nobody, plays no reply, not even one that ends the call. The team adds a voice the roster lacks. See Voices and audio takes.

6. 6

   Save and test

   You

   Press Save Flow, then follow Testing. On a live campaign, start with shadow mode.

Number rules

The system checks number rules, not the AI. If the AI chooses a path with a rule such as ages 45 to 85, and the caller said a number outside it or the AI read a different number from the one the caller said, the path is dropped. “Born in 1940” is read as an age above 85, so the call cannot take that path. With no number in the answer, such as “yes, I am”, the AI’s reading of the meaning decides.

The range check can take a caller off a transfer or qualified path, and it does not move a caller onto one by itself. The exception is a question that also uses Listen for a number (the builder warns you when its ranges differ): if the AI leaves the caller on Not understood or on a path whose range does not fit, the number the caller said can send the call to the range it fits, even a qualifying one. The Simulator’s Rule line then starts with “numbers:”; the test box does not apply this part.

What happens after a reply

With Ask again after answering off, the call does not repeat the question after a reply or a rebuttal. It continues on an AI path that carries the call forward, or to Not understood if that line leads to another question; otherwise the bot asks again. It never picks a path that ends the call, transfers the caller or records a sale or do-not-call.

Shadow mode lets the AI decide and write its choice down while keywords keep routing the call. The node shows a SHADOW badge instead of SMART.

• Keywords route the call exactly as without Smart Listen. No reply, short response or rebuttal plays, and the answer is not captured as a value.
• The AI engine is still asked each turn, so the turns still use your provider. The test box shows the AI’s pick, not what the call will do.

To see what the AI would have done, use the Simulator’s “Smart Listen (shadow)” line. On real calls, press Show System Nodes in Call Logs (open to the Admin and SubManager roles): each shadow turn is a Smart Listen entry reading “Shadow:”, next to the Logic Branch entry for where keywords sent the call. No customer report compares them, so read a sample of calls, then switch Shadow mode off. See Monitoring and reports.

The Smart Reply node is in Add Node only when Smart Listen is on for your account. It is a Play Audio node with no recording of its own: it takes the caller’s last words, finds the recorded reply that fits and plays it in the call’s voice, then continues.

1. 1

   Connect a Listen node to it

   You

   Draw the Listen node’s Not understood (Any Other Input) line to the Smart Reply node, right after it. One or two Play Audio nodes in between are fine. Reached from anywhere else, or later than that, it plays nothing.

2. 2

   Choose what happens after the reply

   You

   Under After the reply, choose Ask the question again or AI picks one of this node’s paths. For the second, add paths with Add path or Add AI path and a When to take this path (optional) hint for each.

3. 3

   Connect Can’t Answer

   You

   Connect Can’t Answer, usually to where Any Other Input went before. A call the node cannot answer takes this line. If it is empty after a Listen node, the bot asks the question again, but the builder warns that an empty line ends those calls, so connect it and decide where the call goes.

The node plays nothing, and the call follows Can’t Answer, when it was not reached from a Listen node, the caller said nothing, the Listen node is in shadow mode, the voice is not linked or has no fitting reply, the Replies per call limit was reached, or the AI could not be reached. Call Logs shows “No reply this time”.

The recorded answers come from a library the VoiceOI team curates and records; you cannot edit it or record your own replies in the dashboard. The AI is offered only entries the call’s voice has recorded, and they play in that voice. Ask the team which topics your voices have recorded.

There are four kinds. A reply answers a side question, such as who is calling or where the number came from. A short response is a few words said first when the caller shares something, at most three a call. A rebuttal is a reply library’s answer to an objection, and each rebuttal is said at most once a call. A reply that ends the call, such as a request to be removed or to be called back later, plays, applies its disposition (the outcome code, such as DNC) and hangs up. Answer the caller’s questions and Replies per call do not switch it off, but it works only when the call’s voice is linked to the voice roster and has recorded that reply.

A script uses the universal library plus, with the second switch, one named reply library, picked from the builder header under Replies come from (Universal only or Universal + <name>). Paused, or a bot server on an older release, means the universal library only.

Rebuttals

A rebuttal answers an objection such as “not interested”. Rebuttals per call appears only when rebuttals are on for you. A rebuttal is not used when the caller is being transferred, has qualified or asks to be removed.

It is a setting on a Smart Listen node with three values: Off, Test only and On. When nothing in your replies fits what the caller said, an AI service that VoiceOI runs (not your LLM provider) writes a short answer and a generated voice says it in the call’s voice. If it is not ready in time, the call goes on as before. When it is slow, the bot may first say a short word such as “Sure.” in the call’s voice.

• The answer is spoken as soon as it is ready, and nobody reviews it first. VoiceOI staff can review answers afterwards and reject one; a rejected answer is not said again.
• The system drops an answer that contains a number, a price or promise word, a link, advice, a callback promise or a claim to be a person. This is a word check, not a review.
• The AI works only from your About this call text and any facts the VoiceOI team stored for your reply library, so keep About this call accurate and free of anything you do not want a caller to hear.
• A live answer counts toward Replies per call and needs at least three words from the caller. An account with a very low balance gets no new generated answers, and the speech is charged to your balance (a repeated question is free).
• It runs on the VICIdial® (SIP) engine only (see Which setup do I have?), needs a voice VoiceOI can generate lines for, and is off when Answer the caller’s questions is off.

Test only changes nothing a caller hears. VoiceOI staff keep what the AI would have said, and you will not see it in the Simulator, the test box or Call Logs. With On, the test box and the Simulator show what a call would say, and Show System Nodes in Call Logs shows an Event entry reading “Answered live:” and the sentence. The test box only previews: it stores and charges nothing. The Simulator behaves like a call: with On it is charged to your balance, and with On or Test only the words you type are kept for staff review. On a built-in dialer account both show an answer that real calls never say, so do not rely on them there.

Smart Listen with AI (in the builder header, or the Scripts row menu for the Admin and Manager roles) sets up Smart Listen on a script’s questions in a copy, so your original does not change, and its work is charged to your balance. The steps are in ways to build a script. When every question already has Smart Listen, the button becomes Smart Join.

1. 1

   Use the test box on the node

   You

   In the Smart Listen section, type a caller’s answer under Test: what does the caller say?, for example “I was born in 1940”, and press Route.

   The result shows Path, Rule, Reply, First (the short response), Rebuttal, Live, Value and Library. It tests the settings as they are on screen, but reads where each path leads from the last saved script, so save first. On a Shadow mode node it shows the AI’s pick, but a live call is routed by keywords.

2. 2

   Run the Simulator

   You

   Start it as in How do I test a script? and answer in your own words, including side questions. Under each bot reply a “Smart Listen” line shows the Path, Answer, Reply, Rebuttal, First and Rule. It cannot show mishearing. If a turn says “Simulator limit reached for now; keywords routed this turn”, wait a minute and try again. Answer unknown questions live set to On charges a Simulator turn to your balance, and On or Test only keeps your typed words for staff.

3. 3

   Call a phone you control

   You

   Speak the answers a real caller would give, with some noise and side questions. Then read the Smart Listen, Smart Reply, Short response and Rebuttal entries under Show System Nodes in Call Logs.

• The AI is a language model and can choose the wrong path, miss a side question or play the wrong reply. Test with real phrases, run Shadow mode first, and read a sample of calls after launch. When the engine does not answer in time, keywords route the turn and Call Logs shows “AI unavailable, keywords routed this turn”.
• It reads text from speech recognition, so a misheard answer is routed on what was heard, and callers are understood in English only today. It works on Listen nodes of Classic Call Flow scripts, not in the AI Voice Agent editor.