Every call your team handles gets an automatic quality score. dialnote's AI analyzes the transcription after each call ends and assigns a 1–10 rating based on how well the conversation went. You don't need to set up scorecards or rubrics — it just works once transcription and AI summaries are turned on.
How Call Scoring Works#
After a call ends and the transcription is ready, dialnote's AI reads through the full conversation and evaluates it. The same Gemini model that writes your AI summaries does the scoring, so there's nothing extra to set up. It looks at whether the caller's issue got resolved, how the conversation flowed, and the overall caller sentiment.
Each scored call gets these data points:
- Call score — A rating from 1 to 10 (1 = very poor, 10 = excellent)
- Call successful — Whether the call achieved its purpose (yes/no)
- Sentiment — The caller's mood: positive, neutral, or negative
- Evaluation summary — A brief 1–2 sentence description of how the call went
- Confidence — How confident the AI is in its assessment (0.0 to 1.0)
The scoring follows these guidelines:
| Score Range | What It Means |
|---|---|
| 8–10 | Issue fully resolved, caller satisfied |
| 5–7 | Partial resolution or adequate handling |
| 1–4 | Unresolved, caller dissatisfied, or call failed |
Pro tip
Call scoring runs automatically on human-handled calls. AI agent calls already get evaluation data from the voice AI provider, so they're scored separately in the AI Agents analytics dashboard.
Enabling Call Scoring#
Call scoring is tied to the AI call tagging setting on each phone number. If AI call tagging is turned on, call scoring runs automatically too.
To turn it on:
- Go to Settings → Phone Numbers and select a number
- Open the AI & Intelligence section
- Turn on AI transcription
- Turn on AI call tagging — the call score, sentiment, and evaluation summary are produced by the tagging pass, so this is the toggle that switches on scoring
- Calls on that number will now get scored automatically
Minimum conversation length
The score comes from the same transcription pass that generates call tags, so it runs on any call that has transcribed text — there's no minimum word count. Only calls with no transcribed conversation at all (silent calls, quick hangups, wrong numbers) get skipped. Note that the separate AI summary does require at least 50 words, so a very short call can still get a score without a full summary.
Where Scores Appear#
Call scores are stored alongside each call's summary data. You can see them in:
- Inbox — Call details show the sentiment and a Resolved / Not resolved outcome chip alongside the transcript and AI summary. The numeric 1-10 score and evaluation summary aren't shown here — read them from Analytics or through the API.
- Analytics — The reports dashboard aggregates scoring data so you can spot trends across your team. The disposition codes report rolls these scores into categories like billing, sales, and technical support.
Scores aren't in webhook payloads
The call score, evaluation summary, and confidence aren't included in the standard call.completed webhook payload. If you need them in another tool, read them from a call's record through the API.
AI vs. Human Call Scoring#
dialnote scores AI agent calls and human-handled calls separately:
- AI agent calls get scored by the voice AI provider (Retell) with containment rate, outcome tracking, and sentiment analysis. You'll find these in Reports → AI Agents.
- Human-handled calls get scored by dialnote's built-in evaluation engine using the same AI that generates summaries. These appear in your standard call analytics.
Both types feed into your overall analytics, giving you a complete picture of call quality across your entire phone system.
What's Next#
- Call transcription — turn on the transcription that scoring depends on
- Sentiment analysis — see how the positive/neutral/negative read is calculated
- Disposition codes — group scored calls into reporting categories