LLM Usage Dashboard

In LLM Usage Dashboard, you can monitor token usage on LLMs.

PREREQUISITES

  • At least one active Knowbler client.

  • At least one active LLM integration.

  • The dashboard works for Salesforce and Zendesk. An LLM Usage dashboard for Microsoft Dynamics is under development.

Access LLM Usage Dashboard

  1. Expand Large Language Model and open Usage Dashboard.

  2. You'll either see "LLM Integration is not activated." or a report and two filters.

  3. The filters are Service Desk Management Tool and Date Range. You can use the filters to generate a report.

Report Total Requests

The first report in the dashboard is Total Requests. It lists the total calls made to an LLM.

The report is collapsed in the default state. Upon expanding it, you can see five columns:

  • Serial Number is the row number.

  • Requester Email: The email ID of the support rep who clicks Create an Article in Knowbler when Knowbler is connected with an LLM. The column is searchable.

  • Case ID. The Salesforce or Zendesk ID of the case. This column is searchable.

  • Case Subject. The title used in Salesforce or Zendesk. This column is searchable.

  • Requested Time. The time when the support rep clicks on Create an Article in Knowbler.

In the default state, 10 rows are listed. The data can be emailed and downloaded.

Four Tiles

The second part consists of four (or five tiles).

  • Input Tokens counts the total number of tokens shared from Knowbler to an LLM.

  • Generated Token counts the total number of tokens shared from an LLM to Knowbler.

  • Average LLM Response Time is the time spent on generating responses divided by the total requests.

  • Error Rate is the ratio of number of requests with errors divided by the total number of requests.

NOTE.

The tile Integration can be seen only when multiple LLM integrations are in use.

Last updatedWednesday, April 10, 2024

Or, send us your review at help-feedback@searchunify.com