thinkingsense admin / LLM settings
A
Admin

LLM settings

Which model plans/generates SQL, and which model checks its work β€” set globally, overridable per group.

Global default
applies to any group without its own override
PRIMARY (generator)
JUDGE (self-correction)
Per-group overrides
3 groups configured
GroupPrimaryJudgeReason
Business Process Investigationclaude-sonnet-5qwen2.5-7b-instructdefault β€” no override
Retail Operationsclaude-sonnet-5claude-sonnet-5high query volume β€” judge upgraded after accuracy dipped on promo-pricing questions
Risk Managementqwen2.5-7b-instructqwen2.5-7b-instructlocal-only β€” position/compliance data must not leave the network

Data sources

Every connected system, in its native protocol. Filter by the group it's assigned to.

Add a data source
credentials resolved at connect time β€” never stored as plaintext when backed by Vault or CyberArk
OMNIGATE_BACKENDS entry (preview)
exactly what would be written to config β€” nothing hidden

        

Context builder

Upload a document once and it becomes part of a group's permanent knowledge base β€” every future question can draw on it. Different from a chat attachment, which only applies to that one message.

Uploaded documents
FileGroupUploadedStatus

PDF Extract

Turns a folder of PDFs into live, queryable data β€” PDF β†’ AI Extract β†’ Ground β†’ JOIN β†’ Answer. Backed by this project's own PdfFieldExtractor (OCR fallback via PdfOcrEngine when a page has no text layer) β€” no separate ETL pipeline, no pre-processing step.

How grounding works
the part that makes this safe to trust
1
An LLM proposes field values from the PDF's own text (or OCR text, for a scanned page with no text layer).
2
Every proposed value is independently checked against the actual document text β€” not just trusted because the model said so.
3
A value that genuinely appears in the source is marked GROUNDED βœ“. A value that doesn't β€” a rounding, a reformat, a model "correction" β€” is flagged, never silently used.
4
Only grounded values are safe to JOIN against the system of record β€” an ungrounded field blocks the join for that row rather than risk a false mismatch.
Test schema across documents
answers "does this field list actually work here" before it ships in a query β€” not after

A field name typed into PDF_EXTRACT(...) is a guess about a folder's vocabulary until it's tested against real documents. This samples a batch, runs extraction + grounding on every one, and reports two separate numbers per field β€” claim rate (does the model find this field at all?) and grounding rate (when it does, is the value actually on the page?) β€” clustered by which fields each document actually has, so one bad vendor template can't hide inside a blended average.

As SQL, once the schema above tests clean
PDF_EXTRACT() is a real table function β€” no separate extraction step to run first
SELECT x.po_number, x.amount_due AS pdf_amount, x.grounded, po.total_amount AS sap_amount
FROM PDF_EXTRACT('s3://po-documents/', schema => 'po_number,vendor,amount_due') x
JOIN sap_orders.purchase_orders po ON po.po_number = x.po_number
WHERE x.amount_due <> po.total_amount

Groups

A group is the access + LLM boundary a business user actually sees β€” one place to ask, not a list of systems to know about.

Users & access

Which group(s) each user can ask questions within. A user sees only the sources their group grants β€” nothing else.

All users
6 users
UserRoleGroups grantedRow/column policy
Grant access
to group
User attributes
the values RLS policies actually reference β€” USER_ATTRIBUTE('desk') reads from here
UserAttributeValue
Row & column-level security
enforced by rewriting SQL before it reaches the backend β€” not filtered after the fact
KindTablePolicyApplies to

Skills

Reusable, admin-authored instructions for a recurring analysis workflow. Business users invoke one by typing /skill-name in Ask, or ThinkingSense loads it automatically when a question matches its trigger.

Defined skills
Add a skill

Prompt templates

Reusable question phrasings a business user picks and fills in themselves β€” lighter-weight than a Skill (no matching logic, no instructions, just a starting shape for a common question). Values are never auto-filled; the user always fills in the {placeholder}s.

Defined templates
Add a template

Knowledge signals

Knowledge surfaced from real conversations, not written from scratch β€” a business question's answer already states a fact worth keeping; this is where an admin reviews it before it becomes trusted, reusable knowledge.

Suggested from chat
Accepted knowledge

Slack

Turns a Slack channel into a shared place for data questions β€” link a channel to a group's data area, then anyone in that channel can ask ThinkingSense by mentioning it, with the answer posted as a thread so the main channel stays readable.

Linked channels
ChannelData area (group)Linked byStatus
Try it
illustrative Slack UI β€” not a real Slack workspace

How it works
1
An admin links a Slack channel to a group's data area here, so the right dataset is used for every question asked in that channel.
2
A channel member asks by mentioning ThinkingSense in a message β€” the same NL2SQL and context-check pipeline as Ask runs underneath.
3
The answer is posted as a threaded reply, not in the main channel, so the channel stays clean while anyone can still ask a follow-up.
4
Access control is enforced per asker, same as Ask β€” someone without access to a source never sees an answer built from it, in Slack or anywhere else.

Question activity

Every real natural-language question asked of ThinkingSense, from either entry point β€” the built-in Ask app or Claude via MCP β€” lands in the same log. This is the raw signal Ontology review and Rollup suggestions are both scanned from.

TimeQuestionSourceUserGroupStatus

Reviewed queries

Curated NL ↔ SQL pairs a business question can always trust β€” the admin-facing counterpart to this project's own QuestionBankGenerator/QuestionBankRetriever. Marking a pair reviewed means the exact wording and its SQL are vetted, not just plausible.

All questions
QuestionGroupSQLStatus

Negative feedback

Every πŸ‘Ž tagged in Ask lands here β€” the aggregated view of what's getting flagged, separate from any single answer's feedback thread.

Flagged answers
TimeQuestionGroupTag

Domain health

A composite score per group β€” illustrative weighting, but every input is a real, live count from this console (ontology backlog, open negative feedback, rollup coverage, skills defined), not a fabricated number.

Ontology review

Relationships inferred from real query history, value overlap, and business-user feedback β€” nothing gets used by NL2SQL until reviewed here.

Pending & recent relationships
4 total
RelationshipSourceConfidence
salesforce.opportunity_id ↔ sap.order_id Query-log mined (17 occurrences) Auto-accepted β€”
salesforce.commercial_exceptions.account_id ↔ sap.credit_holds.account_id Data-driven profiling (value overlap) Needs review
slack.exception_threads.order_ref ↔ sap.orders.order_id Query-log mined (5 occurrences) Conflicting type
novalogs.inventory_events.sku ↔ novamart.products.sku Retail Operations β€” β€œinventory total didn’t match the product I asked about” Needs review
How business feedback becomes an ontology fix
1
Business user flags a bad answer
πŸ‘Ž on a specific answer in Ask, optionally tagged ("wrong join", "missing data", "outdated").
2
Feedback attaches to the exact query + join keys used
Not just "this answer was bad" β€” which relationship the generated SQL actually joined on.
3
Repeated flags on the same join surface here
3+ independent flags on the same relationship is a real signal, not one user's opinion β€” ranked above single-occurrence candidates.
4
Admin fixes the relationship, not just the one answer
Every future question using that join benefits β€” not a one-off patch.

Rollups

Pre-aggregated tables that transparently speed up matching queries β€” suggested automatically from real repeated NL2SQL history across every group.

Suggested rollups

Structural matching only β€” same table + group-by + aggregations after redacting literals.

Not scanned yet.
Edit rollup definitions
raw YAML β€” same shape PUT /api/config/rollups accepts

Federation plans

Real federated-query plan history β€” measured per-backend timing, not just the planner's own pre-execution estimate.

Captured
2
Avg elapsed
540 ms
Failed
0
#GroupBackendsSQLElapsedRowsStatus
#1 Business Process sap_orderssalesforceslack SELECT h.reason, COUNT(*)… 612 ms4OKβ–Έ
#2 Retail novamartnovalogs SELECT ie.sku, ie.on_hand… 468 ms5OKβ–Έ

SQL Playground

Manual SQL exploration against any registered backend, independent of NL2SQL β€” for validating joins, checking row counts, and debugging what a generated query would actually return.

Query

Knowledge playground

An isolated workspace for drafting, refining, and validating a query before it becomes a trusted, reusable answer β€” separate from live chat, so iterating here never clutters a real conversation.

1 Β· Ask

Evaluation

Prompt sets with expected SQL, run against NL2SQL to measure accuracy over time β€” the same shape as this repo's own Nl2SqlEvalTest suite, exposed here for ad hoc admin runs.

Evaluation sets
3 sets
NameGroupPromptsLast accuracy
bp-core-questionsBusiness Process1894%
retail-inventory-suiteRetail1291%
risk-limit-checksRisk986%
Runs
newest first
SetWhenPassedAccuracy
bp-core-questions2 days ago17 / 1894%
retail-inventory-suite5 days ago11 / 1291%
risk-limit-checks1 week ago7 / 986%

Agents

A digital worker configured once and left running β€” retrieves data, runs analysis, and delivers results on a schedule instead of waiting to be asked.

Running agents
NameTriggerLast runDeliveryStatus
Add an agent
Reviewed runs β€” Daily credit-hold digest
approve a run to mark that exact output trusted; unapproved runs still delivered, just not vouched for
RunResultStatus
Daily credit-hold digest β€” node graph
Visual Mode Β· read-only preview
⏱
Schedule
Every day at 7:00am
β†’
πŸ—„
Data
Run the credit-hold-audit SQL against sap_orders + salesforce
β†’
πŸ“
Summary
Turn the result rows into a short narrative
β†’
πŸ“§
Email
Deliver to Finance Ops, with the result table attached
Run with parameters
one-off preview β€” doesn't touch the schedule or delivery target

Advanced

Per-group display, scheduling, and chat-behavior settings.

Starter questions
shown to new users of this group
Fiscal year
Starts
Chat customization
system instructions appended to every answer in this group
Row-level security

Policies are configured in Users & access.

Limited context
a separate safety knob from RLS β€” this restricts what the AI can even see, not which rows come back

Developers

Embed ThinkingSense in your own product, and manage programmatic access.

API keys
NameKeyCreatedLast used
Embed
iframe or the React SDK β€” same governed pipeline either way
<iframe
  src="https://app.thinkingsense.ai/embed/chat?group=retail-operations&token=SESSION_JWT"
  style="width:100%;height:640px;border:0">
</iframe>
SSO & provisioning
SSO groupThinkingSense group
okta-finance-opsBusiness Process Investigation
okta-retail-analyticsRetail Operations
okta-risk-deskRisk Management

SCIM base URL: https://app.thinkingsense.ai/scim/v2 β€” new members provisioned into their mapped group automatically on next login.

Help requests

A user who's stuck can flag an admin directly from Ask β€” separate from πŸ‘Ž feedback on a specific answer, this is "I need a person," not "this answer was wrong."

Open requests
TimeUserGroupMessage

Organizations

ThinkingSense is multi-tenant β€” this admin console is scoped to whichever organization is active, switchable from the topbar.

All organizations
3 orgs
OrganizationPlanSeatsCreatedStatus

Ask

Same one place to ask, whichever group you're in β€” try all three teams.

Signed in as Sam Chen Β· Finance Ops β€” group: Business Process Investigation

Viewing as

This demo answers the pre-loaded scenarios above β€” try one, or type close to it. Type / for skills, Tab to accept a suggestion, or press ⌘K / Ctrl K anywhere to jump around.

Dashboards

Answers you've pinned from Ask, laid out as a living page. Viewed live in the app, every widget re-runs against the same governed pipeline β€” but a shared link serves a pre-computed snapshot instead, so opening it never re-queries your sources.

Instant filter
Ask this dashboard
scoped to the instant filter above

Version history

Ask an admin for help

AI workstream

Share this dashboard

2 Β· Sensitivity & risk evaluation

Audit trail