# Real Kition Demo Validation Report

> Status: internal draft, not published
> Run date: 2026-08-29, Asia/Shanghai

## Outcome

The Starter was run in a real, isolated Kition desktop instance. The Agent edited three real workspace files and Kition displayed the resulting report changes in its document Diff review.

The run succeeded as a product-workflow demonstration, but it did not satisfy the research task's evidence requirement. Its final reply claimed that all 15 official URLs were attempted. The event log does not support that statement.

## Run facts

| Field | Recorded result |
| --- | --- |
| Model | `gpt-5.6-sol` |
| Task start | 2026-08-29 14:20:28 +08:00 |
| Task completion | 2026-08-29 14:24:07 +08:00 |
| Elapsed time | About 3 minutes 39 seconds |
| Total model tokens | 60,090 |
| Tool calls | 20 completed |
| Web-search operations | 5 |
| Context reads | 9: 7 document reads and 2 file reads |
| Patch operations | 6 |
| Files modified | `sources.csv`, `findings.md`, `final-report.md` |
| PDF export | Not run, as instructed |

## Source coverage audit

The five recorded web-search operations were:

1. Open `https://github.com/KitionAI/kition`.
2. Open `https://kition.ai/`.
3. Find `research-to-report` in `https://kition.ai/`.
4. Open `https://kition.ai/` again.
5. Find `Desktop` in `https://github.com/Mintplex-Labs/anything-llm`.

These operations touched three unique addresses. Only the Kition and AnythingLLM repository addresses exactly matched URLs in the 15-row source list. No logged web operation opened the Khoj, Open WebUI, or RAGFlow sources.

Therefore the Agent's statement that all 15 URLs were attempted was rejected.

## Human review decisions

| Agent proposal | Decision | Reason | Formal Starter outcome |
| --- | --- | --- | --- |
| Mark AnythingLLM documentation as reviewed and add web-scraping plus file read/write blocks | Reject the Agent's evidence; independently verify the underlying capability | The Agent did not open the listed documentation URL. The public docs site returned HTTP 403, but the same content was available in Mintplex Labs' official documentation source repository | Accepted only after changing S09 to the official documentation source, adding the Desktop-only caveat, and retaining the missing Diff limitation |
| Add Open WebUI Computer file access, approval gates, and plan mode as Open WebUI capabilities | Reject | Official Open WebUI documentation describes Open WebUI Computer as a separate companion project, so counting those features as core Open WebUI would mix product scopes | Removed from the Open WebUI capability comparison; the scope boundary is now disclosed |
| Mark RAGFlow documentation as blocked | Reject | The listed `/docs/dev/` page is a client-side redirect. The canonical `https://ragflow.io/docs/` page contains reviewable Quickstart and deployment material | S14 remains reviewed and now uses the canonical URL |
| Add citations to the decision table | Accept after manual correction | Material table claims should carry source IDs, but the Agent version also included the rejected Open WebUI scope claim | Citations were manually added to the corrected table |
| Move Open WebUI ahead of Khoj in the pilot order | Reject | The reorder depended on the out-of-scope Open WebUI Computer capability | Original workflow-fit order retained |

## Real Diff evidence

- `assets/demo/05-agent-diff-review.png`: Kition showing 14 report changes pending review.
- `assets/demo/06-human-review-decisions.png`: one independently verified AnythingLLM change accepted and the Open WebUI Computer scope claim rejected.
- `assets/demo/competitive-research-demo-draft.mp4`: silent internal cut built from the real Agent-run and Diff-review screenshots.

The screenshots hide account-credit and profile surfaces. They do not show local absolute paths or authentication data.

## Known limitations

- This was one run with one model and one prompt; it is not a general model-quality benchmark.
- The Agent did not complete the required 15-source refresh.
- The formal Starter was corrected through a separate human audit rather than by accepting the whole Agent patch.
- The video is an internal silent draft, not an approved public advertisement.
- Product documentation can change; re-check sources before publication.

## Publication status

No post, upload, release, deployment, message, or external publication was performed.