swarm repositories / source
aboutsummaryrefslogtreecommitdiff
path: root/docs/qa-checklist.md
diff options
context:
space:
mode:
Diffstat (limited to 'docs/qa-checklist.md')
-rw-r--r--docs/qa-checklist.md119
1 files changed, 119 insertions, 0 deletions
diff --git a/docs/qa-checklist.md b/docs/qa-checklist.md
new file mode 100644
index 0000000..fea0719
--- /dev/null
+++ b/docs/qa-checklist.md
@@ -0,0 +1,119 @@
+# MCP QA Checklist
+
+This is the repo-local smoke checklist for `adequate-rust-mcp`.
+
+It is intentionally narrow:
+
+- run against this repository
+- omit destructive calls
+- accept on two axes only:
+ - no error
+ - model vibe check: yes/no
+
+There are no brittle output regexes. The point is to exercise the live MCP surface,
+inspect short previews, and answer "does this feel right?".
+
+## Run
+
+```bash
+python3 qa_checklist.py
+```
+
+The runner:
+
+- builds `adequate-rust-mcp`
+- launches the default host mode
+- initializes an MCP session
+- executes the checklist in order
+- prints one short preview per step for the vibe pass
+
+## Acceptance
+
+Each step is judged on:
+
+1. `error`: did the JSON-RPC call complete without a request/tool error?
+2. `vibe`: does the preview look legible, model-friendly, and semantically sufficient?
+
+The script enforces only `error`. The `vibe` call is human/model judgment.
+
+## Targets
+
+The checklist uses these repo-local files:
+
+- `crates/adequate-rust-mcp/src/main.rs`
+- `crates/adequate-rust-mcp/src/worker/mod.rs`
+
+Important positions in `main.rs`:
+
+- `parse_launch_mode` call: `27:16` in the typed one-indexed tools, `26:15` in raw LSP zero-indexed positions
+- `run_host`: `30:35` in the typed one-indexed tools, `29:34` in raw LSP zero-indexed positions
+- `mode`: `29:11` in the typed one-indexed tools, `28:10` in raw LSP zero-indexed positions
+- completion probe after `args_os().`: `27:54` one-indexed, `26:53` zero-indexed
+- signature-help probe inside `io::Error::new(...)`: `28:42` one-indexed, `27:41` zero-indexed
+
+## Checklist
+
+### Session Setup
+
+1. `initialize`
+ - params:
+ - `protocolVersion: "2025-11-25"`
+ - `capabilities: {}`
+ - `clientInfo: { name: "adequate-qa-checklist", version: "1.0.0" }`
+ - then send `notifications/initialized {}`
+2. `tools/list {}`
+3. `tools/call(name="advanced_lsp_request", arguments={ method: "textDocument/documentSymbol", params: { textDocument: { uri: main_uri } } })`
+
+### Warm-Up
+
+1. `tools/call(name="advanced_lsp_request", arguments={ method: "textDocument/prepareRename", params: { textDocument: { uri: main_uri }, position: { line: 28, character: 10 } } })`
+2. `tools/call(name="advanced_lsp_request", arguments={ method: "textDocument/completion", params: { textDocument: { uri: main_uri }, position: { line: 26, character: 53 } } })`
+3. `tools/call(name="advanced_lsp_request", arguments={ method: "textDocument/signatureHelp", params: { textDocument: { uri: main_uri }, position: { line: 27, character: 41 } } })`
+4. `tools/call(name="advanced_lsp_request", arguments={ method: "textDocument/documentHighlight", params: { textDocument: { uri: main_uri }, position: { line: 28, character: 10 } } })`
+5. `tools/call(name="health_snapshot", arguments={})`
+ - expected shape: short porcelain line, not JSON
+
+### Core Typed Tools
+
+1. `tools/call(name="hover", arguments={ file_path: "crates/adequate-rust-mcp/src/main.rs", line: 27, column: 16 })`
+2. `tools/call(name="definition", arguments={ file_path: "crates/adequate-rust-mcp/src/main.rs", line: 27, column: 16 })`
+3. `tools/call(name="references", arguments={ file_path: "crates/adequate-rust-mcp/src/main.rs", line: 27, column: 16 })`
+
+These typed read-only tools default to porcelain text; use `render: "json"` only when
+you explicitly want structured payloads for assertions or downstream processing.
+
+### Diagnostics Surface
+
+1. `tools/call(name="diagnostics", arguments={ file_path: "<absolute path to crates/adequate-rust-mcp/src/main.rs>" })`
+2. `tools/call(name="diagnostics", arguments={ file_path: "<absolute path to crates/adequate-rust-mcp/src/main.rs>", render: "json" })`
+3. `tools/call(name="diagnostics", arguments={ file_path: "<absolute path to crates/adequate-rust-mcp/src/main.rs>", mode: "full", render: "json" })`
+4. `tools/call(name="diagnostics", arguments={ file_paths: ["<absolute path to crates/adequate-rust-mcp/src/main.rs>", "<absolute path to crates/adequate-rust-mcp/src/worker/mod.rs>"], render: "json" })`
+5. `tools/call(name="diagnostics", arguments={ file_path: "<absolute path to crates/adequate-rust-mcp/src/main.rs>", path_style: "relative" })`
+6. `tools/call(name="diagnostics", arguments={ file_path: "<file:// URI for crates/adequate-rust-mcp/src/main.rs>" })`
+7. `tools/call(name="diagnostics", arguments={ file_path: "crates/adequate-rust-mcp/src/main.rs" })`
+
+### Clippy Surface
+
+1. `tools/call(name="clippy_diagnostics", arguments={ file_path: "<absolute path to crates/adequate-rust-mcp/src/main.rs>" })`
+2. `tools/call(name="clippy_diagnostics", arguments={ file_path: "<absolute path to crates/adequate-rust-mcp/src/main.rs>", render: "json" })`
+3. `tools/call(name="clippy_diagnostics", arguments={ file_path: "<absolute path to crates/adequate-rust-mcp/src/main.rs>", mode: "full", render: "json" })`
+4. `tools/call(name="clippy_diagnostics", arguments={ file_paths: ["<absolute path to crates/adequate-rust-mcp/src/main.rs>", "<absolute path to crates/adequate-rust-mcp/src/worker/mod.rs>"], render: "json" })`
+
+### Advanced LSP Safe Calls
+
+All raw LSP positions here are zero-indexed.
+
+1. `tools/call(name="advanced_lsp_request", arguments={ method: "workspace/symbol", params: { query: "LaunchMode" } })`
+2. `tools/call(name="advanced_lsp_request", arguments={ method: "textDocument/definition", params: "{\"textDocument\":{\"uri\":\"main_uri\"},\"position\":{\"line\":29,\"character\":34}}" })`
+3. `tools/call(name="advanced_lsp_request", arguments={ method: "textDocument/foldingRange", params: { textDocument: { uri: main_uri } } })`
+4. `tools/call(name="advanced_lsp_request", arguments={ method: "textDocument/selectionRange", params: { textDocument: { uri: main_uri }, positions: [{ line: 29, character: 34 }] } })`
+5. `tools/call(name="advanced_lsp_request", arguments={ method: "textDocument/inlayHint", params: { textDocument: { uri: main_uri }, range: { start: { line: 0, character: 0 }, end: { line: 40, character: 0 } } } })`
+6. `tools/call(name="advanced_lsp_request", arguments={ method: "textDocument/diagnostic", params: { textDocument: { uri: main_uri } } })`
+7. `tools/call(name="advanced_lsp_request", arguments={ method: "textDocument/semanticTokens/range", params: { textDocument: { uri: main_uri }, range: { start: { line: 26, character: 0 }, end: { line: 31, character: 80 } } } })`
+
+## Explicit Omissions
+
+- `rename_symbol`
+- `fix_everything`
+- `advanced_lsp_request` methods with side effects such as `workspace/executeCommand`
+- formatting/code-action flows that could rewrite files