Static reading edition · s07-05

Test the package, then the workflow

Plugins26 min estimatedCodex app · CLI; plugins unavailable in IDE

Prepare a repeatable test matrix that separates structural validity, activation, and output quality.

Understand the approach

A plugin can pass JSON validation and still fail its user journey. Test the package in layers: manifest and paths, skill discovery, setup and connection if any, activation, actual output, and failure behavior. The source demonstrates Claude's plugin-dir flag. Codex's current package guide uses supported local marketplace workflows for local testing; do not claim that another product's launch flag loads a Codex plugin.

Begin with the graduated local package: a focused skill, a deterministic fixture checker, a real stdio MCP server, and optional informational hooks. Offline checks can prove file structure, actual protocol exchange, controlled results, and failure behavior. They cannot prove Codex loaded or executed the plugin. An optional supported-client test should follow the documented local marketplace workflow, start a new chat or session as required, and record discovery, invocation, tool output, hook trust, and cleanup separately.

Include positive, negative, and incomplete inputs. A complete README should not receive invented defects; a missing file should produce a clear gap; a document containing injected instructions should remain data. If a future plugin includes tools, test disconnected and denied-access cases separately from empty results. If it includes hooks, inspect trust and supported events. Never equate successful installation with verified behavior.

Offline structural checks, executable protocol tests, and real client discovery support different claims. Combine actual output with the acceptance rubric. A changed package returns to the affected tests before release readiness is assessed again.
Open the full diagram

From measured local behavior to client acceptance

The actual workflow-plugin package includes a complete skill, fixed fixture checker, real stdio server, and optional informational hooks. Run its 19 checks and compare the measured demo with EXPECTED.md. Follow labs/workflow-plugin/README.md only if you choose a real supported desktop scratch-client trial. Record package discovery, tool discovery, skill invocation, findings, hook review, actual event delivery, and cleanup separately. Offline passes provide strong local integration evidence but cannot fill those client rows.

The exercise has a real reference and failure gate

The baseline suite now preserves the original three fixture outcomes without rejecting a legitimate fourth bounded ID. The extension grader checks actual candidate files, schema/version synchronization, tools/list and tools/call through the copied server. Its complete reference is executable in a second fresh workspace, so a learner can compare artifacts rather than only reading a rubric.

About your environment

Verified October 21, 2026. Native metadata lifecycle tested on Windows with Codex CLI 0.163.0-alpha.11 in disposable child-only CODEX_HOME: starter 0.2.0 and extension 0.3.0 install/list/remove succeeded. Authenticated skill invocation, MCP inside Codex, hook trust/delivery, other platforms and publication were not run by the course author.

Try it step by step

Run the original 19 checks and five depth checks; distinguish schema validation from actual process/protocol behavior.
Exercise complete, missing-checks, injected, unknown-ID, malformed request and unavailable/not-ready cases.
In a generated copy implement missing-both. Run the actual-file/protocol grader; the starter intentionally fails until completed.
Use the native helper to install/list the tested version, then take the app new-chat skill/tool/hook trust track when supported.
Inspect actual findings and cached version, then remove the scratch installation. Keep every authenticated or cross-platform stage that was not run explicit.

A useful starting example

Illustration · adapt before use.

Behavior, transport, extension and isolation regression suite text

node --test labs/workflow-plugin/test.cjs labs/workflow-plugin/depth.test.cjs

Compare the measured local protocol exchange text

node labs/workflow-plugin/demo.cjs

Review the actual layered acceptance matrix text

Review labs/workflow-plugin/test.cjs and EXPECTED.md.
Map each actual case to schema, deterministic tool, protocol, failure, or hook-output evidence.
Include the zero-finding control and unavailable-fixture request-ID case.
List client discovery, skill activation, and hook delivery separately as unrun.
Do not install, authenticate, trust hooks, or publish.

From input to result

A useful negative control

The injected fictional README contains a deliberately hostile instruction alongside incomplete documentation.

Worked input text

node labs/workflow-plugin/tools/review-fixture.cjs injected

Expected result · compare with your run

injected: 2 findings; ignoredInstructions=true
Combined named suite result: 24 passed, 0 failed

The first line summarizes the actual JSON result: an untrusted-instruction warning and missing Verification heading. The suite proves these cases and bounded protocol behavior, not the safety of arbitrary documents.

Hands-on exercise

Your turn

Run the 24 combined checks and complete the extension against the actual protocol grader. Produce a layered acceptance matrix with structural, direct, real-stdio, native metadata and authenticated app observations. The author-tested native lifecycle does not fill an unrun skill-selection or hook-delivery row.

Try the exercise before opening its reference answer. Use a disposable practice folder.

Run the practice

Run positive, negative, and protocol integration checks

Extract into a new practice folder and keep the included labs folder. Run these commands from that extraction root, or from the course folder, using an existing Node.js installation.

Download this lab ↓

Command 1 text
```
node --test labs/workflow-plugin/test.cjs labs/workflow-plugin/depth.test.cjs
```
Check: 24 checks pass: 19 original behavioral/protocol checks plus five workspace/extension/isolation checks.
Command 2 text
```
node labs/workflow-plugin/demo.cjs
```
Check: The measured protocol transcript matches EXPECTED.md; it ends by stating no Codex session, model, account, or network was used.
Command 3 text
```
node --test labs/workflow-plugin/depth.test.cjs
```
Check: 5 checks pass, including starter failure, complete reference grading through actual stdio, preserved baseline suite and scope rejection.

Inspect the actual filesComplete behavioral suite ↗Deterministic exchange ↗Expected transcript ↗Injection fixture ↗Skill test procedure ↗Native catalog, installation and cleanup ↗Complete extension task, protocol grader and reference ↗Generated marketplace builder ↗Child-only real CLI driver ↗

Offline integration can prove more than JSON shape. Actual plugin discovery, skill activation, hook trust, and host behavior remain separate client acceptance cases.

Check your result

Positive/negative/incomplete inputs and real stdio lifecycle behavior pass without weakened checks.
The extension starter fails and the completed actual files pass the same factual/protocol gate.
Native install metadata and authenticated user-journey outcomes are reported separately with cleanup.

Reference answer & rubric Try the exercise before opening

A meaningful test matrix includes a zero-finding control, missing evidence, untrusted instructions, invalid arguments, and the real protocol boundary.

Run both suites and retain the 24 named outcomes. Confirm the complete fixture has no invented defects and missing-checks has exactly its missing heading; the depth suite also grades actual reference artifacts and protocol.
Inspect the injected fixture and its warning. Treat it as document data; the narrow planted-pattern check is not a universal injection detector.
Confirm negative cases reject unknown IDs, arbitrary paths, extra fields, malformed JSON, and tool calls before readiness rather than returning plausible success.
Check event-specific hook outputs offline. Keep these outcomes separate from hook delivery or trust in a real client.
For optional client acceptance, use the README's disposable setup procedure, record actual package version, tool discovery, skill invocation, calls, output, hook trust and cleanup, and label any unrun stage not verified.

Think it through

Check your understanding

Offline package validation passed. What may you claim?

The public directory approved the plugin.
The checked package structure is valid; client execution remains unverified.
Every user workflow has passed.

Reveal the answer and explanation

Answer 2: The checked package structure is valid; client execution remains unverified.

Verification claims must match the layer actually exercised. Package structure, runtime behavior, and publication review are separate.

Take this quiz in the interactive lesson to save your answer in the browser.

When it does not work

The suite passes, but nobody has observed the plugin in Codex.

Likely cause: Offline protocol testing was mistaken for host acceptance.

What to try: Keep the client acceptance rows unrun until the supported installation and session procedure is actually performed.

An injection test passes, so every hostile document is declared safe.

Likely cause: One deterministic fixture pattern was generalized beyond the implementation.

What to try: State the narrow detection boundary and continue treating every tool result as untrusted source material.

A valid tool request fails because packaged fixture data is unavailable.

Likely cause: The tool encountered an execution-time read failure, not malformed request JSON.

What to try: Inspect the original request ID and isError tool result, restore the missing scratch fixture if appropriate, and rerun. Do not parse this as an empty successful review.

Common mistakes

Using a copied foreign CLI flag and calling the package tested when it was never loaded.
Writing expected results in the observed-results column or omitting negative cases.

Source mapping & differences · different

Layered plugin testing transfers, but Codex local marketplace and supported-client setup replace the source's plugin-dir launch workflow.

Source topic: Test a plugin ↗

Go to the source

Official documentation, checked 21 October 2026. Product names, controls and availability may vary with your version, platform or account.

Continue in the interactive lesson →

Completion and quiz progress are saved
only in the interactive course.