ES · Leer en español Explore for free

Recipe · Intermediate · 20 min

Create a Claude Code plugin: init, testing, evals and marketplace

Build a Claude Code plugin with plugin.json and a skill, validate it, try it with --plugin-dir, run an eval and publish it in your own marketplace. Tested.

Cover: Create a Claude Code plugin: init, testing, evals and marketplace

Tested with Claude Code 2.1.316 · reviewed

See the files on GitHub Free · MIT license

The problem

You have skills, agents or hooks that work for you and you want to use them in several projects or hand them to your team without copying folders by hand. A plugin is exactly that: a folder with skills, agents, hooks or MCP servers and a plugin.json manifest that names them, so you can install them in one go and publish versions. The problem is that between creating the structure, validating it, checking that Claude uses it well and publishing it there are several steps with traps that the documentation does not bring together in one place.

The solution

We will create commit-en, a plugin that writes commit messages in English, then validate it, try it without installing it, run an automatic eval on it and publish it in our own marketplace. Everything was tested with Claude Code 2.1.316 on Linux. The complete plugin is in the en/ folder of the free repository ctrlia-crear-plugin-claude-code (its root holds the Spanish twin, commit-es).

1. Decide whether you need a plugin

If you only want a skill for yourself or for one project, you do not need a plugin: .claude/skills/<name>/SKILL.md is enough (we explain it in the first skill recipe). Build a plugin when you want to share the configuration with other people, install it in several projects or publish versions.

A plugin can carry skills (skills/), subagents (agents/), hooks (hooks/hooks.json), MCP servers (.mcp.json) and output styles, all at the plugin root; only plugin.json goes inside .claude-plugin/. Moving to a plugin changes the names: skills get the plugin name in front, like /commit-en:message, so two plugins can each have a message skill without clashing.

There are two ways to start. claude plugin init <name> creates a plugin in ~/.claude/skills/<name>/ that loads automatically in all your sessions (handy for yourself). For something you are going to publish, the documentation creates the folder by hand and tests it with --plugin-dir; that is what we will do.

2. Create the structure and the manifest

The final structure of the plugin:

commit-en/
├── .claude-plugin/
│   ├── plugin.json
│   └── marketplace.json        (step 5)
├── skills/
│   └── message/
│       └── SKILL.md
└── evals/                      (step 4)

Create commit-en/.claude-plugin/plugin.json with name (required, no spaces: it becomes the skills' prefix), description, version, author and license:

{
  "name": "commit-en",
  "description": "Writes commit messages in English using Conventional Commits",
  "version": "0.1.0",
  "author": { "name": "Ctrl+IA" },
  "license": "MIT"
}

Then create commit-en/skills/message/SKILL.md. The skill's description is what Claude compares with what the user asks for, so include real phrases like "write the commit" or "commit message". With allowed-tools (a YAML list) the skill can run git diff and git status without asking for permission during that turn. The instructions are short steps:

---
name: message
description: Writes the commit message for the staged changes (git diff --cached) in English using the Conventional Commits format. Use it when the user asks to "write the commit", for a "commit message" or to "make the commit".
allowed-tools:
  - Bash(git diff *)
  - Bash(git status *)
---
1. Run `git diff --cached --stat` and `git diff --cached`.
2. If nothing is staged but the user describes the change, use their description; if there is neither, say so and stop.
3. Write ONE title line of ≤ 72 characters: `type(scope): summary` in English, in the imperative mood
   (types: feat, fix, docs, refactor, test, chore).
4. Leave a blank line and, only if the change is not obvious, 1-3 bullets explaining why.
5. Return only the message, with no comments before or after. Do not run `git commit`.

If you would rather have the skeleton generated, claude plugin init commit-en --with skills creates the manifest (with "skills": ["./"]), a root SKILL.md and an example one, with TODO in the description. Remember that it creates them in ~/.claude/skills/, not in your current folder.

3. Validate it and try it without installing anything

Validate the manifest and each skill's frontmatter:

claude plugin validate ./commit-en

It ends with ✔ Validation passed. In 2.1.316 it also runs the Anthropic plugin directory checks and warns about what you would need to submit it there: in our test it asked for a README.md of at least 40 words, a LICENSE (or license in plugin.json) and a square PNG icon of 512 to 2048 px. Those warnings do not make validation fail.

Then load it for one session only and try it with some changes staged with git add:

# Interactive session with the plugin loaded just this once
claude --plugin-dir ./commit-en
# > /commit-en:message

# Or headless, from a repo with staged changes (git add)
claude -p --plugin-dir ../commit-en --output-format json "/commit-en:message" < /dev/null | jq -r '.result'

If you edit plugin files with the session open, /reload-plugins reloads them. In our test repository, the Spanish twin (claude -p --plugin-dir ../commit-es "/commit-es:mensaje") returned a correct feat(calc): ... title in about 10 seconds. If something does not show up, /plugin has an Errors tab with the reason, and claude plugin list shows what loaded. If you have never used claude -p, the Claude Code headless recipe explains it.

4. Check that it changes behavior: claude plugin eval

A plugin loading does not mean Claude uses it well. claude plugin eval (v2.1.269 or later; git 2.31 or later if you have git) runs test cases with and without the plugin and scores the difference. Each case is a folder in evals/ with a prompt.md (what a user would type) and one or more graders in graders/:

claude plugin eval init proposes them in a session; claude plugin eval init --bare name creates an empty template. This is our case, with each file marked by a comment:

<!-- evals/commit-rename/prompt.md -->
---
max_turns: 6
allowed_tools: [Read, Glob, Grep, Skill]
---
Write me the commit message for this change: I renamed getUser to fetchUser and updated the three calls.

<!-- evals/commit-rename/graders/format.md -->
---
type: regex
flags: m
pattern: '^refactor(\([^)]+\))?: .*fetchUser'
---

<!-- evals/commit-rename/graders/skill-used.md -->
---
type: tool_used
tool: Skill
input_match: '"skill"\s*:\s*"(?:[\w-]+:)?message"'
---

<!-- evals/commit-rename/graders/criteria.md -->
---
type: llm
---
PASS if the response is a single commit message in English whose title uses the refactor type and mentions fetchUser.
FAIL if it is in Spanish, if it includes text before or after the message or if it says it has run git commit.

By default each case runs 3 times with the plugin and 3 without it, and each run is a real model call that uses your plan. To iterate cheaply:

cd commit-en
claude plugin eval . --runs 1 --ablation none --model haiku --max-cost-usd 0.5 --no-publish
# Before trusting the result: with the 3 runs and the comparison without the plugin
claude plugin eval .
# Add evals/results/ to .gitignore

Our first eval (run on the Spanish twin) failed (0.33) and taught us two things. The regex with (?m) failed because patterns are JavaScript: flags go in flags:, as in the file above. And the judge rejected the answer because, without Bash in allowed_tools, the skill explained that it could not read the diff before giving the message: exactly the kind of failure an eval should find.

5. Publish it in your own marketplace

A marketplace is a .claude-plugin/marketplace.json file in a git repository that lists your plugins. To publish from the plugin's own repository, save it next to plugin.json with an entry whose source is "./" and whose name matches the manifest's (if they do not match, installation fails):

{
  "name": "ctrlia-en",
  "description": "Ctrl+IA plugins for Claude Code, in English",
  "owner": { "name": "Ctrl+IA" },
  "plugins": [
    { "name": "commit-en", "source": "./" }
  ]
}

Pick an uncommon name for the marketplace, because nobody can have two marketplaces with the same name. The description is optional, but without it claude plugin validate warns that it is missing. Run claude plugin validate ., commit and push it to GitHub. This is what your users will do:

claude plugin marketplace add your-user/commit-en
claude plugin install commit-en@ctrlia-en
# or in a single step (v2.1.292 or later)
claude plugin install commit-en --marketplace your-user/commit-en
# update when you bump the version
claude plugin update commit-en@ctrlia-en

(your-user is your GitHub user.) Updates arrive with claude plugin update when you change version. We tested it with a local folder instead of GitHub and it worked: marketplace add and install with user scope. If you used plugin init earlier with the same name, claude plugin list warns that the copy in ~/.claude/skills does not load because the installed plugin already has that name: rename one of the two.

6. Review security: what a plugin can do on your machine

A plugin is not just text: its hooks and MCP servers run programs with your user permissions, and its skills can bring allowed-tools that approve commands. Before installing someone else's, read its hooks/hooks.json, its .mcp.json and the scripts they call, just as you would review an npm package (the official Plugin security and trust page details what to review).

Neither claude plugin validate nor claude plugin eval is a security review: eval's own help says its isolation limits what a malicious plugin can reach, but it is not a guarantee, and that you should only evaluate plugins you trust. In your own plugins, grant the minimum: allowed-tools with specific commands (Bash(git diff *), not Bash), no secrets in the repository and a version in plugin.json so your users do not get changes unintentionally. To remove a plugin: claude plugin disable or claude plugin uninstall, and for those created with plugin init, delete their folder in ~/.claude/skills. If your plugin carries hooks, the security hooks recipe explains how they work.

Check it

With Claude Code 2.1.316 on Linux, and with a temporary HOME and CLAUDE_CONFIG_DIR so as not to touch the real configuration, this is what we got with the Spanish twin, commit-es in the ctrlia marketplace (excerpt; the English copy in en/ also passes claude plugin validate):

$ claude plugin validate ./commit-es
...
✔ Validation passed

$ claude plugin marketplace add ./commit-es
✔ Successfully added marketplace: ctrlia (declared in user settings)

$ claude plugin install commit-es@ctrlia
✔ Successfully installed plugin: commit-es@ctrlia (scope: user)

$ claude plugin list
Installed plugins:

  ❯ commit-es@ctrlia
    Version: 0.1.0
    Scope: user
    Status: ✔ enabled

With the plugin loaded, /commit-es:mensaje on staged changes returned a correct feat(calc): ... title. claude plugin eval . --runs 1 --ablation none --model haiku ran the test case and scored 0.33 the first time because of the two failures in step 4. We have not yet tested publishing on GitHub for real or submitting to the Anthropic directory.

Why it works

Claude Code looks for the manifest in .claude-plugin/plugin.json and loads the folders it knows (skills/, agents/, hooks/…) from the plugin root, putting the manifest's name in front of each skill so they do not clash. --plugin-dir loads that folder for one session only, without installing anything. A marketplace is just a catalog in git: marketplace add registers it, install installs the plugin it lists and update fetches it again when version changes. plugin eval runs the same prompt with and without the plugin and scores the difference, so it measures whether the plugin changes behavior, not just whether it loads. More details in create plugins, publish plugins, plugin evals, the plugins CLI reference and the skills page.

Common mistakes

FAQ

Why is my skill called /commit-en:message and not /message?

Plugin skills carry the plugin name in front so they do not clash with other plugins' skills. The exception is the root SKILL.md that plugin init creates, which is invoked as /name.

I put skills/ inside .claude-plugin/ and they do not load.

Only plugin.json goes inside .claude-plugin/. Move skills/, agents/ and hooks/ to the plugin root and run /reload-plugins.

"no marketplaces configured": what does it mean?

That you have not added any marketplace yet. Add one with claude plugin marketplace add owner/repo (or a local path) and try the installation again.

How much does an eval cost?

Each run is a real model call with your credentials: by default, 3 with the plugin and 3 without it per case, plus the judge's. Use --runs 1 --ablation none --model haiku --max-cost-usd while iterating; our case cost less than an estimated cent.

The eval says "Invalid regular expression" in my regex grader.

Patterns are JavaScript: they do not support inline (?m) or (?i). Put the modifiers in the flags field, for example flags: m.

Validate warns me about README, LICENSE and icon even though it says "Validation passed"?

Those are checks for Anthropic's official directory: you only need them if you are going to submit it there. For your own marketplace they are not required, although a README with the install line helps your users.

The whole library