You build a product that AI agents use. An MCP server is how they reach it: MCP (Model Context Protocol) is the standard way an agent app, such as OpenCode or Codex, connects to outside tools. Your own tests pass. Then a customer connects their agent, and it fails at the first step.
The problem is that every agent app connects in its own way. The only sure test is to use your server the way your customers do, in each app they use.
A life-science analytics company asked us to do that. A d5s agent tested their MCP server in OpenCode and Codex, on its own computer, over two days. It worked like a tester on their team: it reported each fault, the team fixed it, and the agent tested again.
- Five faults found before customers found them. Each came from a real agent app against the live server, and the team fixed most within hours.
- Two agent apps, two model providers. After the fixes, agents in OpenCode and Codex both read the company's data.
- A person approves every change. The agent asked before it created an account, uploaded data or started an analysis.
- The agent works on its own. It installed the apps, used its own test account and read its own sign-in codes.
How to test an MCP server across agent clients
Use the server in each agent app your customers use, sign in as they do, and ask the agent to use each tool.
A check that the server is up proves little. The real question is whether an agent can find and use its tools. We gave the agent one goal: connect the MCP server to agent apps on its computer, and make sure an agent can use it.
The agent has its own Linux computer in d5s. It downloaded each app from its publisher, checked that the download was genuine, and installed it. It set up three apps:
- OpenCode Desktop, the graphical app, with its free model.
- OpenCode CLI, the terminal version.
- Codex CLI, signed in to ChatGPT by a person.
The agent sent the same requests through each app, so we could compare the results.
Step 1: Give the agent a test account
The agent registered its own test account on the company's app, after a person approved the details.
The MCP server needs a user account. The agent opened the registration form, filled in a test identity and stopped:
May I submit the registration with these details? I haven't created the account yet.
After the yes, it registered with its own d5s email address and made its own password. It never asked anyone to paste a password into chat.
The company's sign-in sends a six-digit code by email. The agent's mailbox only accepts senders that an admin allows, so the admin allowed the company's address with one click in the chat. From then on, each code reached the agent directly, and it signed in before the code expired. The agent email docs explain the sender rules.
Step 2: Sign in from each client
Sign-in failed first, in a different way in each app. Most faults were here.
Both apps tried to register themselves with the server, and the server refused. The agent reported the exact error and stopped. It did not look for a way around the refusal. The team then set up a fixed sign-in for each app.
OpenCode then asked for one more permission than the server allowed. The agent sent back the full error, with the list of permissions that OpenCode had asked for. The team fixed it in about 25 minutes.
After both fixes, the agent in OpenCode Desktop listed 10 of the company's 54 datasets. Within the hour, Codex found the same 54.
- 01Day 1, 14:49Both apps failed to sign in: the server did not let them register themselves.The team set up a fixed sign-in for each app.
- 02Day 1, 17:29OpenCode asked for a permission the server did not allow, so sign-in failed.The team allowed that permission.
- 03Day 1, 18:10The server had no tool to upload a dataset, so the agent stopped the upload test.The team shipped three upload tools the same evening.
- 04Day 1, 20:21Every upload failed with “The file could not be staged”, and the error gave no cause.The server logs showed the cause in minutes: no permission to save files. The fix was live in about 40 minutes.
- 05Day 4, 12:37The upload check passed without comparing any sample groups, because the test data was too small.The team wrote a new test plan with larger data.
Step 3: Test uploads, with a person in control
The agent showed each upload check to a person and waited before it started the analysis.
Reading data is safe to repeat. Changing data is not, so the agent treated the two differently.
At first, the server had no upload tool. The agent did not use the web app instead, because that would not test the MCP server. The team added upload tools the same evening.
Then every upload failed with "The file could not be staged". The message did not say if a retry could help, or if the failed upload had left a file behind. The agent did not retry, because a retry could create a duplicate. It asked the team to check their logs.
The logs had the answer. A second d5s agent, with access to the server logs, found the cause in minutes: the server had no permission to save files in its new storage. The fix was live in about 40 minutes.
The next upload reached the company's quality check. The agent read it and saw a gap: the check compared no sample groups, because the test data was too small. It showed the check and asked before it started the analysis.
- RNA-seq, Homo sapiens: 4 features, 4 samples
- File matching: no unmatched sample IDs
- Findings: none returned
- Comparisons: empty, so the control against treated check is not confirmed
The team stopped that run and wrote a full test plan with larger data. The agent read the plan word for word and found a contradiction in it. It said how it would resolve it and carried on.
With the larger data, the quality check did its job. It found the sample groups to compare, and flagged the samples that fit no group. Each time, the agent showed the check before it started the analysis.
What to check in your own MCP server
Test sign-in, tool lists and write recovery in each client, not only the tool logic.
If you build an MCP server, these are the checks we now run on every one:
- Client registration. Decide if you allow anonymous registration. If not, pre-register a client for each harness you support, and document its client ID.
- Scopes. Check the scopes that each client adds on its own, such as
offline_access. - Connection status. Codex showed the server as enabled but listed no tools until the agent renewed the OAuth session. OpenCode showed "connected" while a new sign-in never reached the consent screen. Make a tool call; do not trust the status line.
- Errors an agent can act on. "The file could not be staged" gave the agent nothing to act on. Say if a retry can help, and what the failed call left behind, such as "server problem, nothing was staged".
- Recovery tools. Give agents a way to read the state of a write, such as a list of staged files. Then an interrupted call is not a dead end.
- Credential-shaped output. For large files, the staging tool returned a signed upload form with a temporary access key. OpenCode's model provider blocked the replies that repeated it; Claude Code did not. The model sees everything a tool returns, and hosts scan it. Return one opaque upload URL instead.
- App-only tools. One tool was marked for the embedded viewer only. Claude Code hid it, and OpenCode listed it to the model. App-only is a hint to the host, not a control. Check the tool list in every host you support.
- Complete reviews. Agents read your output literally. A review with no comparisons and no findings made the agent stop and ask, because the two did not add up.
- Agent identities. Most stops were at sign-in, not in the tools. Give your test agent a mailbox for sign-in codes and a test account, as a d5s agent has.
Questions and answers
How do you test an MCP server?
Connect the server to the real clients your users run, such as OpenCode and Codex, and sign in the way a user does. Then ask a model to call each tool and check the result. In our case study, a d5s agent did this on its own computer and found five faults in two days.
Why test an MCP server in more than one client?
Each agent app signs in, asks for permissions and lists tools in its own way. In our test, both apps tried to register themselves, and one asked for an extra permission. A server that works in one app can still fail in the next.
Can an AI agent test an MCP server without a person?
Mostly. A d5s agent installs the clients, configures them, reads its own sign-in codes and runs the tool calls. A person still approves each new account, each upload and each analysis, and signs in where a model provider needs a human login.
How does the agent get verification codes?
Each d5s agent can have its own email address. An admin decides which senders can reach it. In this test, the agent received the server's sign-in codes directly and used them before they expired.
Does the test agent change production data?
Only after a person approves it. The agent showed each upload check and waited before it started an analysis. It also did not repeat a write whose result it could not confirm.
Why did my MCP host block a tool response?
Hosts and model providers scan what a tool returns. In our test, a signed upload form with a temporary access key made one model provider block the reply. Keep credential-shaped values out of tool output, for example with one opaque upload URL.
Which models can run a d5s test agent?
Any model your workspace allows. The agent and the clients it tests are separate: in this case study, the clients under test ran on two different model providers.




