How to Review Your CIO or CTO's Performance

To review a CIO or CTO's performance, judge them on outcomes across everything the technology function is accountable for, not just the projects you can see. Use the same areas every time: strategy, people, cloud and network, security, policies, risk, vendors and technology performance. Check what they report against other evidence, and compare against a fixed baseline so you are measuring change rather than impressions.

In practice, most reviews of technology leaders run on two inputs: whether the visible projects shipped, and how the CEO feels about the relationship. Neither tells you whether the backups would restore.

This guide is for CEOs, founders and board members who want a review that answers questions like that.

Why reviewing a CIO or CTO is harder than reviewing other executives

So much of a technology leader's work is hidden beneath the surface. When they do the job well, very little happens: no outage, no breach, no supplier surprise, no project quietly running six months late. Good work looks like nothing, and nothing is hard to give credit for.

Your other executives come with an outside reference point built in: the CFO has the accounts (and often an auditor), sales has a number. Technology rarely has an equivalent, so almost everything you know about it reaches you through the person you are reviewing. That is not dishonesty: people report what they are working on, not the problem they have not reached yet. But a review built only on the leader's own reporting ends up grading the reporting, not the function.

If you are not technical, it is also hard to push back on an answer you do not fully understand, and the reassuring answer is usually the vague one.

What to measure in a CIO or CTO performance review: eight areas

There is no single industry-standard list, but these eight areas cover most of what a technology leader is accountable for, and they are the eight that StackUp's Technology Assessment scores. Delivery and budget run through all of them, and are covered under outcomes below.

Innovation and Strategy: Is technology work heading in a stated direction that the rest of the executive team recognises, or reacting to whichever request lands first? Look for a written technology plan tied to business goals that changes when the business strategy does.

Staff and Training: Is the team capable, growing and well supported? An effective CIO or CTO does not just build technology, they build teams. Look at key-person dependency (what happens if one named engineer leaves), turnover, and whether skills are being refreshed or slowly going out of date.

Cloud and Network: Does anyone have a clear picture of what the environment is made of, what it costs and who understands each part? Look at whether costs are tracked against a budget, and whether the environment can grow without a rebuild.

Data Security and Privacy: Who has access to what, how often is that checked, and what could you tell a customer on the day something goes wrong? Look for regular access reviews, especially after people change roles or leave, and for a tested response to a data incident.

Policies and Procedures: Are the policies everyone assumes exist actually written, current and followed? A policy that exists only for audit season is not a control.

Risk Management: How well are risks identified, assessed and mitigated? Does someone own the risk register, and has the recovery plan ever been tested? A flashy app does not matter if the underlying systems are vulnerable.

Vendor Management: Are third-party suppliers managed for value and compliance, or are there risks hiding in the partnerships? Look at contract terms, renewal dates, how dependent you are on a single supplier, and what it would take to switch.

Technology Performance: Is performance measured against agreed targets, or judged by whether anyone has complained? Look for agreed service levels, incident trends over time, and whether the systems are stable, scalable and delivering value.

Measure outcomes, not activity

Activity is easy to report and easy to mistake for progress. Tickets closed tell you the team is busy, not whether the business is better off.

Outcome measures answer a different question. Did the system the sales team depends on stay up through the busy period? Did the cost of running the platform fall as a share of revenue? Has the number of systems only one engineer can support gone down? Did the projects that mattered most land roughly when and for roughly what was agreed, and were slips flagged early?

If you run software teams, the delivery measures from Google's DORA research programme are a useful starting point: how often changes reach production, how long a change takes to get there, how often a change fails, and how quickly the team recovers when one does. The current guide adds a fifth, how often deployments are unplanned fixes for problems in production. Treat these as a view of the delivery system the leader has built, not a score for the leader personally, and compare them with the team's own history rather than with another company's.

Where to find evidence beyond the leader's own reporting

The most useful thing you can do is add sources: not to catch anyone out, but to see the function from more than one angle.

Incidents and post-incident reviews. Ask for the last three serious incidents and what changed afterwards. A function that learns writes this down.
A restore you witness. "We restored from backup last quarter" is a claim. Watching a restore of something that matters is evidence.
The supplier and contract register. Renewal dates, notice periods and spend.
Budget against actual. Not whether technology spent less, but whether variances were flagged before they happened.
The people in the team. Turnover, exit conversations, and what engineers say about workload and tooling.
The people the function serves. Ask your other executives how well technology supports their teams.
Project health against the status report. Every project is green until it is not. Compare the reported status with what the people doing the work say.
External reports. Penetration tests, audit findings and cyber insurance questionnaires, along with how quickly the findings were closed.

StackUp has two tools for some of these. Discovery Agents run tech health and project check-ins: an AI agent calls or messages the people you choose on a schedule you set, and reports the findings with the transcript behind each one, and the recording where it was a call. Run it openly, with your technology leader involved in what is asked and able to see the findings, so it builds a shared view of the function rather than a case against one person. The Project Assessment tests a status report: it records the submitter's own red-amber-green rating separately from the score, and flags it when the two disagree sharply.

How often to review a CIO or CTO

Technology risks and costs can shift within a quarter, so a single annual review leaves long gaps. A simple rhythm works better:

Quarterly: a short review against the same baseline and the same measures. What moved, what did not, and the top risks.
Annually: the formal performance conversation, built on four quarters of evidence rather than last month's memory.
After a major incident or a failed project: a review of the decisions involved, not just the outcome (more on this below).
At a leadership transition: when a new CIO or CTO starts, set a baseline in the first weeks so that, a year later, you can see what they changed rather than what they inherited.
Before a major initiative or an investor discussion: so the conversation starts from a clear view of where things stand.

Questions to ask in the review

These work whether or not you are technical. Listen for names, dates, numbers and documents.

What are the three biggest technology risks to the business right now, and what are you doing about each?
What would happen if our most important system went down for a day? When did we last test that?
Which systems can only one person support, and what is the plan to change that?
Which supplier would hurt us most if they failed or tripled their price, and what is our exit option?
What did technology cost us this year, and what did we get for it?
What did you decide not to do this year, and why?
What is the gap you have been meaning to close that has not been closed yet?
How do you know the security controls we have are working, rather than just in place?
What do you need from me, or from the board, that you are not getting?

The last question matters. Results depend heavily on the budget, headcount and executive attention a leader is given, and a review that ignores that is not fair.

Red flags to watch for

Every update is green, every quarter, with no risks worth mentioning.
Answers stay general when you ask for specifics.
The same risk appears in every report with no change in its status.
Critical systems, credentials or supplier relationships sit with one person.
Recovery plans exist but have never been tested.
Costs rise without an explanation that was given before the rise.
Your other executives describe technology as something that happens to them rather than with them.
A request for independent review meets resistance rather than interest.

None of these on its own proves poor performance. Each one is a reason to look more closely.

How to keep the review fair and avoid hindsight bias

Two well-documented biases make reviews of technology leaders harder, especially after an incident. Hindsight bias makes past events look more predictable than they were once you know how they turned out. Outcome bias leads people to judge a decision by its result rather than by the quality of the reasoning at the time. After an outage or a failed project, both pull the same way: the decision looks obviously wrong, and the person who made it looks careless.

A few habits make the review fairer for everyone:

Agree the measures at the start of the period. Changing what counts afterwards makes a review feel arbitrary.
Judge decisions on what was knowable at the time. Ask what information was available, what options were considered and what risks were flagged. A sound decision can still produce a bad outcome.
Keep a light decision log. The date, the reasoning and the trade-offs accepted for each major decision. It protects good decisions that turned out badly, and exposes lucky ones.
Separate what the leader controlled from what they did not. A budget cut, a hiring freeze or a supplier collapse changes what was achievable.
Never penalise disclosure. If raising a risk gets someone marked down, you will stop hearing about risks. Reward the leader who surfaces problems early.
Share the evidence first. Give the leader the same information you are using and let them respond before you form a view.

Using a consistent baseline to track change

The hardest part of reviewing a technology leader is comparison. Without a fixed set of questions, every review asks different things, and a year of reviews produces a series of opinions with dates on them rather than a trend.

A consistent baseline fixes that. StackUp's Technology Assessment asks eighty-four questions across twelve categories, eight of which carry the score: the same eight areas listed above. It produces one number out of 100, with a band from At Risk to Excellent, the eight scored areas opened up, and the four priority actions worth starting with. The score is calculated by fixed rules from the answers, not generated by AI, so the same answers always produce the same number, and a rerun a quarter later shows what changed in each area, not what everyone remembers changing.

Be clear about what the baseline is. It is usually completed by the person who runs technology, which in a performance review is the person being reviewed. Fixed questions and fixed scoring make it consistent. They do not make it a third-party check. Three things make it more than a self-report:

Bring others in. One person can start the assessment and invite others in, and you can bring your board, executives and technology team into the same view. Ask the people closest to each area whether they would answer differently.
Test the answers. You can click into the At Risk answers, the opportunities and the strengths and see the individual answers behind each count. Each is a specific, factual thing to ask for evidence of.
Add a separate voice. A Discovery Agents check-in round hears from the team directly, so the picture does not rest on one account.

Done well, the baseline is something you do with your technology leader, not to them. It gives them evidence for what they have been asking for, and a way to show what moved once they got it. If you are the technology leader being reviewed, see StackUp for technology leaders.

Running the Technology Assessment is free. The pricing page lists ongoing assessment tracking under the paid Continuously Improve plan.

What boards should ask for

Boards depend on management reporting, and are entitled to. But a board is expected to bring its own judgement to that reporting, and that is hard to do when the only view of technology comes from the person responsible for it.

So ask for a repeatable, structured view of the technology function alongside management's update: the same measures each quarter, the movement since last time, and the top risks with owners and dates. Ask, too, what evidence sits behind each risk rating. If you sit on a board, see StackUp for board members. If you are the CEO or founder accountable for the technology function, see StackUp for C-level executives.

Putting it together: a simple review structure

At the start of the year, agree the eight areas, the outcome measures and the baseline score you will review against.
Each quarter, rerun the baseline, review the movement in each area and discuss the top risks.
Throughout the year, gather evidence from at least two sources beyond the leader's own reporting.
Before the annual review, ask the leader for a short self-assessment against the same areas.
In the review, discuss decisions as well as outcomes, and the support they received as well as the results.
Finish with agreed priorities for the next period and what the leader needs to deliver them.

If you are a non-technical founder, the hardest part is often the conversation itself. Our piece on managing technology leaders as a non-technical founder covers how to close that communication gap.

Rather than relying on gut instinct or incomplete data, a structured review lets you recognise great performance properly and address gaps before they become serious liabilities.

How to Review Your CIO or CTO's Performance: A Guide for CEOs and Boards