Are You the Next CIO?
Questions to help you understand how you may fit as a future CIO in a member-centric nonprofit organization.
Recently I came across a set of articles written by the team at Cimatri on the topic of hiring your next CIO. As a technology vendor to the nonprofit community, Cimatri framed the articles to primarily guide and help the hiring company. However two articles in particular grabbed my attention 18 CIO Interview Questions and IT Maturity: What Type of CIO is Best-Fit? because they highlighted how important it is for you, the CIO candidate, to understand yourself and to understand what type of CIO the organization is seeking.
Both of these articles got me thinking about my own journey as a CIO and the successes and challenges I have faced along the way. I started to think about the questions I asked at my interview. What questions might have helped inform the strategies needed in the role? What questions might have helped anticipate the “type” of CIO the organization was seeking vs. the type CIO they needed? Was I that type of CIO?
So before we get to the list of questions, it is important to perform an honest assessment of the type of CIO you want to be - especially if this is your first CIO role. That type of assessment is hard to make as a first-timer. But an honest understanding of who you are as a potential leader, how you understand strategy and relate it to your teams and the organization, and what you could do with the “magic wand” will help you identify the “right” answers when you ask your questions at the interview.
Finally, if this is your first executive technology role, fight hard against the allure of the title. Why? Because just focusing on title attainment will cloud your judgement of the role’s fit and it may cloud your judgement about how to relate the title to the rest of the organization.
Launch Pad Questions
If you’re looking to lead change, the technical foundations present in the organization will help you gauge where you might start. Enabling certain strategies may require tools and team knowledge not yet present. These questions will help you understand where the organization stands. These will be especially useful if you get to interview with other technology team members.
1. What platforms or tools can’t you live without?
Ideally this question will reveal what functionality the teams value most. Tools and platforms are a manifestation of what’s important to an organization, where money has been spent, and to some degree how modern or mature the technology stack may be. You can follow up with more domain specific questions such as what is being used for MFA, content filtering, CI/CD, QA, monitoring, etc. You can also flip this as a good anti-pattern question; What tools aren’t being mentioned or do wish you had?
2. What metrics are you relying on?
This one will help you identify two key attributes; first, do the teams value metrics and second, what initiatives are driving metrics. You want teams that aren’t afraid of transparency of their operations. Good follow up questions might include asking which of their metrics are leading and lagging, how often the metrics are reviewed as a team, and in the case of KPIs how teams respond to unfavorable changes. Depending on where the conversation leads, some questions around SLAs, SLOs, and SLIs will give some indication of how developed the team’s SRE practices are.
3. What are your top security breach vectors?
This one will help you understand collective blind spots to security. Top teams may answer in ways that may even reveal your own blind spots! Fundamentally you are looking for a cohesive understanding of the threat landscape as it relates to the organization. Where mitigation, training, budget, and governance converge. Good follow ups might explore how teams relate security responsibilities to each other. Be alert if you see signs of security gate keeping or “that other team handles that” mentality. You may also probe about password management and overall privileged-account structures and account use. Finally, probe a bit into breach insurance and breach response plans to determine the depth of the team’s understanding in these areas. If you’re feeling bold, ask about past breaches.
4. How often do you release?
If the technology team has any responsibility for software engineering, asking this question will help reveal their underlying execution strategies. Answers that point to consistent release cadences might indicate the team has a well established CI/CD pipeline. It might also help you understand how development work arrives to the team - does the work come “over the wall” in tickets/requests or does it arrive as a byproduct of integrated product team participation. This will give you clues if the team in place is either transactional or participatory. Follow up questions might explore the level of QA automation, behavior-driven development practices in place, basic test coverage of code, and product roadmaps being followed. You may even ask to see actual roadmaps.
Cave Painting Questions
If you take the CIO role, expect your first months to be in the role of a technology anthropologist. I often use the term “cave paintings” to describe what culture has left behind as clues to what the organization truly values. In other words, if you find it painted on the caves, you can infer what values are worth keeping and those that may need to be tuned.
1. How does the organization relate to the technology team?
This question may help you understand how technical work arrives to the team, but more importantly how the organization perceives the quality and delivery of that work. In cave painting terms, does it show teams shaking hands or does it show teams firing weapons at each other? If possible, ask the executive team and the technical team the same question. Grab popcorn.
2. How does technical failure impact the organization?
With this question, fundamentally the goals is to understand if learning is encouraged within the organization or if an idealized state of perfection is valued. If discussion turns to specific failures, ask about any retrospectives that were done and ask about specific learning that was gained. This question may also reveal the strength of the organization’s compliance programs - if any.
3. How does the organization find information and data for decision making?
The cave painting you hope to see with this question would be a group of people huddled around a fire basking in its warmth. And in this case, “fire” is a well functioning data pipeline that allows both for “solo” data exploration by individuals, but also includes data expeditions in which those seeking information can pair up with a competent data team to find new insights. Be careful if the question leads to “IT handles all reports” or “Staff opens a ticket to get a report generated” answers. The days of the politburo of reports is long over.
4. How do staff performance evaluations relate to organizational goals?
This question presupposes that there is an agreed benefit of decoupling individual performance evaluations (i.e. how do we get our raises) and organizational goal attainment (i.e. how do we measure outcomes of our work). This question may also reveal if the organization engages in an annual or biannual evaluation cycle or if greater importance is given to ongoing 1:1 CFR-style interactions. Good follow up might be to probe on what the average performance evaluation scores in the technology group have been for the past years. The idea is to infer if evaluation scores are valuable indicators to the organization or simply a proxy for raise distribution curves.
Back To The Future Questions
A sure sign of execution problems are organizations struggling with problems that either span years without solving, or keep bubbling to the top of the problem list over and over. Time to unpack what is contributing to this.
1. What is the longest running technical problem you face?
If you have the opportunity to ask this question of the Board, the executive team, and the technology team you will gain an excellent understanding of how aligned everyone is on what the “next” technical challenge to address should be. If, on the other hand, you find wildly different answers, it’s time to drill in to the dimensions each group is using in their assessment. Good follow ups here might be to ask how long the problem has persisted and what the key barrier to progress has been. How Board members answer this may also reveal how robust the lines of communication are between technology leadership and governance leadership.
2. What technology solution has been on the roadmap but has not been implemented?
Somewhat related to the above question but framed in the context of a roadmap. By framing the question this way you can uncover if a) there are roadmaps at all and b) if there is a roadmap, what is interfering with arriving at its objectives. Also note that this question does not speak to a problem but to opportunities that haven’t been realized. Answers may reveal that roadmaps aren’t valued or followed. That might prompt follow up question about how often roadmaps are developed and adapted. And although roadmaps are just one mechanism out of many to orient actions, the real value of this question is to explore what impediments exist that may impact your ability as CIO to execute a strategy. You’ll find a relationship to the “Entropia” questions further down.
3. What’s the forecast for technology spend over the next few years?
The idea here is to try and understand if there is a correlation between capital allocation and execution problems. For example if you hear lofty objectives for technology transformation but also get a “flat” answer for budget growth, it’s time to explore how the organization reconciles flat spending with growth. Further, the difference in how an executive team and technology teams answer this question may reveal the quality of vision alignment and what it will take to fulfill that vision. You are trying to avoid the spinning gerbil wheel of defending lack of execution as a byproduct of lack of funding.
To be clear, the insinuation isn’t that you can only execute with a growing budget. However, understanding the relationship between spend and growth within the organization is educational. A good follow up question is to ask about CAPEX and OPEX spending in the future. This may reveal how the organization is balancing its technology solutions between hardware and subscriptions. How does that align with your envisioned strategy?
4. What are one or two KPIs that have been underperforming consistently?
The question is tailored to be generic on purpose. Which KPIs are discussed may indicate correlations between technology barriers that are persistent and the organizations ability to execute. Likewise different teams will gravitate to KPIs that matter to them. Again, this provides a good lens on the prevailing values embedded within each team. Follow up questions would aim to understand how or why teams believe an underperforming KPI is connected to technology strategy.
“Entropia” Questions
If you read my post about how to frame technology management around the idea of entropy, then you may agree that understanding how much technical debt an organization is carrying will impact your ability as CIO to activate change.
1. How much spare capacity do teams have?
An organization’s ability to manage technical debt is directly proportional to amount of spare capacity available to address the debt. In financial terms, this is how much money you send to a credit card above the minimum monthly payment. Answers to this question will reveal true capacity to manage debt, how well sized teams may be in relation to workloads, whether teams are measuring capacity at all, and finally if teams value spare time in the first place. While no specific amount of spare capacity is the “right” amount, you certainly want to investigate any answers that begin with “zero.”
2. What are the top 2-3 things you wish the teams had time to work on?
Here your goal is to understand where the technical debt is hiding. By revealing this, you may be able to understand how to frame a “now, next, later” roadmap on addressing the debt. For example, if all three things point to issues likely to impact your cyber-security posture, perhaps that’s an excellent “Now” bucket of work. You may follow up with asking about the age of the debt. For example, asking how long those 2-3 things have been on the list.
3. Who sets the next thing to work on?
Understanding how full a team’s pipeline is, who drives the order of items in the pipeline, and how often the items are revised may yield some clues about how much technical debt may exist. You hope to understand if there is any room for technical debt payback on a consistent basis. If answers reveal that teams have little ability to dictate their work, it is unlikely that technical debt is being paid down. The business rarely identifies the debt accrual at a macro scale. So unless teams that see the costs of the debt daily are able to inject work as part of normal planning, there is little hope in making headway.
4. What is the most significant technology barrier that has been solved in the last year?
This question can be framed in different ways but this approach will hopefully reveal that the organization understands and identifies technical debt as a real and ongoing concern. Answers that speak to opportunities that were unlocked by way of removing a barrier are a good sign. Try not to accept answers that speak to new features or platforms unless they were deployed as the solution to the barrier. A good follow up might be to ask about new technical debt the new solution introduced. This will reveal that teams have a comprehensive understanding that technical debt accumulates to any solution on day one.
Conclusions
Certainly each organization and each interview will be different. Here I have tried to bundle together the concepts I have found to be useful in my own experience. I have also tried not to give overly deterministic answers to the questions but instead I have provided some breadcrumbs of answers that might be better than others. Additionally, consider that as you lead up to your interview, there is an abundance of information you can gather about the organization ahead of time. Reviewing IRS 990 forms, Glassdoor reviews, LinkedIn networks, key vendors used by the organization, and the position description itself can give you important insights and clues into how these questions might be answered.
Finally, the point is to understand these questions within the frame of the type of CIO you wish to be. If the answers align well to that image you’re in a great place to be a successful technology leader. Good luck!