Toward Conversational Capability Assessment in the Age of AI
I have been thinking a lot about how we evaluate technical capability in the current AI landscape.
Not just output. Not just velocity. Not just visible artifacts.
But judgment, systems thinking, and the ability to carry ideas from concept to architecture, implementation, operation, maintenance, trust, and long-term human use.
The traditional signals still matter. Resumes, portfolios, interviews, references, GitHub profiles, and public work can all be useful. But they are incomplete proxies, especially for people whose most meaningful work happens in private repositories, institutional environments, confidential systems, or contexts where responsibility matters more than visibility.
That is becoming more important in the age of AI.
AI has changed what output looks like. A person can now produce more text, more code, more prototypes, and more visible motion than ever before. That does not automatically mean the work is thoughtful, maintainable, secure, accessible, or aligned with human reality.
The stronger signal is not simply how much someone can produce.
It is how they reason.
It is how they decide what should be built, what should not be built, what must be reviewed, what should remain private, what needs consent, what can be automated, what must remain accountable to people, and what will still make sense after the demo is over.
Public Artifacts Are Not the Whole Record
Some of the most important systems I have worked on do not live cleanly in public.
They live in private repositories, local infrastructure, institutional code, client-sensitive contexts, research prototypes, operational workflows, and systems that require care before they can be shown widely.
That does not make the work less real. It means the work exists inside real constraints.
In some environments, publishing everything would be irresponsible. In others, the work is too contextual to understand from a repository listing alone. A commit graph can show activity, but it cannot explain tradeoffs, institutional constraints, accessibility decisions, governance concerns, or the shape of the human problem the system was meant to serve.
That raises a serious question:
How do we evaluate capability when the most meaningful work is not always publicly visible?
I do not think the answer is to discard public evidence. I think the answer is to treat it as one layer.
I have started treating GitHub as a curated, public-safe evidence layer rather than a complete record of work:
Not as a claim that everything important is visible there, but as an anchor for deeper conversations about systems, AI, infrastructure, accessibility, and how real work gets built and maintained.
From Inspection to Conversation
The future of technical assessment should not be less human.
It should be more conversational, more contextual, and more honest about what public artifacts can and cannot prove.
Instead of asking only:
- What does the resume say?
- What is visible on GitHub?
- What keywords match the job description?
- How many public contributions are there?
We should also ask:
- How was the system designed, and why?
- What constraints shaped the implementation?
- What tradeoffs were made?
- What was deliberately not built?
- How were accessibility, security, privacy, maintenance, and trust handled?
- How did AI assist the work, and where was human judgment essential?
- What would the person change if they had to maintain the system for five years?
- What evidence can be shared publicly, privately, or only in conversation?
Those questions are harder to automate. That is part of why they matter.
AI Should Assist Assessment, Not Replace Judgment
I am not interested in a future where AI becomes a more polished filter for excluding people.
I am interested in a future where AI helps people prepare better evidence, recover context, explain their reasoning, and participate in more meaningful assessment conversations.
Used well, AI can help a candidate:
- organize a large private body of work into public-safe summaries;
- prepare role-aligned evidence without exposing sensitive material;
- map projects to skills, responsibilities, and institutional outcomes;
- reflect on tradeoffs, failures, constraints, and lessons learned;
- create accessible documents that support different communication needs;
- prepare for deeper conversations instead of only passing keyword screens.
Used poorly, AI can make hiring worse. It can amplify shallow signals, obscure accountability, punish disability, flatten unusual paths, and make exclusion look more objective than it really is.
The ethical boundary matters.
AI should support context, reflection, structure, and accessibility. It should not become an opaque judge of human worth, a disability filter, or a replacement for accountable human assessment.
Candidate-Owned Evidence Packages
One model I am experimenting with is the idea of a candidate-owned evidence package.
Not a public dump of everything a person has ever done.
Not a performative portfolio optimized for metrics.
But a structured, consent-aware body of evidence that can include:
- a public profile;
- selected project summaries;
- private appendices where appropriate;
- role-specific CV or resume variants;
- case studies;
- technical notes;
- AI-assisted professional synthesis;
- disclosure boundaries;
- links to public repositories where useful;
- and a clear invitation to discuss the reasoning behind the work.
This is especially relevant for people who have worked across institutional systems, private infrastructure, research prototypes, confidential client work, accessibility, governance, security, or other areas where the most important evidence may require context.
It is also relevant for people with non-linear careers, disabilities, caregiving responsibilities, unconventional education paths, or deep experience that does not compress neatly into a standard resume.
The goal is not to avoid scrutiny.
The goal is to make scrutiny better.
What This Looks Like in Practice
For me, this has meant building a public profile that does not try to pretend GitHub is the whole story.
It is a deliberately curated entry point into a larger body of work:
- institutional AI strategy;
- responsible AI memory and context systems;
- higher-education technology;
- accessibility-minded interfaces;
- full-stack software;
- infrastructure;
- compilers and systems work;
- creative technical practice;
- and the human consequences of software decisions.
The public profile is not the evidence package. It is the front door.
The real assessment should happen through conversation:
- show the artifact;
- explain the context;
- discuss the tradeoffs;
- identify the risks;
- describe how AI was used;
- clarify what remains private and why;
- and test whether the reasoning holds up.
That is a much stronger signal than a contribution graph alone.
A More Human Assessment Model
In the AI era, I think the best assessment processes will become less obsessed with inspection and more committed to accountable conversation.
That does not mean lowering the bar.
It means raising the quality of the evidence.
It means recognizing that serious work often includes invisible labour: maintaining systems, protecting privacy, making interfaces accessible, documenting decisions, repairing trust, mentoring others, preserving context, and choosing not to build harmful shortcuts.
It means evaluating not only whether someone can produce, but whether they can reason responsibly about what production means.
That is the kind of assessment I want to see more of.
Not just:
Show me what you made.
But:
Help me understand how you think, what you considered, what you protected, what you learned, and why the system deserves trust.
That is where I think hiring, research collaboration, institutional leadership, and AI-assisted work all need to go.
Less performative inspection.
More authentic assessment.
More evidence with context.
More conversation.