05. From Writing Unit Tests to Delivering Entire Tasks: My Journey Through the Evolution of AI Agents

In late 2022, shortly after ChatGPT became publicly available, I was working at Rakuten on an SDK for large-scale web data collection. Once I heard it could write code, I immediately tried asking it to add unit tests.

The results were better than I expected. Most of the generated tests became usable after minor edits, saving me considerably more effort than writing them from scratch. Later, I also asked it to generate tests for React components. The code often looked convincing, but it frequently failed to run.

Something could appear perfectly reasonable when read in isolation and still fall apart inside a real project. That was the first time I clearly saw the distance between model capability and software delivery.

My first practical experience with AI-assisted software development was remarkably ordinary: using ChatGPT for unit testing. Yet that small experiment became the starting point of a much larger shift—from unit tests to autonomous task delivery.

At the time, I thought AI’s most practical value was helping engineers write less repetitive code. A few years later, it can inspect an entire repository, examine logs, create a repair plan, modify files, and run tests.

Looking back at AI agent evolution over the past few years, the biggest change is not that models have become better at answering questions. It is that they have begun leaving the chat window and entering real working environments.

It Started with an Unreliable Chat Window

When I first began using ChatGPT, it quickly became a tool I opened every day. While many people were still discussing what a chatbot could do beyond conversation and simple questions, I was already using it to add tests, explain code, and generate components.

I never fully trusted it, though.

In 2023, ChatGPT Plus began offering web browsing. To adapt to this new way of working, I forced myself to ask AI first whenever I needed to research something, then repeated the same search in a browser.

Its answers often needed verification. Sometimes the sources were unclear; other times, incorrect information was presented with impressive fluency. I still liked its summaries because they handled the most time-consuming part of research: the first round of filtering.

My workflow during that period never changed at a fundamental level. I asked ChatGPT a question, and it gave me some content. I copied that content into an editor or terminal, then decided whether it worked.

AI could assist with work, but it could not take over work.

Meanwhile, major technology companies rushed into the AI race. Microsoft had invested early in OpenAI and began integrating related capabilities into Edge, GitHub Copilot, and Windows. Google used Gemini to connect Search, Workspace, Android, and its cloud services. OpenAI continued adding browsing, tool use, multimodal input, and more advanced reasoning to ChatGPT.

The announcements were exciting, but I had only one standard for deciding whether a development mattered:

Did it change how I worked every day?

From Producing Answers to Taking Over Tasks

There is a clear boundary between conversational tools and working agents. That boundary also marks a broader change in AI-assisted software development.

A conversational tool tells me how code should be changed. I still need to find the file, copy the code, fill in missing context, run tests, and paste the resulting errors back into the chat.

AI coding agents such as Claude Code and Codex can enter the project directly. They can inspect repositories, search for related implementations, examine logs, propose repair plans, modify files, and use tools in the real environment to verify results.

These capabilities turn isolated code generation into coding agent workflows that can carry a task from investigation to delivery.

The questions raised by agents can also become valuable. Sometimes, a question forces me to reconsider a requirement, an edge case, or an assumption I had treated as settled. At that point, AI is no longer only answering questions. It is participating in how the problem gets defined.

During the first half of 2025, I was still using the Code Companion plugin in Neovim for code completion. Even then, I felt as if my programming ability had moved to another level. I could type a few characters, press Tab, and let the tool complete the rest of the line—or several lines at once.

Take the following LanguageSwitch.tsx component as an example. I wrote only the function declaration. Copilot and the Neovim plugin filled in everything else.

The first version caused a few problems and could not be used directly. After I gave Copilot the error message, it identified the issue and proposed a fix. The code below shows the corrected version. Code Companion could then apply the change directly to the current file.

// LanguageSwitch.tsx
'use client'

import { Languages } from 'lucide-react'
import { usePathname } from 'next/navigation'
import Link from '@/components/Link'
import {
  getLocalizedPath,
  getLocaleFromPathname,
  getOppositeLocale,
  getTargetPath,
} from '@/lib/i18n'

// I only wrote the function declaration below.
// Everything else was completed with Tab—and it works.
export default function LanguageSwitch() {
  const pathname = usePathname()
  const isEnglish = pathname === '/en' || pathname.startsWith('/en/')
  const targetPath = getTargetPath(pathname)

  return (
    <Link
      href={targetPath}
      className="inline-flex items-center gap-1.5 text-sm font-semibold text-gray-900 underline-offset-4 hover:text-primary-500 hover:underline dark:text-gray-100 dark:hover:text-primary-400"
      aria-label={isEnglish ? 'Switch to Chinese' : 'Switch to English'}
    >
      <Languages className="h-4 w-4" aria-hidden="true" />
      {isEnglish ? '中文' : 'EN'}
    </Link>
  )
}

If the feature depended on other files, I could ask Code Companion to load them. Sometimes, however, I still had to paste code or earlier context directly into the conversation to help it understand the relationship.

# Read the files below
./src/LanguageSwitch.tsx
./src/lib/i18n.ts
./src/components/Link.tsx

Writing code this way felt impressive. As long as I kept providing ideas and context inside the same environment, much of the output could be used directly.

When Claude Code arrived, I felt this transition more strongly.

Early versions of Claude Code still required frequent user approval to keep AI behavior under control. Those confirmations could be bypassed through less restrictive permission settings, as some autonomous task agents do. Even with those limitations, Claude Code could already reason through file contents and solve problems on its own.

It could also work directly in the terminal. I no longer needed to keep moving information between a chat window and an editor. Since I was already comfortable with tmux and Neovim, the learning cost was low. I could launch the tool inside an environment I understood and begin working with it.

Before agents became mature enough for everyday use, many developers treated VS Code or a JetBrains IDE as the center of software development. As agentic tools spread, the terminal became busy again.

On the surface, it looked as if developers had collectively returned to a more primitive interface. The opposite was happening.

Anthropic also introduced Agent Skills, which I found especially useful as someone with fairly high standards for generated code. Claude Code and OpenAI Codex could already produce usable code that often required little modification. Skills made their capabilities composable.

I think of them like upgrades in a game. One Skill improves your aim. Another increases movement speed. A third improves precision. To maximize output, you combine several Skills rather than depending on one.

Over time, Skills such as caveman and using-superpowers became part of my daily workflow. I also noticed that I was becoming increasingly dependent on the improvements they provided.

They consumed more tokens, but when the output met my standards, the cost still felt worthwhile.

Agents do not feel like an early technology that forces users to struggle through primitive tools before obtaining any benefit.

They feel more like The Matrix: load a piece of code, and suddenly you know how to fly a helicopter.

New Concepts Arrive Faster Than We Can Learn Them

After ChatGPT became popular, “prompt engineer” quickly emerged as a fashionable job title. Early models had shorter context windows, and keyword-heavy prompts became common in parts of the community.

Adding, removing, or rearranging one word could noticeably change the result. People searched for more reliable wording while trying to reduce context consumption.

At the time, the value of prompt engineering was direct: use less input to produce more controlled output inside a limited context window.

The field then accelerated.

In 2023, function calling gave models a structured way to connect with software tools. Retrieval-augmented generation and longer context windows entered more products.

Near the end of 2024, Anthropic released the Model Context Protocol (MCP), providing a shared protocol for connecting agents with external systems and data.

In 2025, AI coding agents such as Claude Code, Codex, and Cursor shifted attention away from whether a model could write code. The more important question became whether it could complete tasks inside a real engineering environment.

These AI engineering tools also changed what mattered in practice. Tools, permissions, sandboxes, filesystems, tests, context compression, and human approval began determining the upper limit of an agent’s capability.

Prompt engineering still mattered, but it was no longer enough. The goal was not to write one perfect prompt. It was to build an environment in which an agent could act reliably.

I think of this as a form of orchestration engineering.

By 2026, people had begun discussing “loop engineering”: how to design execution cycles, verification, failure recovery, and context updates for agents.

New terms keep piling up, each presented as the next essential skill.

The problem is straightforward: engineers have limited time, but new concepts do not have a pause button.

Experiment Regularly, but Do Not Rush into Deep Study

During a period of rapid technological change, I think refusing to study every new idea in depth is a useful habit.

This does not mean rejecting change. It means allocating time carefully.

I still follow improvements in reasoning speed, accuracy, formatting, and tool use. I also test mature products against real work. But I do not read large numbers of papers or source code about memory mechanisms and agent theories that have no practical use case. I do not immediately rebuild an existing system because a new framework has become popular.

The same pattern has repeated over the past few years. A complicated orchestration method can be presented as the future standard, only to become unnecessary several months later because the underlying model has improved.

Prompt engineering provides one example. One-shot and few-shot techniques remain useful, but modern models no longer need them in the same way to understand many task definitions.

Earlier investment does not always produce higher returns.

When I encounter a new tool, I look for several signals. Has it existed for more than three months? Has it received enough coverage and discussion? Have multiple users reproduced meaningful productivity gains in real work?

For a GitHub project, I also examine forks, maintenance activity, and community participation. More than one thousand forks is a rough threshold I often use. It shows that a project has received attention, but it cannot replace direct evaluation.

If a tool passes this first filter, I normally try it for three days. Three days is enough to determine whether it solves an existing problem and whether it deserves a permanent place in my toolbox—or whether it offers only a brief moment of novelty.

Regular experimentation keeps me from falling behind. Waiting for a technology to mature keeps me from becoming an unpaid participant in every new trend.

This has become my answer to a practical question: how should software engineers learn AI tools when those tools change every few months?

I invest more time when one of three things happens:

• Industry practice begins converging around the concept.

• A complicated capability becomes available through a stable interface, reducing learning cost.

• A genuine bottleneck appears in my workflow, and deeper knowledge would produce a direct return.

Unless researching these technologies is the work itself, there is little reason to rush to the front of every new wave.

Building Organizational Knowledge with Agent Skills

Agent Skills changed how I think about extending an agent.

A Skill is not merely a longer prompt. It can contain operating instructions, scripts, reference material, and review standards that an agent loads when needed. The model provides reasoning, while the Skill preserves a tested way of working.

I have used many Skills. In particular, caveman, using-superpowers, ponytail, and frontend-design have produced significant returns in my daily work.

caveman reduces excessive politeness, repeated explanations, and empty summaries. It preserves a high-density communication style: keep conclusions, evidence, code, and next steps; remove everything else.

This reduces reading time and prevents unnecessary language from consuming the context window.

using-superpowers preserves experience in decomposing complex tasks and coordinating tools. Large engineering problems often exceed what basic text generation can handle reliably.

This type of Skill tells the agent when to use subagents, automated scripts, testing tools, or external systems, turning a team’s problem-solving process into an executable workflow.

More importantly, it can preserve substantial documentation and provide useful context for later engineering work. Its main drawback is token consumption.

ponytail preserves the perspective of a senior reviewer. It checks edge cases, performance bottlenecks, security risks, and code standards while pushing the implementation toward the smallest amount of code needed to complete the task.

When I need to write code every day, ponytail is one of the Skills I use almost automatically.

frontend-design addresses a common problem in AI-generated interfaces: inconsistent styling and weak visual coherence. It can apply Tailwind conventions, component reuse, responsive layouts, and custom color systems.

I mostly use it for demos and high-fidelity templates.

These Skills are not first principles. They are closer to procedural knowledge and organizational memory. Models will change. Tools will be replaced. Review standards, design rules, and collaboration processes that have survived real work can remain.

Building organizational knowledge with Agent Skills means turning those standards and processes into resources that agents can repeatedly apply.

When a team continuously develops Skills while preserving usage notes, working examples, and lessons learned, an agent gains more than additional features. It gains access to the way that organization has learned to solve problems.

Compared with chasing a short-lived framework, I believe this kind of accumulation deserves more investment.

I Am Not Afraid Only Because AI Can Write Code

As agents become more capable, layoff news feels more personal.

My concern is not limited to AI directly taking my job. I also worry that younger engineers can use search and agents to acquire output capabilities that once required years of experience.

The relationship between AI agents and junior software engineers makes this change especially visible. Agents can provide context, explain unfamiliar systems, review code, and help engineers investigate problems that once required frequent guidance from senior colleagues.

This is one of the clearest examples of how AI agents are changing software engineering. They do not merely increase individual output. They change how quickly engineers can acquire context and how much experience a task appears to require.

I felt this clearly while working with my team at Fast Retailing. Junior engineers sometimes produced excellent code. They did not have the same depth of experience, but agents could provide them with more context. Through research and learning, they could still submit solid pull requests.

To be honest, even after working in software for many years, I cannot claim that a pull request written entirely by me would always be as reliable as something produced by a junior engineer working with an agent.

The same applies to architecture.

If an engineer can obtain enough context, AI can reduce some of the advantages traditionally associated with seniority. Look at the design documents, flowcharts, sequence diagrams, and object diagrams an AI can produce in tens of minutes. Much of the content is broadly accurate.

A human team might spend an entire day—or sometimes a week—creating the same documents. They finally appear ready to guide development, only for several problems to emerge during review and discussion.

A senior engineer usually understands the existing system better. Their design may fit real requirements more closely and avoid side effects caused by missing redundancy or incomplete context.

Comparing the resulting documents can reveal whether that extra time produced enough value.

The difficult part is that agents also become more “senior” when we give them enough context.

What happens if junior engineers gain access to the same amount of context?

The answer is not difficult to imagine.

Media reports increasingly claim that one experienced engineer using AI can perform work that once required an entire team. At the same time, hiring markets have shown signs of contraction in parts of software engineering.

I have also begun seeing more Forward Deployed Engineer roles. These positions place engineers inside real customer environments and combine technical execution, business understanding, and delivery responsibility.

This suggests that companies may no longer need only people who can write code. They need people who can turn ambiguous problems into working outcomes.

This shift will shape software engineering careers in the age of AI. The advantage may move away from producing code quickly and toward understanding business constraints, defining ambiguous problems, evaluating trade-offs, and taking responsibility for delivery.

Forward Deployed Engineer roles offer one early signal. Whether this becomes a dominant model remains uncertain, but it points toward a broader question about the future of software engineering:

What kind of value should engineers provide when code itself becomes cheaper to produce?

Junior engineers face a related risk. AI lowers the barrier to completing basic tasks, but it may also reduce the incentive for companies to create entry-level roles where people traditionally learned through supervised work.

If that entrance becomes narrower, younger engineers will not face less pressure than senior ones.

For now, I still believe humans carry more responsibility in three areas: taste, judgment, and accountability.

Taste determines which result is worth keeping. Judgment operates when information is incomplete and rules are unclear. Accountability means someone must own the consequences of an investment, hiring decision, product strategy, or system failure.

Agents can analyze, recommend, and generate. Humans still carry the consequences.

That is not a permanent guarantee. It is only the boundary I can see today.

The telephone industry offers a limited historical comparison. Early telephone exchanges created large numbers of manual operator jobs. Automated switching systems later replaced part of that workforce over several decades while creating work in equipment engineering, maintenance, and network management.

This history does not prove that AI will create more jobs than it removes. It shows only that technology tends to redraw work rather than preserve existing job titles unchanged.

The hardest question today may not be whether programmers will disappear.

It may be which work will remain worth giving to programmers.

I No Longer Build Five-Year Technology Roadmaps

These changes have already affected how I plan my career.

Engineers once built five-year technology roadmaps: learn a language, move into an architectural specialization, then accumulate experience in selected frameworks and systems.

I find it increasingly difficult to believe that a fixed roadmap can survive the next five years.

Instead, I am building a modular skill library.

Some modules contain durable engineering foundations, such as system design, debugging, testing, security, and performance. Others contain business knowledge developed through long-term exposure to real systems and users.

Another group connects those foundations with AI: tool selection, task decomposition, permission control, result verification, and Skill development.

Modules can be replaced and recombined. When an AI pattern becomes stable enough to matter, I do not need to restart my career. I only need to connect it to an existing capability system.

I do not chase every bubble because I want to preserve time for business knowledge and real problems. I keep experimenting because I want to recognize an important change when it arrives.

In 2022, I asked ChatGPT to add several unit tests to an SDK. I still had to copy, edit, and run the output manually. A few years later, I could hand an entire task to an agent and let it inspect the environment, perform the work, and verify the result.

As tools take responsibility for more execution, my own responsibility becomes clearer:

Decide what is worth doing. Judge whether the result is acceptable. Preserve the experience that deserves to survive.

That may be the most important lesson I have taken from AI agent evolution.