By Mykhailo Marunchak

In today’s tech landscape, AI isn't just a tool; it’s a partner in innovation. Over the past months, we’ve explored how AI can fundamentally change how we design and build systems. Our most recent experience with designing a memory aid for individuals with cognitive challenges has shown just how transformative this process can be.

From Core Functionality to Scalable Design

The journey began with a clear focus on core functionality: converting real-time speech into text and processing it through multiple large language models (LLMs) working in parallel. This design is the heart of the system and serves as the foundation for all other capabilities. Speech-to-text translation, which serves as the input layer, was implemented with a sharp focus on accuracy and real-time responsiveness. It captures spoken words and transcribes them efficiently, enabling immediate downstream processing.

Once transcribed, the text is fed to three distinct LLMs, each with a specialized role:

  • Hint Generation: The first LLM generates conversational hints to aid the user during live interactions.
  • Fact Management: The second LLM updates the user’s memory by managing recent events and integrating them into the user’s biography.
  • Feedback and Calibration: The third LLM analyzes the conversation flow, providing feedback on timing and content to enhance system performance.

These models operate in parallel, ensuring no unnecessary delays. Key to this design is the asynchronous, non-blocking nature of the code. Every LLM works independently while seamlessly contributing to the overall functionality, allowing the system to maintain real-time performance without bottlenecks. This level of parallelism is not trivial to achieve, but it was critical for creating a system capable of supporting dynamic, real-time interaction.

The Role of Incremental Development

The system didn’t spring into existence fully formed. Instead, it was built incrementally, starting with the simplest working version and gradually adding complexity. This approach allowed us to evaluate and test every piece thoroughly before moving on. For example, we started by ensuring the speech-to-text engine worked flawlessly. Only after achieving consistent results did we move on to integrating the LLMs, one at a time.

Each new capability was tested in isolation and within the broader system to ensure it met our design goals. The AI’s ability to generate functional, high-quality code quickly was a significant advantage, but it also meant that each interaction with the AI had to be carefully crafted. The prompts guiding the AI needed to be precise, and the outputs required rigorous review to ensure they aligned with the design intent.

Frequent testing wasn’t just about catching errors—it was about understanding how each new layer interacted with the existing ones. The modular nature of the design, supported by incremental development, allowed us to spot potential issues early and address them without disrupting the entire system.

Collaboration with AI

While AI played a central role in accelerating development, the human contribution was equally critical. Every interaction with the AI was an opportunity to refine the system. This collaboration required a constant balancing act: giving the AI enough direction to produce meaningful outputs while allowing it the flexibility to suggest novel solutions.

Designing asynchronous functionality was one area where human oversight proved indispensable. Ensuring that the LLMs could operate independently without blocking one another required a deep understanding of concurrency and parallelism. The AI was a powerful tool for generating code, but it relied on the human designer to ask the right questions and guide it toward the desired outcome.

Customer Understanding: The Missing Piece in Traditional Development

Traditionally, many software developers have accepted an isolated role, focusing on implementing code without truly understanding the customer’s needs and challenges. I’ve worked with engineering leaders who insisted that understanding the customer wasn’t their job—they only needed a well-defined specification from the product team. They often justified this separation by claiming that the cognitive load of thinking about the customer’s needs and writing code simultaneously was too great.

This mindset has always been flawed, but AI has put it firmly in the past. In an AI-driven development process, intimate customer insight is no longer optional—it’s essential. AI can amplify the pace and efficiency of development, but without a deep understanding of the customer’s problems and the outcomes they need, the system will lack purpose and direction.

The old notion that engineers don’t need to think about the customer is a relic, destined to join the dinosaurs. Modern development demands that we integrate customer insights into every stage of the process, from design to implementation. With AI accelerating the more routine aspects of coding, developers now have the bandwidth to engage deeply with these insights. This evolution isn’t just a benefit—it’s a necessity.

A New Productivity Standard

The results of this process have been remarkable. By combining AI’s speed with human judgment, we’ve achieved at least a 4x improvement in productivity. Tasks that would have taken weeks or months were completed in days, thanks to the AI’s ability to handle repetitive coding work while the human team focused on higher-level design and user needs.

However, this productivity boost comes with an important caveat. The human role in AI-assisted development isn’t diminished; it’s elevated. Understanding the needs of the user, maintaining a clear vision for the design, and ensuring that every piece of the system works toward those goals are responsibilities that AI cannot shoulder.

The Human Element in AI-Led Development

If there’s one key takeaway, it’s this: AI is a powerful collaborator, but it cannot replace human judgment, creativity, or understanding of user needs. The designer’s ability to foresee challenges, set clear goals, and maintain focus is what transforms AI from a code generator into an indispensable development partner.

As we continue refining our system, the lessons we’ve learned are clear. AI can reshape the way we develop software, but its true potential lies in the partnership between human designers and machine intelligence. Together, they can create solutions faster, smarter, and more effectively than ever before.