In this special interview with Joe Leo, the Founder and CEO of Def Method, we discuss the launch of Phoenix, a new service to continuously generate self-healing tests for Ruby on Rails applications. We also look at the schools of programming forming around generative AI, bringing the joy of Ruby to AI development, and the importance of staying curious in an ever-changing technological landscape.

Contents

• Introducing Phoenix for Rails

• Interview with Joe Leo

  • Phoenix

  • Architecture

  • Def Method

  • The Ruby AI Podcast

  • Future of Ruby AI

  • Closing Thoughts

• Additional Resources

Introducing Phoenix for Rails

Phoenix is an AI-powered tool built by Def Method to solve one of the hardest problems in maintaining legacy Ruby on Rails applications: automated testing. Many Rails codebases still run without sufficient test coverage, which makes refactoring, scaling, and shipping new features risky and expensive. Phoenix tackles this head-on by generating high-quality automated test cases tailored to the realities of legacy code. Instead of forcing developers to untangle years of technical debt manually, Phoenix helps teams create a safety net of tests that builds confidence and unlocks momentum.

Phoenix uses a reactive AI approach to test generation, meaning it doesn’t just produce a static set of tests and stop there. Instead, it continuously adapts to the Rails codebase as it evolves, responding to changes in models, controllers, and business logic. This reactive loop allows Phoenix to generate tests that reflect real usage patterns and refine those tests when the code shifts. The result is a living suite of automated tests that grows alongside the application, offering teams an always-current safety net that keeps pace with both new features and ongoing refactors.

Interview with Joe Leo

This week I had the privilege of interviewing Joe Leo, Founder and CEO of Def Method, a New York City-based software development firm dedicated to building high-quality, thoughtfully crafted software. A long time leader in the Ruby community, Joe is an accomplished conference speaker, co-author of The Well Grounded Rubyist, and co-host of The Ruby AI Podcast. Previously, he was a member of the Tech Advisory Board at The New York Foundling and an organizer of the Gotham Ruby Conference. As one of the first people in the community to recognize the potential of generative AI with Ruby, Joe brings advanced AI integrations to clients and services at Def Method, highlights the leading insights on the Ruby AI Podcast, and sponsors the Artificial Ruby meetup.

Since its inception in 2014, Def Method has been a cornerstone of the Ruby and Rails development communities, specializing in test-driven, robust engineering practices for high stakes technical challenges. This month, Def Method is launching Phoenix, a service to seamlessly add AI-generated tests to Rails applications, adhering to conventions, best practices, and domain expertise to ensure your tests reflect real-world Rails logic. Connect your repository or upload your application, and Phoenix will analyze your models, controllers, and business logic to understand your requirements and generate complete test coverage for Rails applications of all ages and versions.

So here it is! Presenting Joe on:

Phoenix

This month you’re launching Phoenix, the first AI-assisted coding agent specifically tailored to Ruby on Rails. What can you tell us about the service? What features and capabilities are you most excited about? What is the long term vision for Phoenix?

Joe: I’m most excited how engineers will use this new power. Imagine having a backstop of tests constantly being generated, updated, and evolved as your code evolves. It changes everything, including our relationship to TDD and testing in general. TDD engineers can still TDD for design but know that they don’t have to check every branch of logic. Vibe coders can vibe and know that Phoenix will have their back with automated tests.

But I’m also excited about the reactive element to Phoenix. I’ve long wanted Ruby to be a player in how AI is delivered and utilized across all of software development. This is a new way to use AI - not as a chatbot, not as an agent, but as a complete system that never sleeps and always watches over your code.

Architecture

What are the underlying technologies and models powering Phoenix? What challenges did you face building the service, and what have you learned from building an AI-powered customer-facing application?

Joe: One of the biggest challenges we faced early on was building in Ruby! Originally there weren’t many tools to choose from, and we ended up building the first version mostly in Python. Now, of course, we have a huge footprint in Ruby that gets bigger every day.

Though the software powering Phoenix is model-agnostic, we’ve had the best results using the Claude Sonnet series of LLMs. We make heavy use of RubyLLM, GitHub Actions and the GitHub API, Rails, and a Python CLI.

Def Method

You gave a great talk at RailsConf on wrangling the complexity of large, legacy Rails applications. How can Rails developers best leverage those lessons when building with generative AI to avoid unnecessary complexity? In over a decade of working with Def Method clients, what are some of the other impactful insights you’ve gained on the development process? How have recent clients embraced AI in their applications and organizations?

Joe: I’m not sure the unnecessary complexity I spoke about will hold a candle to the amount that’s being generated today by engineers using AI code gen tools. Engineers build everything faster than they used to and we spend much less time reading and reviewing the code that they write. It’s a recipe for spiraling complexity over the life of a successful project.

Regarding process, I think two schools of programming with generative AI are starting to emerge: Stewards and Sprinters.

The Stewards still value object oriented design, emergent design, and test-driven design above all else. They use generative AI to build but they read every line, ensure the right patterns are in place, and refactor regularly. Stewards do not code with agents, preferring just one stream at a time using Cursor or Windsurf or good ol' ChatGPT.

The Sprinters throw some of the old patterns out the window. It’s not that they don’t value design - nearly all of them do - it’s just that they won’t let a less-than-optimal design get in the way of the ruthless speed of generative AI. Sprinters are the ones with five Codex or Claude Code agents running at one time.

At Def Method we have more Stewards than Sprinters. But that doesn’t mean it’s the best way! Some of the best non-Def Method engineers I’ve ever met are firmly in the Sprinter school. We’re still at the beginning of this journey and it will be fun to see how these processes get shaped over time.

The Ruby AI Podcast

Congratulations on eight exciting and informative episodes of the Ruby AI Podcast! What can we expect from the podcast in the coming months? What insights and key takeaways have stuck with you long after the episodes aired? How have your views on AI changed since you started recording the podcast?

Joe: Thank you! I’m learning something new all the time. The biggest thing that Valentino and I are trying to uncover is: what is Ruby’s unique contribution to this new world of artificial intelligence? It’s wonderful to talk to people about what they’re building and how they’re leveraging AI. That’s a great place to start. But ultimately I want to know what things are we, as Rubyists, doing with or for AI that nobody else could do? What’s the Ruby way of building AI?

I don’t think we’ve answered that yet, which is good! It means we have lots more people to talk to and work to do.

Future of Ruby AI

What do you foresee as the future of Ruby and AI? What could Ruby or Rails do better to build and support AI applications? What AI tooling, technologies, or libraries would you like to see built by the Ruby AI community?

Joe: Ah, that’s the question I ask all of my guests so I don’t have to answer it! OK, I’ll give it a shot.

Ruby’s future in AI is in how it gets applied. We’re not going to start building LLMs with Ruby as anything other than a toy project. But LLMs are not AI. AI is vast and we’ve just begun scratching the surface of its use cases. It’s perhaps not the language itself but the community of like-minded engineers and enthusiasts who will ultimately make an impact on artificial intelligence as we know it today. That could be in building new ways to interact with AI, like Phoenix does for Ruby on Rails developers. Or it could be in wrangling the agents we use to develop software so that they are more useful and do less harm. It could simply be in building useful interfaces that make programming with AI less onerous and more joyful. That’s why all of us fell in love with Ruby in the first place - it brought us so much joy! And I think that’s the opportunity we have again today with AI.

Closing Thoughts

What skill sets, technologies, and disciplines do you think Rubyists should pursue to level up their careers and expertise in the age of AI? Is there anything else exciting you would like to share?

Joe: As an engineer, you’ve gotta be out there using AI in all of its imperfect forms every day. Don’t wait or hide behind excuses like, “It’s not smart” or, “It makes too many mistakes.” I could have excused you for saying that until we started the podcast. Now I know for a fact that the biggest names in Ruby are using it every day. And they’re learning how to deal with the imperfections and wring some incredible efficiency and creativity out of their tools. If you don’t know where else to start, just use ChatGPT and your favorite code editor and start plugging away. Look at and try the many useful AI tools that are highlighted in this newsletter!

At the end of the day, it comes down to a mindset. You don’t have to love AI or sing its praises every day. You do need to understand that it’s a tool, and it’s a tool being leveraged by everybody. Take a beginner’s mindset with you and strive for curiosity and learning every day.

Additional Resources

Joe Leo is the founder and CEO of Def Method and regularly posts updates on LinkedIn and X/Twitter.

Def Method is a New York City-based software development firm dedicated to building high-quality, thoughtfully crafted software. Learn more on LinkedIn, X/Twitter, Github, or YouTube.

Phoenix infuses your Rails application with love through AI-generated test coverage, creating a relationship where your code truly cares for you. Join the waitlist or subscribe to the Phoenix newsletter.

The Ruby AI Podcast explores the intersection of Ruby programming and artificial intelligence, featuring expert discussions, innovative projects, and practical insights. Previous episodes include:

• From Prompts to Programs: Mastering LLMs with DSPy.rb and Vicente Reig

• Real-World Ruby AI: Practical Systems That Work with Amanda Bizzinotto

• Contracts and Code: The Realities of AI Development

• Rails After the Robots: Chad Fowler on AI as the Next Abstraction

• Evaluating LLMs with Leva and Keiran Klaassen

• Roasting Ruby AI Workflows with Obie Fernandez

• Active Agent with Justin Bowen

• Sublayer and Artificial Ruby with Scott Werner

• Beyond Chat: Phoenix Tests, Ruby Agents & the AI Tipping Point

The Well Ground Rubyist, Third Edition by David A. Black and Joseph Leo is a beautifully written tutorial that begins with your first Ruby program and takes you all the way to sophisticated topics like reflection, threading, and recursion

Artificial Ruby is a community of Rubyists who meet regularly in NYC to explore novel approaches to AI development using Ruby.

That’s all for now! Thank you to Joe Leo for taking the time to answer these questions and share Phoenix with us. Stay tuned for the next edition of the Ruby AI Newsletter. Be sure to reach out if you have any stories, content, jobs, or events you would like to see featured.