Great Books for AI Engineering

10 books with valuable insights about AI science and engineering

Great books for AI Engineering — Plus ‘Brave New Words’ (Image is Author’s own work)

A few years ago I recommended 21 books in Great Books for Data Science and Great Books for Data Science 2. Since then a lot has changed. While data scientist and machine learning engineer continue to be necessary functions in large companies, the specific role of AI engineer emerged prominently in 2023 with the release of GPT-4. This new role is still finding its footing, and the most important skills and tools to build robust, useful systems are also in flux.

Most of the books in this set have been released in the past few years, as people have begun to integrate AI tools to build and improve systems. As we learn to adapt to the new roles necessary to build with AI, having access to a broad set of perspectives and tools is a huge advantage. I hope this set of books can help others working to make the next set of advancements in AI engineering possible.

The Alignment Problem

By Brian Christian

The alignment problem refers to a gap between the intention of instructions and exhibited behaviors in AI systems. In this book, Brian Christian brings together technical insights with philosophical depth, examining how our attempts to align machine behavior with human values reveal fundamental questions about intelligence and ethics.

What makes this work particularly compelling is its exploration of real-world cases where seemingly well-designed systems produce unexpected outcomes — not due to bugs or errors, but because of deeper challenges in specifying intentions mathematically.

As an example, one section explored teaching children to clean up their room using candy. The children did become adept at cleaning. But they also learned that they only received candy if the room went from messy to clean, so they made sure to mess up the room far more often.

Most of the fears we have around AI can be traced back to simple misalignments, which in isolation can be easy to address when you understand the problem space and the tools available. This book serves as both a warning and a guidebook for ai engineers working through problems that require a high degree of alignment.

Brian Christian

Brave New Words

By Salman Khan

This is a well timed book by the creator and founder of Khan Academy. Educators and students are at the forefront of the disruptions of AI, but according to this book, the impact will be immensely positive. At times the book focuses a little too narrowly on how AI is being applied at Khan Academy through their chatbot, Khanmigo, but the implications extend to all teachers and students who will share their learning spaces with AI tools.

In Brave New Words, Khan explores how to make AI that works as an accelerator of learning and a way to increase the standards in the education system. The discussions center around designing systems that:

Meet students where they are and advance at their own paceAdapt the learning process to cater to individual interestsNever provide answers outright unless necessary to aid learningUp-level writing quality and grammar

It’s an exciting topic and a refreshing perspective, especially given how many school systems (and states) have banned this new technology outright, over fears students won’t learn the material.

Brave New Words by Salman Khan: 9780593656952 | PenguinRandomHouse.com: Books

Human Compatible

By Stuart Russell

Human Compatible provides a different perspective on the alignment problem, this time attempting to start from first principles to expose the complex nuances of alignment and ways to approach AI that can mitigate some of the risks. The book follows a logical structure, first discussing the nature of intelligence and the ways intelligence resulting from evolution differs from artificial intelligence. It then follows with a discussion of the dangers of developing AI without sufficient safeguards, and the ways that even simple objectives can lead to catastrophic misalignment (like the fact that AI will likely learn to keep itself turned on, since it can’t achieve its objectives when not operating).

The remainder of the book focuses on the nuances of alignment and designing systems that produce the best outcomes for humanity. This includes thinking about AI that can learn the preferences of individuals they serve, while balancing those preferences against the collective good. For example, there is some discussing about AI usage by criminals, and the extent to which an AI should be able to optimize for a user’s preferences when those preferences also affect others.

Overall, this is a relatively deep exploration of the impacts of AI with a strong technical basis. I liked how the author approaches these topics without straying too far into philosophy and ethics, instead focusing the discussion around individual preferences, control, and alignment.

Human Compatible by Stuart Russell: 9780525558637 | PenguinRandomHouse.com: Books

Hidden Games

By Erez Yoeli

Hidden Games presents game theory as a fundamental framework for understanding complex interactions, whether between humans or artificial systems. The book systematically explores how different types of games — from simple competitions to complex resource allocation — can model real-world scenarios. Each chapter builds understanding by introducing a game structure, examining its presence in everyday situations, and revealing the optimal strategies that emerge under different conditions. For AI engineers, these insights prove invaluable when designing systems that must navigate multi-agent environments or optimize for multiple competing objectives.

There are two books about game theory in this list. I found these books incredibly applicable to the world of AI, because games are powerful models for navigating complexity. Useful AI systems will need to be able to successfully perform in a wide range of scenarios, and game theory is the ideal tool for breaking problems down into a computationally accessible form.

Often the challenge is identifying the game — fitting nuanced free-form scenarios to the appropriate mathematical problem. For example, people who seem cold and calculating may be relying on a model that is too narrow, and fails to consider social responses to their behavior. This is the same reason (successful) poker players play the opponent — not the cards.

Hidden Games

Guardrails

By Urs Gasser & Viktor Mayer-Schonberger

Guardrails is another discussion of control in AI systems, one that is more relevant to the tools we interact with today. In existing AI systems, businesses use the term ‘guardrails’ to refer to checks that operate on requests and responses, serving to block malicious requests and switch undesirable responses with some default like ‘I can’t provide that information’. In this book however, guardrails is a much broader term, referring to the design of control systems that shape and direct AI to the benefit of humans.

The study of control in information systems has been a hot topic since long before the release of GPT 3. For example, studies discussed in Guardrails that have found misinformation spreads faster and is shared more broadly than fact based content. Instead of looking only at traditional guardrails that rely on humans to build validation checks for AI, the authors also explore the use of AI in solutions to the larger human-driven propagation of falsehoods. While it’s tough to chart a course between global security and protected free-speech, it does make sense that new AI tools will be heavily involved.

While I was expecting Guardrails to focus primarily on control in AI systems, I was surprised and impressed by the wide range of topics discussed in this book. It’s an important read, that will help as we learn to apply our newest tools to our most difficult problems.

Guardrails

Optimal Illusions

By Coco Krumme

Optimal Illusions examines a central paradox in artificial intelligence development: our drive for optimization often leads to systems that are brittle and inflexible. The book methodically explores how optimization, while powerful within defined parameters, can create vulnerabilities when conditions change. Using examples ranging from economic systems to biological evolution, the author demonstrates how excessive optimization can actually reduce a system’s ability to adapt and survive in dynamic environments. This understanding becomes particularly crucial for AI engineers working to build robust and adaptable systems.

I initially found this book intriguing but a little self-righteous. Rather than providing a systematic analysis of optimization’s limitations, the author chooses to rely on stories to illustrate broader patterns and potential pitfalls. The core idea is sound though, and can be likened to similar arguments in math and physics — optimizations are relative (i.e. to chosen axioms), and optimized systems are inherently fragile.

It’s not hard to apply the same principles to AI. Optimized AI systems work well within the confines of models they were optimized for. Break the assumptions, change the axioms, or alter the rules, and optimized systems flounder. And while we like to believe we can isolate our systems from an otherwise chaotic universe, the breakdown of optimizations is not a question of ‘if’ but ‘when’.

Although this book doesn’t present ready-made alternatives that we can easily incorporate into systems, thinking about the topics in this book may lead to better outcomes as we work to build more ‘general’ intelligences. I also like how Optimal Illusions provides a useful reframing of similar topics in The Black Swan and Antifragile (both by Nassim Nicholas Taleb).

Optimal Illusions by Coco Krumme: 9780593331118 | PenguinRandomHouse.com: Books

Playing with Reality

By Kelly Clancy

Playing with Reality looks at the use of game theory in building modern systems, it’s pitfalls, and how we can frame problems to devise better games that will incentivize behaviors that lead to better results. This is especially relevant in light of how AI systems learn, and the book goes into this in some depth.

One of the pitfalls of games that is discussed is the tendency to look at problems as zero-sum, which leads to adversarial behavior. Because games are discrete, isolated (i.e. simplified) models of reality, there are infinite ways to frame any problem, and this framing can dramatically change the equilibrium states. For example, a primary focus of this book is on expanding the scope of games to incorporate the interests of more players — leading to more altruistic behavior.

While the author doesn’t explicitly focus on the problem of control in AI systems, there is a lot of discussion of development of AI and the design of simulations used to train future models (and people). It’s a truly eye-opening book, and while the tone is cautious of the inherent limitations of games, it is also hopeful. It makes AI systems that could strategize for the benefit of large groups seem more attainable.

Playing with Reality by Kelly Clancy: 9780593538180 | PenguinRandomHouse.com: Books

Complex Adaptive Systems

By John H. Miller & Scott E. Page

When I put together Great Books for Data Science, I described Complex Adaptive Systems as ‘probably the most important book in this list’. That’s why it’s showing up again in books for AI engineering — it’s importance has only grown with recent discoveries and the release of new technologies.

The most interesting problems we face are often related to complex adaptive systems, or systems that exhibit emergent behaviors and non-linear adaptive responses. Biological forms can be modeled as complex adaptive systems at all scales, from the microbiome to the ecosphere. It’s not a stretch to see how the same principles apply to artificial intelligences, and the emergent properties and adaptations they exhibit as a result of interactions during training.

Complex Adaptive Systems is probably still the most important book on this list. It eases the reader into thinking about the world in a new way, and then provides tools and modeling techniques that can easily be applied across new problem spaces. Reading this book ultimately affected my approach to the world significantly, and it has continued to pay dividends while working with AI — helping me to see unrealized potential and unrecognized limitations of systems.

Complex Adaptive Systems

A Human Algorithm

By Flynn Coleman

There are a lot of books out today that offer speculation about either the massive potential benefits or existential threats presented by AI (superintelligence, the singularity is nearer, AI 2041 etc). I haven’t found these books particularly useful, and I was initially skeptical that A Human Algorithm would fall into this category, but there is significant depth in the topics covered.

The book presents AI in the broader context of how technologies have historically become integrated into society. These are patterns worth considering and we rarely get the opportunity to see the whole thread laid out. All the sections of this book take examples from history to help navigate current dilemmas that have arisen with new technologies. While there is ample speculation about the benefits and dangers of AI, it helps to see the precendents that have been set, and how decisions in the past have played out with regards to new innovations.

Overall A Human Algorithm is a great historical guide to the development and integration of technologies, with specific applications to modern AI. Although this book is more likely to be assigned in a liberal-arts curriculum than a computer science course, the author is clearly well informed and the breadth and depth of the topics covered is worth consideration for anyone working in AI.

Books – Flynn Coleman

From Data to Profit

By Vin Vashishta

Of the books listed, From Data to Profit is probably the most immediately applicable to anyone working in data science or AI engineering. As anyone working in technology knows, the promises of innovation are only made real through carefully guided product management and clear strategies for monetization. The ideas and questions presented in the books above will only become relevant as business leaders apply artificial intelligence to solve the right problems and create new value.

From Data to Profit is structured like a course, laying out the frameworks and business considerations you need to understand to build successful products. It is clear from the reading that ‘AI’ is not the product, AI is the principal tool businesses have for creating value and competing for market share. With this in mind the book discusses strategies for data and AI organizations to create maintainable processes, identify actionable opportunities, identify the right tools for the problem, and adjust the approach to a solution by monitoring the value it is driving.

While this book has immense value for people working in AI engineering, product management, and consulting today, there are sections that are universal and sections that may only apply in the near term. For the most part I believe that the contents will continue to pay dividends, and help elevate people into positions where they can build important products that leverage the most cutting edge technologies.

From Data To Profit

There is a lot of buzz in AI right now, and sometimes it’s hard to pick up on a signal through the noise. To navigate this space it’s important to have grounding and build appropriate mental models, and I found this set of books to be incredibly helpful in connecting strands and seeing the larger picture. I hope you can get some benefit from the books in this list, and continue reading, learning, and building in this space!

Great Books for AI Engineering was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.