Don’t trust “AI experts”.

The term “AI expert” is largely a marketing label, often applied to random individuals to drive SaaS sales, secure funding, obtain promotions, or increase ad revenue. Those things as always come in the way of real facts.

Measuring expertise precisely is challenging, but I would argue that ALL who self-identify as “AI experts”, along with 99% of those labeled as such by others, fall into the bottom 20% of the field in terms of true expertise.

As more people are called experts, the value of the term diminishes—a clear case of inflation. The biggest contributors to this inflation and its undesirable effects IMO are business “AI experts” that have barely ever touched a line of code.

An “AI expert” from 10 years ago is vastly different from one today. The field has evolved and grown so much that 1) the requirements to qualify as an AI expert have accumulated beyond what a person can reasonably achieve and 2) the term “AI expert” has lost much of its meaning.

I would call someone an “AI expert”, if they (among other things, and in loosely decreasing order of difficulty):

  • Spent >20 years, >70 hours a week, focused on AI
  • Can build a GPT3.5 level LLM with 50,000 H100 GPUs, a 10-person team, and six months of preparation
  • Are recognized as an expert by >50k people
  • Made accurate predictions about AI >50% of the time in the last 10 years
  • Being able to teach courses on (any of) NLP, RL, CV, AI in Robotics, etc., with just one month of preparation
  • Master frameworks like TensorFlow or PyTorch
  • Can write CUDA kernel
  • Writing a Monte Carlo simulation from scratch
  • Describing gradient descent on the spot

Good new is you NEVER need such a profile. 🎉

Ultimately, the only real AI experts that will ever exist are AI itself. The rest of us are merely trying to keep pace.

“AI expert” is a marketing term to quote mostly random people. It’s used to sell more SaaS, raise more money, get promotions, sell more ads, etc. Those things as always come in the way of real facts.

The field has become way too broad for someone to be an expert in all of it.

It is difficult to precisely measure a degree of expertise, but I would say that 1) all the people who called themselves AI experts and 2) 99% of the people called AI experts by other people are in the bottom 30% of the field in terms of expertise.

An AI expert 10 years ago and an AI expert today are not the same thing. The field has evolved so much that the term “AI expert” has lost much of its meaning.

The term “AI expert” also suffer inflation, the easier it is to call someone an expert, the less value the term has.

The thing I struggle with the most I think is “business” “AI experts”.

The only real AI experts there will ever be without a doubt will be AI itself. The rest of us are just trying to keep up.

I would personally accept to call someone an “AI expert” if they, among many other things:

  • can write a Monte Carlo simulation from scratch
  • spent >30 years, >70h/week average thinking / talking / writing about AI
  • are acknowledged as an expert by >10M people
  • can describe gradient descent without prep
  • have made >50% good predictions about AI in the last 10 years
  • master TensorFlow, PyTorch or the like
  • master the underlying principles / mathematics of the algorithms they use
  • given 6 months, 50k H100s, and a team of 10, can build a state-of-the-art LLM
  • have elaborated opinions on the ethical implications of AI
  • can write a CUDA kernel
  • could teach a course on NLP, RL, CV, AI in Robotics etc. with 1 month preparation
  • have strong intuitions about the future of AI
  • have a deep understanding of the limitations of AI

The good thing is, you never need an AI expert.

In the early days of AI, if you mastered complex algorithms and understand the underlying mathematics you were an experts. If you knew how to write a Monte Carlo simulation or work with TensorFlow, you were seen as an expert. These skills required deep technical knowledge, a strong grasp of data science, and the ability to apply that knowledge to real-world problems. Today, however, the bar for what passes as AI expertise has been lowered to the point where it often means little more than being able to use a pre-built AI tool or library.

The real experts, the ones who can describe gradient descent off the cuff or compute a dot-product without breaking a sweat, are becoming harder to find amid the noise of self-proclaimed experts. Expertise in AI has always been about more than just using the latest tools; it’s about understanding the principles that govern these tools, the math that underlies them, and the broader implications of their use. It’s about knowing when and how to apply a particular technique to a given problem and being able to explain why it works.

What’s more, being an expert in one domain doesn’t automatically make someone an expert in another. AI is a broad field, encompassing everything from machine learning and neural networks to natural language processing and robotics. Yet, the public discourse often conflates these different areas, leading to the mistaken belief that anyone proficient in one aspect of AI must be an expert in all. This misconception is compounded by the media and social platforms, where the term “AI expert” is thrown around so loosely that it’s lost much of its meaning.

Becoming a true expert in AI is a long and difficult journey. It’s not something that can be achieved overnight or by simply reading a few books or taking an online course. It requires years of study and practice, and even then, most experts are only truly knowledgeable in a narrow area of the field. As AI continues to evolve, the number of these narrow areas will only increase, creating a need for more specialized expertise.

In the real world, expertise is not just about knowing how to do something; it’s about knowing when and why to do it. It’s about understanding the context in which a problem exists and being able to navigate the messy, chaotic reality of applying AI to solve that problem. The experts who succeed are the ones who can do this consistently, who have the experience and the intuition to make the right decisions even when there’s no clear answer.

But expertise is not just about knowledge; it’s also about recognition. An expert is someone whose peers acknowledge their superior understanding and skills. This recognition is hard-won and often comes after years of work. It’s not something that can be self-appointed or easily faked. Yet, in today’s world, where anyone can label themselves an expert on LinkedIn or Twitter, this distinction is becoming increasingly blurred.

The misuse and overuse of the term “expert” have led to a growing distrust of those who claim to be experts. People are starting to question whether these self-proclaimed experts really know what they’re talking about or if they’re just riding the wave of AI hype. This skepticism is not without merit. History is filled with examples of experts who were wrong, sometimes disastrously so. And in a fast-moving field like AI, where the state of knowledge is constantly changing, even true experts can find themselves out of their depth.

The concept of an “expert” is also more subjective than we like to admit. What makes someone an expert? Is it their years of experience, their ability to solve complex problems, or the recognition they receive from their peers? In reality, it’s a combination of these factors, and even then, it’s not always clear-cut. Expertise exists on a spectrum, and what makes someone an expert in one context may not make them an expert in another.

Furthermore, expertise is not static. It’s something that must be constantly maintained and updated. The best experts are those who continue to learn, who stay curious and open-minded, and who are willing to admit when they don’t know something. They understand that expertise is not just about having answers but about asking the right questions.

In the end, the real value of an expert lies not in their knowledge alone, but in their ability to apply that knowledge in a way that makes a difference. It’s about solving real problems and making decisions that have a tangible impact. And in a field as complex and rapidly evolving as AI, this is no small feat. True experts are rare because becoming one takes time, effort, and a willingness to embrace uncertainty.

So, while the number of people calling themselves AI experts may be on the rise, the number of true experts—those who have the deep understanding, the practical experience, and the recognition of their peers—remains small. This isn’t necessarily a bad thing. Expertise, after all, is supposed to be rare. It’s what makes it valuable. But it does mean that we need to be more discerning about who we listen to and who we trust in this ever-changing field.

The future of AI expertise is likely to be more specialized, more nuanced, and perhaps more difficult to recognize. But that’s the price we pay for progress. As AI continues to advance, the need for true experts—those who can navigate the complexities of this technology and apply it responsibly—will only grow. And in a world where anyone can claim to be an expert, it’s more important than ever to seek out those who truly are.