How Do I Do AI?
A one-page guide to your company's first AI project.
(Image credit Shubham Dhage on Unsplash.)
What's this about?

So you've read the other guide, "Will AI Help Here?"

And now that you've done your homework, it's time for you to build out your company's custom AI project. Great!

(Or, maybe you're in too much of a hurry to read because someone in the organization really wants to get moving on an AI project right now. In that case, this guide should still help you to run a small pilot project. Just do yourself a favor and keep it cordoned off from anything business-critical.)

AI projects face a variety of pitfalls – including the big one, "you won't know whether it works till the very end" – so I've assembled this guide to help you through the basics.

Wait, who's behind this?

I'm Q McCallum. I help companies navigate AI matters such as risk management and due diligence on the one side, and AI research and strategy on the other.

My main website has more details.

The steps: How Do I Do AI?
0. Data literacy.

AI works best when executives, stakeholders, and product owners understand what AI truly is and what it can actually do.

At a bare minimum, they need to understand concepts such as "training data" and "ML model," and that there's no guarantee that an AI project will pan out.

Trust me. Things go much more smoothly when they have realistic expectations.

1. Planning

You'll want to get everyone – stakeholders, product teams, data scientists – together to work through the following:

Product and vision: What, specifically, will we do here? What do you expect it to predict, or classify, or generate? How will that fit into the product functionality and UI/UX?

Project boundaries: How much time and money will you invest in this project? There's no guarantee that an AI project will work, and there's no definition of "done"; it's up to you to set your criteria for pulling the plug on the effort.

Model metrics: How will you evaluate the model's performance? And what performance will be good enough? Note that "100% correct" won't happen, so you need to figure out how often the model can be wrong before it harms your business, your clients, or even some complete strangers.

Risk assessment: Does this violate any laws, either where your company is located or where your clients are based? Even if your activities are legal, do they raise any ethical issues? And at a functional level, how does your business suffer when the model gives wrong answers?

Runtime costs: I'll get to the details on this in a bit. For now, just know that this model will cost money to run. Be sure to allocate extra budget for that.

2. Gather data

Every AI project is built on a set of training data. This will probably be your internal, proprietary data (such as customer transactions) but you may also purchase a dataset from an upstream provider or draw from a public source.

However you get your data, do yourself a favor and make sure you actually have permission to use it.

And also make sure that you can get more data later on, if need be.

3. Train the model (R&D)

Your company's data scientists or ML engineers will give you the full story on what they're doing, but here's the short version: they'll need to iterate through a variety of techniques and tuning parameters, trying to build a model that performs well enough according to the metrics you defined during the planning stage.

It's tempting to remain in this phase in search of ideal performance. That's precisely why you defined a scope and budget during the planning phases. When the budget runs out, it's time to halt the R&D work. You either go with the model you have at that point, or you scrap the entire project.

4. Deploy

Assuming you get a model that performs well enough, you'll need to deploy it to production so it can take requests and make predictions.

Your data scientists will work with your software developers and IT ops teams here.

As an executive, the important thing for you to note here is that a model running in production incurs ongoing costs. If your technology infrastructure runs on a cloud service (and, really, that's just about everyone these days) expect a bump in your monthly bill.

5. Monitor

I use two main analogies to explain ML models:

  1. They're like interns: full of energy, but minimal experience.
  2. They're like factory equipment: they occasionally emit bad widgets.

Both of those lead to my main point, which is:

Never let the model run unattended.

Do yourself a favor and track the model's activities. Is its error rate – the number of bad predictions – increasing? Does it seem to give the same answer a lot? Has the outside world changed enough that it no longer matches what was in the training data?

Those are all signs that Something Is Wrong. It's now time to retrain the model on updated data. Or, in extreme cases, you may need to build a completely different model, using different data.

Point being: an ML model is not a static affair. So by building a project or product around AI, you've committed yourself to constantly reviewing the model's performance and occasionally refreshing it.

That's it!

This has been a fairly high-level guide, but it should give you a good idea of what you're getting into when it comes to developing and deploying an AI project.

What's next?

Additional reading material

I post longer-form thoughts about business and AI on my blog and at O'Reilly Radar.

Consulting services

For more personalized assistance, you can contact me to discuss a consulting arrangement. My work covers AI risk management and due diligence and also AI strategy and research.

If you're trying to start, restart, or evaluate your company's AI work, I can probably help.

Click here to open a contact form:

Letter 'Q' in white, on a black background

Hello! I'm Q McCallum.

I help companies navigate AI issues such as strategy, due diligence, risk management, and research.

I've assembled this one-pager guide to help you on your AI journey.

ellipsis icon

Where to find me: blog | AI risk management & due diligence | AI research & strategy consulting


Disclaimer: This guide doesn't constitute consulting advice, nor does reading it establish a business relationship between us. Use at your own risk.

Copyright © 2023 Q McCallum. All rights reserved.

Header image credit Shubham Dhage on Unsplash.