Generative AI for Product Leaders

How to delight your customers without seemed forced or accidentally "breaking into jail"

There’s a push from executives for product leaders in every industry to develop plans to integrate generative AI into their company’s offerings. This is based on the belief that generative AI will unlock immense value, drive adoption and enhance user experiences.

Back in June, at my day job, I did a fireside chat with VMware CTO, Kit Colbert that included discussing how generative AI can help create a culture of engineering efficiency within your organization. That got me thinking about how product leaders can thoughtfully approach the topic of generative AI to ensure that the features they build & launch are intuitive and resonate with customers. While speed-to-market matters disproportionately in business, it’s also essential to thoroughly think through legal, compliance, privacy, intellectual property (IP), and ethical considerations – and I’ll cover those further down in this post as well.

In my experience, the best way to ensure you’re on the right track for any major product initiative is by asking yourself some key questions, coming up with answers & writing them down as crisply as possible. This is the kind of topic where the answers to questions can be complex and even raise more questions, so you’ll need to answer those too. Think of this as a creating a list of “frequently asked questions” (FAQs) for the product that you could present to someone internally or externally. Following this approach can help you create meaningful and valuable generative AI features that delight your customers and deliver business results.

It’s an exciting time for technology, and ultimately there’s more to consider around Generative AI implementations than any one blog post can cover – from the choices of proprietary and Open-Source Software (OSS) models and data sets to traditional vs. federated learning platforms, etc. I’ll cover those in a follow-up post. Before we get started if you need an explainer on generative AI and how large language models (LLMs) work at a technical level but with a minimum of jargon, this post by Tim Lee is the best and most accessible explainer I’ve seen. It’s important to have at least a basic understanding of what’s going on inside the models before you design features on top.

With that out of the way, let’s dive into some key FAQs that should be answered and guidance on how to approach these questions around the integration of generative AI into products:

1) What problems can generative AI solve for your customers?

1. Name the pain points or challenges that generative AI can address effectively.

2. Determine if generative AI can automate repetitive tasks, enhance decision-making, or provide personalized recommendations.

3. How are these problems being tackled in your product and competitive products today without generative AI? The key question to answer is why generative AI is necessary to solve these problems vs. a more traditional, simpler approach. For example, think about how, five years ago, everyone was looking at blockchains for every database use case when only a subset actually needed blockchain (or blockchain-like) functionality.

2) How will generative AI align with your product’s core value proposition?

1. Ensure that the addition of generative AI aligns with your product’s overall purpose and goals.

2. Evaluate if generative AI can complement existing features or create entirely new opportunities.

3. If the new generative AI features are highly unrelated to your existing product’s value proposition – explain to your customers why you believe you are uniquely well suited to solve this problem for them, increasing the likelihood that they will engage with your feature.

3) What data is needed for generative AI to work effectively?

1. Assess the availability and quality of data needed to train and run generative AI models. What are the models and platforms you plan to use, and why?

2. Consider data privacy and security concerns and ensure compliance with regulations.

3. Do you have this data? If not, explain how you plan to obtain it (license, generate, etc.) and what timeframe/expense that will take. This also goes to the ROI of developing this feature. It can be costly to generate and support your own corpus of data or pay for bulk API access to existing LLMs (large language models) for commercial use.

4) How will generative AI impact the user experience?

1. Analyze how generative AI can enhance user interactions, streamline workflows, or provide personalized experiences.

2. What’s your plan to provide a seamless integration that feels natural and intuitive vs. “bolting this on” in a way that will be jarring to your existing customers?

3. One way to think about this question is by answering, “What will my customers be delighted about once this feature is released?” – and conversely, “What might my customers be (initially) disappointed by?”

5) How will generative AI be explainable and transparent to customers?

1. Address concerns related to the opacity of AI systems by supplying explanations of generated outputs.

2. Implement mechanisms to enable customers to understand and trust the decisions made by generative AI.

3. Explain your thinking around both first-time and ongoing user experiences – do you plan to provide the ability for customers to inspect how the generative AI features came up with their results? If not, will you provide the ability for them to use reinforcement learning to improve the results – such as providing the ability to “rate” the results on one or more axes (un/helpful, in/correct, etc.)

6) How will you handle potential biases and ethical considerations?

1. Evaluate and mitigate biases that may arise from the generative AI models and datasets.

2. Establish ethical guidelines for the use of generative AI to ensure fair and responsible outcomes.

3. Are there any potential legal, regulatory, or company-reputation risk considerations? If so, explain how you plan to mitigate them.

7) How will you avoid/manage the legal and IP contamination risks? (aka, “breaking into jail”)

1. Thinking of your answers to questions three (data) and six (biases) above – if you are sourcing the data from inside your company as a training set, explain how you’ll avoid company IP “leak” externally via the generative AI functionality (i.e., having Gen-AI derived “answers” or content reveal sensitive company information from within the training set)

2. Examine what controls will need to be in place to prevent legal/IP contamination. And then, think through these controls weaken the benefits or accuracy of the information you provide to customers via these Gen AI features and describe your plan to address this limitation. This includes considering this issue when you’re using customer-provided data/input (i.e., how would you prevent customers from harming themselves by leaking sensitive data through this feature)?

3. If you are using customer data or inputs for training, plan to secure customer permission (if required) to use this data. Investigate whether this needs to differ by country or customer type (federal, educational, commercial).

4. Based on your answers to these questions, consider whether you need to give customers the ability to “opt out”? Or whether it needs to be an opt-in? Or explain why you believe you can simply notify customers as part of the terms & conditions (T&C) or end user license agreement (EULA) for using the service/product.

5. If required to provide an opt-out/in, evaluate what percent of customers will opt out/in. Then, think through what you will do if you don’t have enough customer data to train because customers are too concerned about legal or IP contamination risks.

I believe strongly that adding generative AI to your products can unlock new possibilities and create valuable experiences for your customers and that it’s not the only way to improve your products. I recommend product leaders thoughtfully consider the key questions outlined in this article to ensure that the integration of generative AI aligns with their product’s purpose and enhances user satisfaction.

Remember to thoroughly address data requirements and legal/IP concerns, focus on user experience, prioritize explainability and transparency, and handle potential biases ethically. By doing so, you can confidently harness the power of generative AI and take your products to new heights of innovation and customer delight.