Have GenAI projects increased cloud costs for your org, and are you scaling back on GenAI initiatives as a result?
Sort by:
It’s early days for cloud cost increases to start showing up due to GenAI deployments. Right now, broad-based adoption of simple, ready to consume use cases is the priority.
Eventually though, a business case would need to be worked for each GenAI use case – whether it is worthwhile to use GenAI or traditional approaches for the use case, given the quantum of compute capacity and cost projected for the GenAI solution option of that use case.
It has of course increased because we're quite bullish to experiment with GenAI (with fail fast and cheap mindset), we're not really scaling back, but treading carefully nonetheless, starting with developing/refining our Data&AI Strategy & Governance, working with the lines of business to co-create AI-driven use cases.
We initially looked at training our own models on our own data, motiviated primarily around two mistaken notions: 1) that general LLM couldn't undertand our data, and 2) that we wanted to control the confidentiality of our data by running our own LLM. We saw that this would be very expensive.
So we have pursued a different approach: using the OpenAI API (we're accessing it through our enterprise Azure account) and found that it is quite effective, surprisingly affordable, and their privacy guarantees are convincing. This approach has made the incremental cloud cost of doing some GenAI very acceptable.
The short answer is yes, its driven up Azure Consumption significantly. We’re continuing to proceed but keeping tighter control on what we’re doing with our innovation in that area so it doesn’t continue to balloon.