Australian IT leaders say data gaps stall AI scale

Confluent has published research showing that 69% of Australian IT leaders say a lack of real-time data infrastructure is stalling efforts to scale AI. The findings are based on a survey of 225 Australian respondents within a global sample of 4,625 IT leaders.

The results point to a gap between enthusiasm for AI and the data systems needed to support broader use in business operations. Seven in 10 Australian IT leaders said they had faced at least three challenges when trying to scale AI projects.

Data lineage, timeliness and quality assurance were the most commonly cited obstacles, identified by 72% of respondents. They were followed by insufficient real-time data infrastructure at 69%, fragmented data ownership at 66%, and skills gaps in managing AI projects and workflows at 66%.

The survey also suggests that data management concerns are delaying the use of agentic AI. Only 36% of organisations said they had agentic AI in production, while 70% pointed to the reliability and non-determinism of large language models as a major barrier and 60% cited data infrastructure and quality issues.

Infrastructure focus

Investment priorities appear to be shifting as companies reassess what is needed to move AI from experimentation into day-to-day use. In Australia, 88% of IT leaders ranked data streaming as a key investment priority, ahead of AI and machine learning technologies at 80%.

The shift reflects a broader emphasis on the data that underpins AI systems. Four in five respondents said using enterprise data to drive AI systems was a top business priority.

Governance also featured strongly in the findings. Nearly nine in 10 Australian IT leaders said effective data sovereignty was important, while the same proportion said data provenance and tracking mattered as AI becomes more embedded in business processes.

Respondents linked those concerns to oversight of how data moves through organisations and how AI systems use it. The survey found that 89% believed data streaming platforms could help address governance, risk and compliance issues in agentic AI by enforcing data access and usage policies upstream.

A further 93% said such platforms help address concerns about large language model reliability by keeping data complete and up to date, while 90% said they make data more trustworthy, contextualised and discoverable. Overall, 94% said data streaming had increased or was expected to increase the impact of their AI investments, and 91% said it helped ease the path to AI adoption.

Executive view

Greg Taylor commented on the growing focus on governance in AI rollouts.

"Australian organisations understand that AI governance cannot stop at the model. As AI systems become more embedded in business processes, leaders need confidence in the data behind every output, decision and action. That means knowing where the data came from, whether it is current, how it has been governed, and who has access to it. Data streaming platforms are critical for enabling organisations to govern live data as it moves, not after the fact. That real-time foundation determines which businesses can scale AI safely and create meaningful business value," said Greg Taylor, SVP APAC, Confluent.

The findings add to a wider debate in the technology sector over whether AI spending has outpaced the readiness of corporate data estates. While companies continue to invest in models and tools, the research indicates that many Australian organisations still face unresolved issues around data ownership, quality and control.

Those issues can become more pressing when AI systems are connected to live business processes rather than isolated pilot projects. In these settings, the freshness and traceability of data can affect not only the quality of outputs but also an organisation's ability to explain and govern automated decisions.

Shaun Clowes said the main constraint for many organisations lies in their data foundations rather than in their appetite for AI spending.

"Most organisations do not have an AI investment problem, they have a data problem. AI systems depend on fresh, accurate and contextual information, but too many are still being built on fragmented data, batch processes, and infrastructure that was not designed for continuous intelligence. As organisations move beyond experimentation and start deploying AI across critical business processes, those gaps become harder to ignore. Models need to be connected to the systems, events and signals that reflect what is happening across the business. The companies making the most progress are investing not only in AI itself, but in the data foundations needed to support it. Those foundations will determine which organisations can turn AI investment into business value at scale," said Clowes.