It feels like every data tool on the market promises to solve your biggest problems with a prompt. AI can generate SQL, write pipeline code, analyze datasets, and even help build data workflows.
AI has plenty of useful applications, but applying large language models (LLMs) to a messy, legacy data stack won't fix underlying data problems.
If your source data is inconsistent, your business definitions aren't documented, or your existing pipelines are difficult to monitor, AI makes those problems harder to see. You may get faster answers, but faster doesn't mean more accurate.
Traditional automation is often a better fit for repetitive, predictable pipeline work. The key is knowing where AI adds value and where established data processes still make more sense.
The Risks of Adopting AI Too Quickly
When teams rush AI adoption without a solid foundation, they typically hit three major speed bumps:
Hallucinated Reporting
Relying on AI to write complex SQL or calculate business metrics on the fly can produce incorrect results. An LLM can generate SQL that looks reasonable but applies the wrong filters, joins, or business logic.
When metrics aren't consistently defined, even technically valid queries can produce misleading reports.
High Compute Costs
Sending millions of rows through an LLM for a transformation that could be handled with SQL is an expensive way to solve a simple problem.
Filtering, joining, aggregating, standardizing fields, and converting data types generally don't require generative AI. A conventional ETL process can handle these tasks predictably and efficiently.
Pipeline Fragmentation
If an AI tool generates a new script every time someone needs a data transformation, you can end up with a collection of one-off workflows that are hard to maintain.
That makes it more difficult for engineers to debug and test when something breaks. Clear standards for code storage, testing, review, and ownership are essential for maintaining these workflows.
AI can help write pipeline code, but you still need standards for where that code lives, how it gets reviewed, how it is tested, and who owns it.
Related article: How to Keep BigQuery Affordable as Your Data Grows
Integrating AI Across Your Data Stack
A phased approach lets you match AI capabilities with specific parts of your data pipeline based on their potential value and the level of oversight required.
1. Ingestion and ETL (Low Risk, High Value)
Data ingestion is a good place to introduce targeted AI automation, particularly when you're working with inconsistent formats or unstructured information.
AI can help with:
- Mapping inconsistent source fields to a unified destination schema
- Extracting information from PDFs, emails, and attachments
- Identifying potential data quality issues
- Generating documentation for incoming datasets
For standard source-to-destination data movement, though, ETL is often the better tool.
The goal is to establish a reliable data foundation before adding more sophisticated AI capabilities.
2. Data Transformation and Modeling (Medium Risk, High Governance)
Data transformation directly impacts business logic, making this layer especially important to govern. Reliable engineering is the foundation of accurate analytics.
AI can assist with development, but shouldn't alter data autonomously, and production changes should always include human review and testing.
For example, you can use AI to:
- Generate boilerplate dbt models
- Migrate legacy SQL into modern cloud SQL dialects
- Create technical documentation
- Suggest data quality tests
- Explain unfamiliar transformation logic
- Troubleshoot existing code
Google's current BigQuery tools reflect this approach. Gemini can assist with SQL generation, data preparation, and pipeline development, while Google's documentation emphasizes validating generated output.
The important distinction is who makes the final decision. AI can suggest the transformation, but your data team should determine whether that transformation belongs in production.
3. Insights and Reporting (High Risk, Maximum Guardrails)
This is where AI needs the most structure because the output directly influences business decisions. Exposing an LLM directly to raw data tables can lead to hallucinated metrics.
Suppose someone asks: "What were our sales in Q3?"
The challenge isn't generating SQL, it's knowing exactly what "sales" means in your organization.
A structured semantic layer with certified metric definitions gives AI the context it needs to generate queries based on approved business logic. It defines the calculations, relevant tables, dimensions, filters, and business rules, allowing AI to translate a natural-language question into SQL that follows those definitions.
It's much safer than giving an LLM unrestricted access to raw tables and asking it to figure everything out.
You can also validate queries before they execute through permission checks, SQL validation, cost controls, and tests for expected results.
This approach makes natural-language reporting more useful while keeping metric definitions and data access under control.
Where to Slow Down vs. Accelerate Your AI Adoption
| Pipeline Phase | Slow Down and Limit AI | Accelerate AI Adoption |
|---|---|---|
| Ingestion and ETL | Writing raw extraction logic without human review | Parsing unstructured files into clean, structured tables |
| Transformation | Letting AI change financial or master data autonomously | Generating pipeline documentation and test scripts |
| Reporting | Using generative AI to calculate business metrics from scratch | Translating natural language questions into certified SQL |
| Data Quality | Automatically changing production data | Identifying potential issues for review |
| Documentation | Relying on unreviewed AI-generated workflows | Generating and maintaining technical documentation |
The more a process affects business logic, financial reporting, permissions, or production data, the more oversight and controls you should put around AI.
Building a Data Stack That Supports AI
AI output is only as good as its input, so you need to start with a strong foundation. AI will work best when it has accurate data and clear context.
You should be able to answer some basic questions before putting an agent or LLM into your analytics environment:
- Where does each data source come from?
- How often is the data refreshed?
- Which metrics are officially defined?
- Where does business logic live?
- Who owns each pipeline?
- How are data quality issues identified?
- What happens when a source schema changes?
Google's current BigQuery governance capabilities are built around many of these same principles, including data discovery, metadata, data quality, and consistent data use.
Once those fundamentals are in place, AI has much better context to work with. You can then use it to accelerate development, documentation, analysis, and other tasks without asking it to become the foundation of your entire data architecture.
Use AI Only Where It Adds Value
Start by identifying the repetitive tasks that consume the most time and evaluating whether AI can improve them without compromising accuracy or control.
Use AI to assist with code, documentation, unstructured data, and analysis. Keep core transformations predictable, define your business metrics clearly, and require review when AI-generated output affects production data or business reporting.
You may find that simply improving ingestion, data quality, or reporting infrastructure adds more value than AI.
Related article: Why Your Data Pipeline Doesn't Need an AI Agent
Ready to Streamline Your Data Infrastructure?
At Calibrate Analytics, we help businesses build and maintain reliable data infrastructure, from automated ETL and BigQuery integration to tracking, governance, and reporting.
If you're evaluating where AI or automation fits into your data stack, get in touch with our team to discuss your options.
