BigQuery, Dataproc, Dataflow
Moving from BigQuery, Dataproc and Dataflow to Databricks on Google Cloud brings SQL analytics, Spark, streaming and ML onto one platform with open table formats and unified governance, while data stays in your Google Cloud project.
BigQuery datasets and SQL, Dataproc Spark and Hive jobs, Dataflow pipelines, scheduled queries
Each component has a clear target on the Databricks Data Intelligence Platform.
| Google Cloud | Databricks |
|---|---|
| BigQuery datasets | Delta or Iceberg tables in Cloud Storage, queried by Databricks SQL |
| BigQuery scheduled queries | Lakeflow Jobs |
| Dataproc (Spark, Hive) | Serverless or job compute |
| Dataflow pipelines | Lakeflow Declarative Pipelines or Structured Streaming |
| Cloud Composer | Lakeflow Jobs |
| Dataplex and Data Catalog | Unity Catalog |
| Vertex AI | MLflow, Model Serving and Agent Bricks |
Inventory every table, job, report and user. Map dependencies, find what is unused, and estimate effort and cost.
Output: Migration assessment and business caseDesign the target lakehouse, governance and security. Group workloads into waves by value and risk.
Output: Target architecture and wave planConvert SQL and ETL with LakeBridge and our accelerators, then review and test the output.
Output: Converted, tested code per waveRun old and new side by side and compare results table by table until they match.
Output: Signed-off reconciliation reportsMove users in waves, retire the old system, then tune cost and performance.
Output: Decommission plan and FinOps baselineWe use Databricks LakeBridge for profiling, code conversion and reconciliation, alongside our own accelerators and engineer review of every critical path.
Network and customer data split between BigQuery and Dataproc, with churn models built off-platform.
Read the journey: Google BigQuery and Dataproc to Databricks for a telco →Run Databricks on Google Cloud next to your data, and bring analytics, engineering and ML onto one platform.
Read the article: BigQuery and Dataproc to Databricks on Google Cloud →Yes. Databricks runs on Google Cloud, so data does not need to leave your cloud.
One business domain end to end, such as customer or finance. It proves the pattern and gives users a complete experience early.
