Talk by Dustin Hiatt.
Increasingly we’re coming across customers that are seeking a product that allows them to derive insights from data they have spread across several systems and data stores. In order to achieve this, they need to consolidate their data into a single OLAP store that allows them to easily compose queries using simple joins to cross system boundaries. A major challenge is providing a service to move their disparate data sources into a system like BigQuery.
My talk will focus on using Google Dataflow to complete an ETL lifecycle that pulls data from disparate data sources and loads it into BigQuery for analysis. Customers should come away with a generalized understanding of the benefits of doing so, and how Dataflow might complement their current workflows.