How to Send and Monitor Your Marketing Data in BigQuery

After configuring your data sources, metrics, dimensions, date range, filters, write mode, and schedule in Porter Metrics, you are ready to send your data to BigQuery. This is the final step in setting up your marketing data pipeline.

Here is what happens when you click “Send” and how to monitor your pipeline after it is live.

Previewing Your Table Before Sending

Before you send data to BigQuery, Porter shows you a preview of the table on the right side of the screen. This preview displays a sample of the rows and columns that will land in BigQuery.

Use the preview to verify:

The columns match what you selected. Check that your metric and dimension columns are all present.

The data looks correct. Spot-check a few rows to confirm the numbers match what you see in your ad platforms.

The row count is reasonable. If the preview shows far fewer rows than expected, check your date range and filters.

The preview is a sample, not the full dataset. Your actual BigQuery table will contain more rows than the preview shows. But the column structure and data format in the preview match what lands in BigQuery.

Sending Your Data to BigQuery

Once you are satisfied with the preview, click “Send.” Porter immediately starts the sync. It connects to each of your selected platforms, extracts the data, transforms it, and loads it into your BigQuery table.

For most marketing datasets covering 30 days of data from two to three platforms, the initial sync takes one to three minutes. Larger date ranges or more platforms take longer.

While the sync runs, Porter shows a progress indicator. Once it completes, Porter shows the result: the number of rows loaded and the time it took.

Checking Your Data in BigQuery

After the sync completes, open BigQuery Studio at console.cloud.google.com/bigquery. Navigate to your project in the Explorer panel, expand your dataset, and click on your table.

Click the “Preview” tab to see the first 100 rows of your table. Verify that the data looks correct. Check the date range, the column names, and a few metric values against your ad platform dashboards.

To run a quick verification query, click “Query” to open the query editor with your table pre-filled. Run a simple aggregation:

SELECT date, SUM(spend) AS total_spend FROM your_dataset.your_table GROUP BY date ORDER BY date DESC LIMIT 30

This shows you daily spend totals for the last 30 days, which you can cross-check against your ad platform reports.

Monitoring Your Pipeline in the Porter Dashboard

After your pipeline is live and running on a schedule, you monitor it in the Porter dashboard. Each query in your dashboard shows:

• Last sync time: when the most recent sync ran.
• Status: success (green) or failed (red).
• Rows loaded: how many rows were written to BigQuery in the last sync.

Check this dashboard each morning before you open your reporting dashboards. If a sync failed overnight, you know before your team starts their day.

What to Do When a Sync Fails

If a sync fails, Porter shows an error message explaining why. Common causes:

• Expired credentials: your OAuth connection to a platform expired. Click “Reconnect” next to the affected data source and re-authenticate.
• Platform API error: the platform’s API returned an error. Porter retries automatically, but if the error persists, check the platform’s status page.
• BigQuery permission error: your Google account or service account lost access to the BigQuery project. Check your project’s IAM settings in Google Cloud.

Most sync failures resolve themselves on the next retry. If a failure persists for more than 24 hours, check the error message and address the specific issue.

Your Pipeline Is Now Live

Once your first sync completes successfully and your schedule is set, your marketing data pipeline is live. Every day at your scheduled time, Porter extracts fresh data from your platforms and loads it into BigQuery automatically. Your BigQuery table grows daily, building a historical record of your marketing performance that you query, analyze, and report on.