.png%3Falt%3Dmedia%26token%3Ddadbeb66-a273-411a-afdd-d51dd957a2bf)
.png%3Falt%3Dmedia%26token%3Ddadbeb66-a273-411a-afdd-d51dd957a2bf)
dbt Cloud integration
Import your dbt Cloud models into Count
When you are analyzing data in Count, you and your team can access not just the metadata from your database, but also all the information related to the dbt Cloud models and runs that are connected to that table.
This means you can:
- Import your compiled models from dbt Cloud into canvas cells at the click of a button
- Decide how much of the model lineage you want imported
Giving your team the power to
- take a database table, and with one click, import all the logic in dbt that created that model (from source to final output)
- decompose the CTEs of that logic into a DAG of interactive queries for better comprehension and testing
- run your model queries against any database, including swapping between dev and prod environments to test results
- further analyze any model queries to detect bugs, or to better understand the model
- update model logic and export dbt-ready SQL (customized jinja-syntax)
- prototype and get feedback on models from the data team and stakeholders
As if this wasn't powerful enough already, you can also
- Match up your database materializations and sources with their corresponding dbt components
- Check the latest run information per job for a dbt model/source, including
- run statistics
- test/freshness status: pass (green), fail (red), warn (orange), skip (grey)
- Copy the compiled SQL into the canvas of the model so you can start breaking it down
Giving your team:
- a better understanding of the tables you are querying in Count
- more confidence in the accuracy and timeliness of the results
- complete transparency about where the data actually came from
In an existing database connection, select Manage connection, then scroll to the dbt Cloud integration section:
To connect you will need:
- Read-only
- Metadata only
If dbt Cloud is already connected, the button will read Update rather than Connect.
Count retrieves your dbt metadata when you:
- connect or update your dbt Cloud integration
- open a canvas with a data source containing a dbt Cloud integration
Count retrieves the latest run for each job run on your dbt Cloud account, and matches up the model/source/test information to the corresponding database tables.
- any database materialization (table/view) that has metadata associated
- any dbt ephemeral materializations run on that database
Here stg_orders is table, stg_payments is a view, my_first_dbt_model is an ephemeral materialization
In the pop-out metadata pane, you can:
- match up your database materialisations and sources with their corresponding dbt unique IDs
- view component metadata within Count, or quickly open the dbt docs entry for a particular component (using thesymbol at the top of the pane).↗
- check the latest run information per job for a dbt model/source, including
- run statistics
- test/freshness status: pass, fail, warn, skip (greyed out)
- Add a model (and it's dependencies) to canvas
For models that have compiled sql associated with them (via your dbt Cloud jobs), you can import this sql into a canvas cell. This can be done by the Add model and Add model, split out CTEs options.
Add model: a cell is created with compiled SQL for that model
Add model, split out CTEs: a frame is created for the model, with all constituent CTEs split out into a DAG of cells
Based on the dbt Cloud metadata it can import, Count can intelligently reconstruct your model lineage giving you the options to Add model and dependencies and Add model and dependencies, split out CTEs.
Add model and dependencies: a DAG of cells is created for the model and its lineage
Add model and dependencies, split out CTE: a DAG of cells is created for the model and its lineage, with any model containing CTEs being split out further in its own frame
Now your team is ready to analyze, debug, and fix any of your model queries, and then export a dbt-ready SQL (jinja-sql) for use in your chosen source versioning software.
Last modified 12d ago