Count
count.co
  • Getting Started
    • What is Count?
    • Count FAQ
    • Intro to your workspace
    • Example canvases
    • Getting started guides
      • Set up your workspace and projects
        • 1. Review workspace settings
        • 2. Create and organise your projects
      • Canvas orientation
      • Your first ad hoc analysis
        • 1. Examples and templates
        • 2. Build your first queries
        • 3. Create visuals
        • 4. Caching, local cells and scheduling
        • 5. Collaborating with a stakeholder
      • Your first report
        • 1. Examples and templates
        • 2. Filters and control cells
        • 3. Sharing your report
        • 4. Alerts
  • Connect your data
    • Database connection overview
      • Athena
      • Azure Synapse
      • BigQuery
      • Databricks
      • Microsoft SQL Server
      • MySQL
      • PostgreSQL
      • Redshift
      • Snowflake
    • Refresh database schema
    • Upload CSV files
    • dbt integration
      • ☁️dbt Cloud integration
      • 👩‍💻dbt Core integration
  • Import & Export
    • Import from other tools
      • Import Miro files
      • Import SQL files
      • Import Google Sheets
      • Import Jupyter notebooks
    • Export code and results
      • Export compiled SQL and Jinja-SQL
      • Export CSV files
      • Export images and PDF files
  • THE CANVAS
    • Navigating the canvas
      • Canvas tool bar
      • Data sidebar
      • Customizing the canvas
    • Canvas objects
      • Cells
      • Text and markdown
      • Shapes and tools
      • Sticky notes
      • Frames
      • Images
      • Embeds
      • Stamps
      • Grouping objects
      • Object order and alignment
      • Locking objects
      • Scaling objects
      • Shared styles
    • Overviews
    • Templates
    • Count AI
    • Alerts and subscriptions
      • Slack integration
    • Keyboard shortcuts
  • Querying data
    • Cells overview
      • Dynamic query compilation engine
    • SQL cells
      • Referencing other cells
      • Jinja templating
      • SQL formatting
    • Python cells
    • Visual and low-code cells
      • Calculations in visuals and low-code cells
      • Joins in visuals and low-code cells
    • Control cells
      • Single and multiple selects
      • Date controls
      • Text, number, and boolean controls
      • Custom control cells
    • Local DuckDB cells
      • DuckDB on the server
    • Query caching and scheduling
    • Manage queries and results
    • Troubleshooting
  • Visualizing data
    • Visualization overview
    • Templated visuals
    • Custom visuals
      • Marks
      • Facet
      • Subplots
      • Style
      • Filters
    • Formatting a visual
      • Axes
        • Secondary Axis
      • Colors and labels
      • Legends
      • Tooltips
    • Column summaries
    • Dynamic text
    • Every Visual Under the Sun
  • Presenting and Reporting
    • Reports and Slides
  • Count Metrics
    • Intro to Count Metrics
    • Build and edit a catalog
    • Views
      • Creating views
      • Customizing views
    • Datasets
      • Creating datasets
    • Save changes to the catalog
      • Catalog validation
      • Version control
    • Exposing catalogs to the workspace
    • Caching in Count Metrics
    • Using the catalog
      • Explore from cell
  • Sharing and Permissions
    • Real-time collaboration
    • Comments
    • Sharing permissions
    • Shared links
    • Embedding canvases
  • History and Versions
    • Version control
    • Duplicating and merging
    • Data snapshots
  • Settings and administration
    • Workspace settings
      • Workspace members
      • Groups
      • Tags
      • Billing
      • Single sign-on (SSO)
        • Okta
        • Entra ID
        • JumpCloud
        • Google
        • Generic OIDC
      • Brand
    • Connection settings
    • Project settings
    • User settings
    • Roles and permissions
  • Quick guides
    • Interactive control guides
      • Date ranges
      • Date groupings
      • Search
      • Select All
  • Resources
    • Join the Slack community
  • Blog
  • Security overview
  • Terms of use
  • Pricing FAQ
Powered by GitBook
On this page
  1. Querying data
  2. Local DuckDB cells

DuckDB on the server

Drastically reduce data warehouse load by offloading queries to Count's servers.

PreviousLocal DuckDB cellsNextQuery caching and scheduling

Last updated 3 months ago

When cells return a large results set, by default Count will only return the first 10,000 rows of the results to your browser. When this happens, the footer of the cell will contain some information about the full result set:

You can choose to , in which case downstream Python and DuckDB cells will continue to run in your browser.

Otherwise, any downstream DuckDB cells will be executed on Count's servers rather than in your browser. When running on the server:

  • The full results of any upstream cells are always available

  • DuckDB may have access to more working memory than it does when running in a browser

  • Queries may fail if they use too much memory, or attempt to access files on the local filesystem

The versions of DuckDB in your browser and on Count's servers are the same, so you shouldn't notice any difference when running queries.

You may also see DuckDB cells running on the server when your local DuckDB database is nearing its memory limit.

Query limits

When using DuckDB on the server, your queries are executed in isolated virtual machines (VMs). These VMs execute one query at a time, and have limits placed upon them which may depend on your subscription plan:

  • Total available RAM - this is the total amount of RAM that the VM can use, which includes the space it requires to load any parent cells, execute the query, and stream the query results out of DuckDB. The VM uses a minimal operating system, which adds approximately 100-150MB of RAM overhead while running the query.

  • Total query duration - queries are terminated after they have been running for 1 hour.

  • Maximum query concurrency - each workspace can execute up to 100 concurrent DuckDB queries on the server. (DuckDB queries in the browser have no concurrency limits.)

Why is the maximum result size less than the total available RAM?

When executing queries, the VM needs memory space to load its operating system and allocate memory during the processing of the query. (Results are always streamed out of the VM, so their impact on memory usage is minimal.)

Additionally, it is common for queries to reference multiple parent cells, which requires loading multiple parent result sets into VM memory.

For these reasons, Count uses a default factor of 2 between the VM RAM size, and the maximum query result size. If you find that this default is not appropriate for your workload, please contact Count support.

Maximum result size - Count will stop streaming the results out of DuckDB once the total uncompressed size of the results (in format) reaches the limit configured for your workspace. This limit is typically half of the RAM size allocated to the VM.

Arrow
download all of the results
When partial results are returned, this information is displayed.
When a DuckDB cell is running on Count's servers, the footer will show DuckDB (server).