Workload Management
Gnok automatically classifies queries by complexity, routes them to resource groups with configurable limits, and schedules execution using a multi-level feedback queue. This helps prioritize interactive work alongside batch workloads; response time still depends on query cost and available resources.
Workload Classification
Every query is automatically classified based on analysis of its physical plan. Classification determines default resource limits and scheduling priority.
| Class | Criteria | Default Priority |
|---|---|---|
| Interactive | No joins, no window functions, estimated rows < 100K | High |
| Medium | 1-2 joins, or has window function, or 3+ scans, or estimated rows 100K-10M | Normal |
| Batch | 3+ joins, or window + aggregate combo, or estimated rows > 10M | Low |
Classification is based on structural plan metrics:
- Join count -- number of join operators (hash, sort-merge, broadcast, nested-loop, range, as-of)
- Scan count -- number of table scan operators
- Window/aggregate count -- presence of window functions or aggregates
- Estimated rows -- sum of estimated rows across all scans (from Iceberg metadata)
Manual Override
Override automatic classification at the session level:
SET gnok.workload_class = 'batch';
SET gnok.workload_class = 'medium';
SET gnok.workload_class = 'interactive';
Each class has configurable timeout and memory limits:
| Class | Default Timeout | Default Memory Cap |
|---|---|---|
| Interactive | 30s | 1 GB |
| Medium | 3 min | 4 GB |
| Batch | 10 min | 8 GB |
Service-managed limits determine the effective timeout, memory, and execution settings for each workload class.
Resource Groups
Resource groups provide hierarchical admission control for query execution. They are tree-structured, with per-group concurrency, queuing, and memory limits.
Configuration
Resource-group provisioning and service routing settings are managed by Gnok. Use the authorized workload controls available to your account; a local configuration file does not change the hosted service.
Group Properties
| Property | Default | Description |
|---|---|---|
hard_concurrency_limit | 100 | Maximum concurrent running queries. New queries are queued when this limit is reached. |
soft_concurrency_limit | 100 | Target concurrency. Above this, CPU penalty scaling reduces effective concurrency. |
max_queued_queries | 1000 | Maximum queued queries. New submissions are rejected when the queue is full. |
soft_memory_limit_bytes | (none) | Memory threshold. New queries are blocked when the group's aggregate memory exceeds this. |
soft_cpu_limit_ms | (none) | Cumulative CPU time threshold. When exceeded, effective concurrency decreases linearly toward 1. |
hard_cpu_limit_ms | (none) | Cumulative CPU time hard limit. Effective concurrency reduced to 1. Defaults to 2x soft if unset. |
scheduling_policy | fair | How queries are dequeued within the group. |
scheduling_weight | 1 | Weight for parent's weighted scheduling. Higher weight = more share. |
Scheduling Policies
| Policy | Description |
|---|---|
fair | FIFO ordering -- queries are dequeued in arrival order. |
weighted | Weighted fair sharing across sibling groups. Groups with a lower running/weight ratio get priority. |
query_priority | Dequeue the highest-priority query first (based on the query's execution priority). |
Query Routing
Queries are routed to resource groups using routing rules evaluated top-to-bottom. The first matching rule wins.
| Rule Field | Description |
|---|---|
match_tenant | Match organization (tenant) ID (exact). Omit to match any organization. |
match_user | Match user ID (exact). Omit to match any user. |
match_source | Match query source (e.g., dashboard, etl). |
match_resource_pool | Match resource pool name from session config. |
match_query_type | Match statement type: select, insert, update, delete. |
target_group | Target group path (e.g., root.interactive.dashboard). |
If no rule matches, the query is routed to the root group.
Priority Scheduling (MLFQ)
Within each resource group, fragment execution on workers is scheduled using a Multi-Level Feedback Queue (MLFQ). The MLFQ gives short-running queries natural priority over long-running ones without requiring explicit priority assignment.
How It Works
The MLFQ has 5 levels defined by cumulative scheduled time:
| Level | CPU Time Threshold | Typical Queries |
|---|---|---|
| 0 | 0 - 1s | Simple lookups, point queries |
| 1 | 1s - 10s | Small aggregations, filtered scans |
| 2 | 10s - 60s | Multi-table joins, moderate analytics |
| 3 | 60s - 300s | Large joins, complex aggregations |
| 4 | 300s+ | Long-running ETL, full-table scans |
New queries start at level 0. As a query accumulates CPU time across its fragments, its subsequent fragments are assigned to higher levels. Level 0 gets 2x more CPU time than level 1, which gets 2x more than level 2, and so on.
High-priority queries (priority >= High) bypass the MLFQ and always execute at level 0, regardless of accumulated CPU time.
MLFQ Configuration
Execution concurrency and scheduler thresholds are managed by Gnok. Request workload capacity changes through your administrator or support@gnok.com.
Query Killing
Queries that exceed their resource limits are automatically cancelled:
- Timeout -- queries exceeding their workload class timeout are cancelled
- Memory -- queries exceeding their memory limit trigger spill-to-disk first; if spilling is not possible, the query is cancelled
- Queue overflow -- when a resource group's queue is full, new submissions are rejected immediately
Cancelled queries are recorded in query history with status cancelled and an error message indicating the reason.
Monitoring
SHOW QOS STATS
Admission and queueing per warehouse of your workspace:
SHOW QOS STATS;
One row per warehouse:
| Column | Meaning |
|---|---|
warehouse, warehouse_id, status | The warehouse and its state. |
max_concurrent, running, queued | Its concurrency limit and the statements running and waiting on it right now, across every coordinator. |
admitted, admitted_after_queue, admitted_without_slot | Statements admitted since counted_since: straight away, after waiting in the queue, or without taking a slot. |
queue_timeouts, refused | Statements that waited too long, or were refused at admission. |
queued_here, oldest_queued_ms, max_queue_wait_ms | The queue on the coordinator that admits the warehouse's statements: how many wait now, the oldest wait, and the longest wait seen. |
coordinator, dedicated_coordinator, counted_since | Which coordinator the admission counters come from (a warehouse can have a coordinator of its own), and since when they count. |
error | Why the admission counters couldn't be read, when they couldn't; those columns are then NULL. |
Any member of the workspace can run it; it lists only the workspace's warehouses. It doesn't resume a suspended warehouse.
SHOW RESOURCE GROUPS
The resource groups and how full they are:
SHOW RESOURCE GROUPS;
Returns one row per group: path, running, queued, oldest_queued_ms, hard_concurrency_limit, soft_concurrency_limit, effective_concurrency_limit, max_queued, memory_bytes (of the group's running statements), memory_limit_bytes, scheduling_policy, scheduling_weight and limits_follow_cluster.
SHOW CACHE STATS
The caches the coordinator keeps (query results, CTE results, table metadata, Parquet footers, statistics, …):
SHOW CACHE STATS;
Returns one row per cache: cache, keyed_by, enabled, entries, bytes, capacity_bytes, capacity_entries, hits, misses and evictions. A figure a cache doesn't track is NULL, not 0.
SHOW RESOURCE GROUPS and SHOW CACHE STATS describe compute shared by several workspaces, so they're available to platform administrators only; other callers get an access-denied error.