# 2. Configuration¶

## 2.1. QuasarDB Daemon¶

QuasarDB stores its configuration options in a qdbd.conf file. The precise location on where this file resides depends on on your method of installation, but is typically /etc/qdb/qdbd.conf.

This section is organised per component, and describes the configuration options relevant to each of those.

You can set a license either by file or by putting the license key as a string directly into your configuration.

Variable

Description

local.user.license_file

Absolute location to the license file. Be careful to ensure the license file is readable by the qdb user, otherwise the license will not activate.

local.user.license_key

### 2.1.2. Storage¶

Variable

Description

global.cluster.storage.engine

Specifies which storage engine to use. Accepted values: rocksdb or transient. Defaults to rocksdb.

local.depot.rocksdb.root

Location to the root folder of where your database files are stored. Defaults to /var/lib/qdb.

local.depot.rocksdb.wal

Location to the Write-Ahead-Log (WAL) folder. Defaults to empty, which will make it a subdirectory of the root directory called “wal”.

local.depot.tiered_storage

An array in the form [{“path”: “path1”, “size”: size1}, {“path”: “path2”, “size”: size2}] which specified, which directory, in order should be used for storage. When a storage exceeds capacity, older data is moved to the next storage in the list. Actual usage may exceed the size specified. Size is specified in bytes.

local.depot.rocksdb.compaction

The compaction strategy to use. By default it is “leveled”, but “universal” is also accepted. Universal improves write amplification at the cost of space amplification.

local.depot.rocksdb.disable_wal

When this option is set to true, writes will not go to the Write-Ahead-Log (WAL), increasing performance at the cost of durability. Writes go to the WAL by default. Use this option with caution.

local.depot.rocksdb.threads

Should be increased if the daemon is under constant insertion pressure, and there is still available CPU: it increases the amount of threads the storage engine has available for background operations such as compaction.

local.depot.max_total_wal_size

Size, in bytes, of the Write-Ahead-Log (WAL). The WAL is where every update gets written. When the WAL is full, memtables are flushed to disk. If the WAL is too small, memtables may thus be flushed before they are full, impacting write speed. However, large WALs means increased memory usage and potentially higher compaction. Your WAL should be large enough to absorbs spikes, but cannot be used to compensate for a persistence layer too slow to absorb the load or memtables being too small. See local.depot.rocksdb.table_mem_budget.

local.depot.rocksdb.hi_threads

The number of high-priority threads, in addition to local.depot.rocksdb.threads. Should typically be left to 2.

local.depot.rocksdb.data_cache

The size of the block cache, in bytes. Increasing this may increase the performance at the cost of additional memory usage.

local.depot.rocksdb.table_mem_budget

The mem budget for each column family. Defaults to 1 GiB. For write heavy usages it’s recommended to significantly increase this value.

local.depot.rocksdb.max_background_flushes

How many threads will be dedicated to writing data to disk. Write heavy scenarios may benefit from a higher count. Defaults to 1.

local.depot.rocksdb.max_bytes

The maximum amount of data to store, in bytes. Inserting data will fail with “quota exceeded” when the limit is reached. If set to 0, no check is done.

local.depot.rocksdb.persistent_cache_path

A path to a local disk to be used as a persistent cache. May increase performance when data is stored in a remote disk.

local.depot.rocksdb.persistent_cache_size

The maximum size of the persistent cache, in bytes. Cannot be zero if the persistent cache path is specified.

local.depot.rocksdb.persistent_cache_nvme_optimization

If your persistent cache is on an NVME, enabling this option may increase performance.

local.depot.rocksdb.cloud.provider

A string either “none” or “aws” specifying your cloud provider to use to store the cloud data.

local.depot.rocksdb.cloud.keep_local_sst_files

If false, files will be deleted after being uploaded to the cloud.

local.depot.rocksdb.cloud.request_timeout

The timeout used for cloud queries, in milliseconds. The default of 0 means that the cloud provider’s default will be used.

local.depot.rocksdb.cloud.create_bucket_if_missing

Will create the bucket if it does not exist.

local.depot.rocksdb.cloud.bucket.destination_prefix

The first half of the bucket name where to store the data.

local.depot.rocksdb.cloud.bucket.destination_suffix

The second half of the bucket name where to store the data. On AWS, the bucket name must be unique. If both are empty, the database will not upload its content to the cloud.

local.depot.rocksdb.cloud.bucket.source_prefix

The first half of the bucket name where to read the data from.

local.depot.rocksdb.cloud.bucket.destination_suffix

The second half of the bucket name where to read the data from. On AWS, the bucket name must be unique. If both are empty, the database will not read its content from the cloud.

local.depot.rocksdb.cloud.bucket.path_prefix

Quasardb will use local.depot.rocksdb.root as the path inside the bucket to store the data in the cloud. If path_prefix is not empty, a prefix will be added to this path. That way, different nodes can use the same local path and write to different paths within a same bucket.

local.depot.rocksdb.cloud.bucket.region

The region where the bucket is located.

local.depot.rocksdb.cloud.aws.access_key_id

The AWS access key id to use for authentication.

local.depot.rocksdb.cloud.aws.secret_key

The secret key to use for authentication. You need both the access key and secret key set for authentication to work.

local.depot.rocksdb.cloud.aws.config_file

The configuration file to use for authentication. When using a configuration file, access_key_id and secret_key must be left blank.

### 2.1.3. Networking¶

The QuasarDB daemon uses TCP for network communications and listens to two ports. The primary port is configurable and is the port used by client applications. Its default value is 2836. The secondary port, is used for cluster discovery mechanisms and queries run on separate, higher priority partitions. This design ensures that when the server is busy processing queries, it can still service cluster management requests and preserve stability.

The value of the secondary port is set to the value of the primary port plus one (1). Thus, by default, the value of the secondary port is 2837.

Variable

Description

local.network.listen_on

Local address and port to bind to for accepting new connections. Can also refer to network adapter, e.g. eth0:2836. Defaults to 127.0.0.1:2836. The daemon will listen to the specified port and the port + 1 for service messages. For example, if the server is configured to listen to 127.0.0.1:2836 it will also listen to 127.0.0.1:2837.

local.network.advertise_as

Specify if the local address we bind to is not the same as other nodes or clients should use to connect to the daemon (e.g. when behind a NAT). Defaults to 127.0.0.1:2836.

local.network.idle_timeout

Time (in milliseconds) until a server socket is automatically recycled.

local.network.client_timeout

Time (in milliseconds) until a client connection without activity is automatically disconnected.

local.network.max_in_buffer_size

Size of the buffer the server allocates for receiving data from clients. Increase this size if you experience unexpected closed connections from the client, or the server logs mention cannot send data: Broken pipe.

local.network.max_out_buffer_size

Size of the buffer the server allocates for sending data to clients. Increase this size if you experience unexpected closed connections from the client, or the server logs mention cannot send data: Broken pipe.

global.cluster.publish_firehose

Enable changes publications on the firehose.

local.network.firehose_endpoint

Endpoint the firehose should listen to subscribers. Only active if the firehose publisher is enabled.

### 2.1.4. Clustering¶

Variable

Description

local.chord.node_id

local.chord.bootstrapping_peers

One or more peers that should be connected to to discover and bootstrap the cluster. Peers should be specified as a JSON array of tuples of address:port, e.g. ["192.168.1.1:2836", "192.168.1.2:2836"].

local.chord.replication_factor

Replication factor of data stored inside the cluster. Must be equal to or lower than the amount of nodes in your cluster, and not higher than 4. A replication factor of 1 means no replication.

### 2.1.5. Security¶

Variable

Description

global.security.enabled

Boolean that determines whether security should be enabled. Valid values are true and false. Defaults to false. Enabling this to true requires you to set valid values for user_list and cluster_private_file.

global.security.user_list

Absolute location to the file that contains the QuasarDB users. Typically should be /etc/qdb/users.txt. The file can be created and managed using the QuasarDB user management tool, and needs to be identically replicated accross all QuasarDB daemon nodes.

global.security.cluster_private_file

Absolute location to the cluster private key file. Typically should be /etc/qdb/cluster_private.key. The file can be created and managed using the QuasarDB cluster keygen, and needs to be identically replicated accross all QuasarDB daemon nodes.

global.security.encrypt_traffic

Boolean that determines whether node-to-node communication should be encrypted. Valid values are true and false. Defaults to false.

### 2.1.6. Performance tuning¶

Variable

Description

local.network.server_sessions

How many sessions (connections) each server thread is able to handle concurrently. Too few will cause significant performance degradations. Increase this value if you see “out of free server sessions” messages in the logs.

local.network.partitions_count

The number of threads in the main worker pool. Handles blocking I/O operations, and should be increased if your clients are waiting on the server but the server is not at 100% CPU. These are the queries handled by the main port.

local.network.hi_partitions_count

The number of threads dedicated for high-priority, service, messages. The default value of 2 is adequate for most setups and should be only increased for cluster with more than 20 nodes. These are the queries handled by the secondary port.

local.depot.rocksdb.max_open_files

Maximum number of open files that the storage engine is allowed to use. This should typically be close to the number of files the qdb user is allowed to open (i.e. the output of ulimit -n). Too low will cause significant performance degradations, and you are recommended to use at least 60000 in production.

local.limiter.max_bytes_soft

The maximum amount of memory (in bytes) QuasarDB will use. Any usage above this will cause QuasarDB to evict entries from its cache.

local.limiter.max_bytes_hard

Same like local.limiter.max_bytes_soft, except that processing is temporarily halted when this limit is reached.

### 2.1.7. Observability¶

Variable

Description

local.logger.log_level

An integer representing verbosity. Use 0 for most verbose logs, 5 for only fatal logs. Defaults to 2. Level 1 (debug) will also enable separate storage engine logs.

local.logger.log_directory

Absolute path where log files will be written into. Defaults to /var/log/qdb.

local.network.profile_performance

Enables performance profiling from the server side, so that clients can start running performance traces. It has a small performance impact, and is disabled by default.

global.cluster.enable_statistics

Enables runtime statistics collected server side. It has a small performance impact, and is disabled by default.

global.cluster.statistics_refresh_interval

Time (in milliseconds) between refreshes of the statistics. Should typically be set to the interval in which you query the node statistics; e.g. if you only poll your statistics once a minute, this value should be 60000.

### 2.1.8. Asynchronous Timeseries Inserter¶

The asynchronous timeseries inserter is a mechanism in QuasarDB to buffer inserts server-side, which you can tune using the configuration options below.

Variable

Description

local.depot.async_ts.pipelines

The number of asynchronous time series pipelines. Increase this number if you are experiencing slow inserts, but CPU is not yet at 100%.

local.depot.async_ts.pipeline_buffer_size

Maximum size of the buffer in bytes for each of the pipelines. Increase this value to if you are experiencing slow inserts and CPU is at 100%.

local.depot.async_ts.pipeline_queue_length

Maximum number of entries to buffer for each of the pipelines. Increase this value to if you are experiencing slow inserts and CPU is at 100%.

local.depot.async_ts.flush_deadline

Maximum time duration between consequent pipeline flushes.

## 2.2. QuasarDB Rest Server¶

The QuasarDB Rest Server qdb_rest can be configured using either command line arguments, a json config file or environmental variables. See below for examples of each configuration as well as the various configuration options:

# Environment variable based configuration
QDB_CLUSTER_URI=qdb://192.168.0.1:2837 qdb_rest

# Command line based configuration
qdb_rest --cluster=qdb://192.168.0.1:2837

# File based configuration
qdb_rest --gen-config > rest.conf # Generate a new config file and edit
qdb_rest --config-file rest.conf


Command Line Variable

Environment Variable

Description

Default

--allowed-origins

Allowed origins for cross origin resource sharing

[]

--cluster

QDB_CLUSTER_URI

URI of the cluster to connect to

qdb://127.0.0.1:2836

--cluster-public-key-file

QDB_CLUSTER_PUBLIC_KEY_FILE

Key file used for cluster security

--tls-certificate

TLS_CERTIFICATE

The certificate to use for secure connections

--tls-key

TLS_PRIVATE_KEY

The private key to use for secure conections

--tls-port

TLS_PORT

The port to listen on for secure connections

40493

--host

HOST

The IP to listen on

localhost

--port

PORT

The port to listen on for insecure connections

40080

--log-file

QDB_REST_LOG_FILE

The path to the log file

/var/log/qdb/qdb_rest.log

--assets-dir

QDB_REST_ASSETS_DIR

The path to the directory served by the rest api

/var/lib/qdb/assets

## 2.3. Docker¶

As it can a bit tedious to edit a configuration file inside a Docker container, the bureau14/qdb docker container provides several environment variables you can use to configure the most common configuration options.

Below you find an overview of the different environment variables the image supports:

Variable

Description

Example usage

QDB_LICENSE

docker run -d \
bureau14/qdb


QDB_LICENSE_FILE

docker run -d \
-v /path/to/my/qdb.key:/qdb.key \
bureau14/qdb


QDB_REPLICATION

Controls replication factor, defaults to 1 (no replication)

docker run -d \
-e QDB_REPLICATION=3 \
bureau14/qdb


QDB_MEMORY_LIMIT_SOFT

Controls the soft memory limit, defaults to 50% of host memory.

docker run -d \
-e QDB_MEMORY_LIMIT_SOFT=17179869184 \
-e QDB_MEMORY_LIMIT_HARD=25769803776 \
bureau14/qdb


QDB_MEMORY_LIMIT_HARD

Controls the hard memory limit, defaults to 75% of host memory.

docker run -d \
-e QDB_MEMORY_LIMIT_SOFT=17179869184 \
-e QDB_MEMORY_LIMIT_HARD=25769803776 \
bureau14/qdb


QDB_ENABLE_SECURITY

If “true”, enables security; requires private key and user list to be specified

docker run -d \
-e QDB_ENABLE_SECURITY=true \
-e QDB_CLUSTER_PRIVATE_KEY=... \
-e QDB_USER_LIST=... \
bureau14/qdb


QDB_PRIVATE_KEY

QuasarDB cluster private key (as string)

docker run -d  \
-e QDB_ENABLE_SECURITY=true \
-e QDB_CLUSTER_PRIVATE_KEY=... \
-e QDB_USER_LIST=... \
bureau14/qdb


QDB_PRIVATE_KEY_FILE

QuasarDB cluster private key (as file)

docker run -d \
-e QDB_ENABLE_SECURITY=true \
\
-v /path/to/my/qdb/private.key:/private.key \
-e QDB_CLUSTER_PRIVATE_KEY_FILE=/private.key \
\
-e QDB_USER_LIST=... \
bureau14/qdb


QDB_USER_LIST

QuasarDB user list (as string)

docker run -d \
-e QDB_ENABLE_SECURITY=true \
-e QDB_CLUSTER_PRIVATE_KEY=... \
-e QDB_USER_LIST=... \
bureau14/qdb


QDB_USER_LIST_FILE

QuasarDB user list (as file)

docker run -d \
-e QDB_ENABLE_SECURITY=true \
-e QDB_CLUSTER_PRIVATE_KEY=... \
\
-v /path/to/my/qdb/user.list:/user.list \
-e QDB_USER_LIST_FILE=/user.list \
\
bureau14/qdb
`