Partitioning tables

Table partitioning is a technique that allows you to divide a large table into smaller, more manageable parts called “partitions”.

Each partition contains a subset of the data based on a specified criteria, such as a range of values or a specific condition. Partitioning can significantly improve query performance and simplify data management for large datasets.

Benefits of table partitioning#

• Improved query performance: allows queries to target specific partitions, reducing the amount of data scanned and improving query execution time.
• Scalability: With partitioning, you can add or remove partitions as your data grows or changes, enabling better scalability and flexibility.
• Efficient data management: simplifies tasks such as data loading, archiving, and deletion by operating on smaller partitions instead of the entire table.
• Enhanced maintenance operations: can optimize vacuuming and indexing, leading to faster maintenance tasks.

Partitioning methods#

Postgres supports various partitioning methods based on how you want to partition your data. The commonly used methods are:

1. Range Partitioning: Data is divided into partitions based on a specified range of values. For example, you can partition a sales table by date, where each partition represents a specific time range (e.g., one partition for each month).
2. List Partitioning: Data is divided into partitions based on a specified list of values. For instance, you can partition a customer table by region, where each partition contains customers from a specific region (e.g., one partition for customers in the US, another for customers in Europe).
3. Hash Partitioning: Data is distributed across partitions using a hash function. This method provides a way to evenly distribute data among partitions, which can be useful for load balancing. However, it doesn't allow direct querying based on specific values.

Creating partitioned tables#

Let's consider an example of range partitioning for a sales table based on the order date. We'll create monthly partitions to store data for each month:

To create a partitioned table you append partition by range (<column_name>) to the table creation statement. The column that you are partitioning with must be included in any unique index, which is the reason why we specify a composite primary key here (primary key (order_date, id)).

Querying partitioned tables#

To query a partitioned table, you have two options:

1. Querying the parent table
2. Querying specific partitions

Querying the parent table#

When you query the parent table, Postgres automatically routes the query to the relevant partitions based on the conditions specified in the query. This allows you to retrieve data from all partitions simultaneously.

Example:

This query will retrieve data from both the sales_2000_01 and sales_2000_02 partitions.

Querying specific partitions#

If you only need to retrieve data from a specific partition, you can directly query that partition instead of the parent table. This approach is useful when you want to target a specific range or condition within a partition.

This query will retrieve data only from the sales_2000_02 partition.

When to partition your tables#

There is no real threshold to determine when you should use partitions. Partitions introduce complexity, and complexity should be avoided until it's needed. A few guidelines:

• If you are considering performance, avoid partitions until you see performance degradation on non-partitioned tables.
• If you are using partitions as a management tool, it's fine to create the partitions any time.
• If you don't know how you should partition your data, then it's probably too early.

Examples#

Here are simple examples for each of the partitioning types in Postgres.

Range partitioning#

Let's consider a range partitioning example for a table that stores sales data based on the order date. We'll create monthly partitions to store data for each month.

In this example, the sales table is partitioned into two partitions: sales_january and sales_february. The data in these partitions is based on the specified range of order dates:

List partitioning#

Let's consider a list partitioning example for a table that stores customer data based on their region. We'll create partitions to store customers from different regions.

In this example, the customers table is partitioned into two partitions: customers_americas and customers_asia. The data in these partitions is based on the specified list of regions:

Hash partitioning#

You can use hash partitioning to evenly distribute data.

In this example, the products table is partitioned into two partitions: products_one and products_two. The data is distributed across these partitions using a hash function:

Other tools#

There are several other tools available for Postgres partitioning, most notably pg_partman. Native partitioning was introduced in Postgres 10 and is generally thought to have better performance.