S3 Parquet Import

Credentials and Configuration

To read data from S3, set up the credentials and S3 region:

Query

CREATE SECRET (    TYPE s3,    KEY_ID '⟨AKIAIOSFODNN7EXAMPLE⟩',    SECRET '⟨wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY⟩',    REGION '⟨us-east-1⟩');

Tip

If you get an IO Error (Connection error for HTTP HEAD), configure the endpoint explicitly via ENDPOINT 's3.⟨your-region⟩.amazonaws.com'.

Querying

Once the S3 configuration is set correctly, Parquet files can be read from S3 using the following command:

Query

SELECT * FROM read_parquet('s3://⟨bucket⟩/⟨file⟩');

Result

 id | name  | value----+-------+-------  1 | alpha |    10  2 | beta  |    20

Google Cloud Storage (GCS) and Cloudflare R2

SereneDB can also handle Google Cloud Storage (GCS) and Cloudflare R2 via the S3 API. See the relevant guides for details.