Storage¶

Common properties¶

All blob stores have the default, id, and enabled properties.

• default is an optional attribute which defines if the blob store is the default one. Only one blob store can have this attribute set to true. Having more than one blob store with default="true" will raise an exception at startup time. Yet, if no blob store has default="true", a default FileBlobStore will be automatically created at the directory specified by the GEOWEBCACHE_CACHE_DIR application argument for backwards compatibility.

• id is a mandatory string property defining a unique identifier for the blobstore for the geowebcache instance. Not defining a unique id for a blobstore, or configuring more than one with the same id, will raise an exception at application startup time. This identifier can then be referred to by the blobStoreId element of a wmsLayer in the same configuration file, in order to explicitly state which blob store to use for a given layer.

• enabled is an optional attribute that defaults to true. If a blobstore is not enabled (i.e. <enabled>false</enabled>), then it cannot be used and any attempt to store or retrieve a tile from it will result in a runtime exception making the operation fail. Note that it is invalid to have the default="true" and <enabled>false</enabled> properties at the same time, resulting in a startup failure.

Besides these common properties, each kind of blob store defines its own, as follows:

File Blob Store¶

The file blob store saves tiles on disk following the traditional geowebcache cache directory layout.

Example:

<FileBlobStore default="false">
  <id>defaultCache</id>
  <enabled>false</enabled>
  <baseDirectory>/opt/defaultCache</baseDirectory>
  <fileSystemBlockSize>4096</fileSystemBlockSize>
</FileBlobStore>

Properties:

• baseDirectory: Mandatory. The absolute path for the cache’s root directory.

• fileSystemBlockSize: Optional, defaults to 4096. A positive integer representing the file system block size (usually 4096, 8292, or 16384, depending on the file system where the base directory resides. This value is used to pad the size of tile files to the actual size of the file on disk before notifying the internal blob store listeners when tiles are stored, deleted, or updated. This is useful, for example, for the “disk-quota” subsystem to correctly compute the cache’s disk usage.

Amazon Simple Storage Service (S3) Blob Store¶

The following documentation assumes you’re familiar with the Amazon Simple Storage Service.

This blob store allows to configure a cache for layers on an S3 bucket with the following TMS-like key structure:

[prefix]/<layer id>/<gridset id>/<format id>/<parameters hash | “default”>/<z>/<x>/<y>.<extension>

• prefix: if provided in the configuration, it will be used as the “root path” for tile keys. Otherwise the keys will be built starting at the bucket’s root.

• layer id: the unique identifier for the layer. Note it equals to the layer name for standalone configured layers, but to the geoserver catalog’s object id for GeoServer tile layers.

• gridset id: the name of the gridset of the tile

• format id: the gwc internal name for the tile format. E.g.: png, png8, jpeg, etc.

• parameters hash: if the request that originated that tiles included parameter filters, a unique hash code of the set of parameter filters, otherwise the constant default.

• z: the z ordinate of the tile in the gridset space.

• x: the x ordinate of the tile in the gridset space.

• y: the y ordinate of the tile in the gridset space.

• extension: the file extension associated to the tile format. E.g. png, jpeg, etc. (Note the extension is the same for the png and png8 formats, for example).

Support for S3-compatible servers other than Amazon is also present.

Configuration example:

<S3BlobStore default="false">
  <id>myS3Cache</id>
  <enabled>false</enabled>
  <bucket>put-your-actual-bucket-name-here</bucket>
  <prefix>test-cache</prefix>
  <awsAccessKey>putYourActualAccessKeyHere</awsAccessKey>
  <awsSecretKey>putYourActualSecretKeyHere</awsSecretKey>
  <access>private</access>
  <maxConnections>50</maxConnections>
  <useHTTPS>true</useHTTPS>
  <endpoint>http://putYourServerEndpointHereOrLeaveOutIfUsingAmazon:9000</endpoint>
  <proxyDomain></proxyDomain>
  <proxyWorkstation></proxyWorkstation>
  <proxyHost></proxyHost>
  <proxyPort></proxyPort>
  <proxyUsername></proxyUsername>
  <proxyPassword></proxyPassword>
  <useGzip>true</useGzip>
</S3BlobStore>

Properties:

• bucket: Mandatory. The name of the AWS S3 bucket where to store tiles.

• prefix: Optional. A prefix path to use as the “root folder” to store tiles at. For example, if the bucket is bucket.gwc.example and prefix is “mycache”, all tiles will be stored under bucket.gwc.example/mycache/{layer name} instead of bucket.gwc.example/{layer name}.

• awsAccessKey: Mandatory. The public access key the client uses to connect to S3.

• awsSecretKey: Mandatory. The secret key the client uses to connect to S3.

• access: Optional. Whether direct access in S3 will be readable by the public or only to the owner of the bucket. Defaults to public, set to private to disable public access.

• maxConnections: Optional, default: 50. Maximum number of concurrent HTTP connections the S3 client may use.

• useHTTPS: Optional, default: true. Whether to use HTTPS when connecting to S3 or not.

• endpoint: Optional. Endpoint of the server, if using an alternative S3-compatible server instead of Amazon.

• proxyDomain: Optional. The Windows domain name for configuring an NTLM proxy. If you are not using a Windows NTLM proxy, you don’t need to set this property.

• proxyWorkstation: Optional. The Windows domain name for configuring an NTLM proxy. If you are not using a Windows NTLM proxy, you don’t need to set this property.

• proxyHost: Optional. The proxy host the client will connect through.

• proxyPort: Optional. The proxy port the client will connect through.

• proxyUsername: Optional. The proxy user name to use if connecting through a proxy.

• proxyPassword: Optional. The proxy password to use when connecting through a proxy.

• useGzip: Optional, default: true. Whether gzip compression should be used when transferring tiles to/from S3.

Note: It is possible to set above properties from environment variable as long as they are of string type. In the example below, The awsAccessKey is set from environment variable named AWS_ACCESS_KEY

<awsAccessKey>${AWS_ACCESS_KEY}</awsAccessKey>

<div class="row-fluid">
  <div class="span12">
    <div id="map" class="map"></div>
  </div>
</div>

var map = new ol.Map({
  target: 'map',
  controls: ol.control.defaults(),
  layers: [
    new ol.layer.Tile({
      source: new ol.source.XYZ({
        projection: "EPSG:4326",
        url: 'http://my-geowebcache-bucket.s3.amazonaws.com/test-cache/topp%3Astates/EPSG%3A4326/png8/default/{z}/{x}/{-y}.png'
      })
    })
  ],
  view: new ol.View({
    projection: "EPSG:4326",
    center: [-104, 39],
    zoom: 2
  })
});

Google Cloud Storage (GCS) Blob Store¶

This blob store allows to configure a cache for layers on a Google Cloud Storage bucket with the same TMS-like key structure as S3:

[prefix]/<layer id>/<gridset id>/<format id>/<parameters hash | “default”>/<z>/<x>/<y>.<extension>

Configuration example:

<GoogleCloudStorageBlobStore default="false">
  <id>myGcsCache</id>
  <enabled>true</enabled>
  <bucket>my-gwc-bucket</bucket>
  <prefix>test-cache</prefix>
  <projectId>my-gcp-project</projectId>
  <useDefaultCredentialsChain>true</useDefaultCredentialsChain>
</GoogleCloudStorageBlobStore>

Properties:

• bucket: Mandatory. The name of the GCS bucket where to store tiles.

• prefix: Optional. A prefix path to use as the “root folder” to store tiles at.

• projectId: Optional. The GCP project ID. Can be omitted if using service account credentials that already specify the project.

• quotaProjectId: Optional. Project to bill for quota when using requester-pays buckets.

• endpointUrl: Optional. Custom endpoint URL for use with GCS emulators or compatible services.

• useDefaultCredentialsChain: Optional. Set to true to use Application Default Credentials. This will look for credentials in the following order: environment variable GOOGLE_APPLICATION_CREDENTIALS pointing to a service account key file, GCE/GKE metadata service, or gcloud CLI credentials.

• apiKey: Optional. API key for authentication. If both apiKey and useDefaultCredentialsChain are provided, apiKey takes precedence.

Note: Like S3, all configuration properties support environment variable expansion using the ${VARIABLE_NAME} syntax:

<bucket>${GCS_BUCKET}</bucket>
<projectId>${GCS_PROJECT_ID}</projectId>

Authentication options:

• Application Default Credentials (recommended): Set useDefaultCredentialsChain to true. This works automatically on GCE/GKE and when GOOGLE_APPLICATION_CREDENTIALS points to a service account key.

• API Key: Set the apiKey property. Less secure, mainly for testing.

• No auth: For use with emulators only. Leave both auth options unset.

Implementation notes:

Delete operations run asynchronously in a background thread pool. When deleting tile ranges or layers, tiles are removed in batches using the GCS batch API for efficiency. The thread pool is sized based on available processors and shuts down gracefully on blob store destruction.

Microsoft Azure Blob Store¶

The following documentation assumes you’re familiar with the Azure BLOB storage.

This blob store allows to configure a cache for layers on an Azure container with the following TMS-like key structure:

[prefix]/<layer id>/<gridset id>/<format id>/<parameters hash | “default”>/<z>/<x>/<y>.<extension>

• prefix: if provided in the configuration, it will be used as the “root path” for tile keys. Otherwise the keys will be built starting at the bucket’s root.

• layer id: the unique identifier for the layer. Note it equals to the layer name for standalone configured layers, but to the geoserver catalog’s object id for GeoServer tile layers.

• gridset id: the name of the gridset of the tile

• format id: the gwc internal name for the tile format. E.g.: png, png8, jpeg, etc.

• parameters hash: if the request that originated that tiles included parameter filters, a unique hash code of the set of parameter filters, otherwise the constant default.

• z: the z ordinate of the tile in the gridset space.

• x: the x ordinate of the tile in the gridset space.

• y: the y ordinate of the tile in the gridset space.

• extension: the file extension associated to the tile format. E.g. png, jpeg, etc. (Note the extension is the same for the png and png8 formats, for example).

Configuration example:

<AzureBlobStore default="false">
  <id>myAzureCache</id>
  <enabled>false</enabled>
  <container>put-your-actual-container-name-here</container>
  <prefix>test-cache</prefix>
  <accountName>putYourActualAccountNameHere</accountName>
  <accountKey>putYourActualAccountKeyHere</accountKey>
  <maxConnections>100</maxConnections>
  <useHTTPS>true</useHTTPS>
  <serviceURL>http://putYourServerEndpointHereOrLeaveOutIfUsing.blob.core.windows.net</serviceURL>
  <proxyHost></proxyHost>
  <proxyPort></proxyPort>
  <proxyUsername></proxyUsername>
  <proxyPassword></proxyPassword>
</AzureBlobStore>

Properties:

• container: Mandatory. The name of the Azure container where to store tiles. The code will try to create it if missing.

• prefix: Optional. A prefix path to use as the “root folder” to store tiles at. For example, if the container is gwc.example and prefix is “mycache”, all tiles will be stored under gwc.example/mycache/{layer name} instead of gwc.example/{layer name}.

• accountName: Mandatory. The account name used to connect to Azure storage (found in the archiving account, choose the account, and then access keys).

• accountKey: Mandatory. The secret key the client uses to connect to S3.

• maxConnections: Optional, default: 100. Maximum number of concurrent HTTP connections the client may use. The more the merrier, as Azure REST API does not have support for bulk deletes, so each tile needs to be deleted in a separate request on cleanup.

• useHTTPS: Optional, default: true. Whether to use HTTPS when connecting to Azure or not.

• serviceURL: Optional. The full service URL, in case the default one is not suitable. The default is build using the account name, e.g. https://accountName.blob.core.windows.net

• proxyHost: Optional. The proxy host the client will connect through.

• proxyPort: Optional. The proxy port the client will connect through.

• proxyUsername: Optional. The proxy user name to use if connecting through a proxy.

• proxyPassword: Optional. The proxy password to use when connecting through a proxy.

Unlike S3, access level in Azure can be set at the container level only, so if you desired to pre-seed a publicly available cache, please create a container that has “public” or “BLOB” access level. The access level can be modified also after the container creation.

<div class="row-fluid">
  <div class="span12">
    <div id="map" class="map"></div>
  </div>
</div>

var map = new ol.Map({
  target: 'map',
  controls: ol.control.defaults(),
  layers: [
    new ol.layer.Tile({
      source: new ol.source.XYZ({
        projection: "EPSG:4326",
        url: 'https://<accountNameHere>.blob.core.windows.net/<containerNameHere>/<prefixIfAny>/<layerId>/EPSG:4326/png8/default/{z}/{x}/{-y}.png'
      })
    })
  ],
  view: new ol.View({
    projection: "EPSG:4326",
    center: [-104, 39],
    zoom: 2
  })
});

OpenStack Swift (Swift) Blob Store¶

The following documentation assumes you’re familiar with the Openstack Swift Documentation.

This blob store allows to configure a cache for layers using a Swift container with the following TMS-like key structure:

[prefix]/<layer id>/<gridset id>/<format id>/<parameters hash | “default”>/<z>/<x>/<y>.<extension>

• prefix: if provided in the configuration, it will be used as the “root path” for tile keys. Otherwise the keys will be built starting at the bucket’s root.

• layer id: the unique identifier for the layer. Note it equals to the layer name for standalone configured layers, but to the geoserver catalog’s object id for GeoServer tile layers.

• gridset id: the name of the gridset of the tile

• format id: the gwc internal name for the tile format. E.g.: png, png8, jpeg, etc.

• parameters hash: if the request that originated that tiles included parameter filters, a unique hash code of the set of parameter filters, otherwise the constant default.

• z: the z ordinate of the tile in the gridset space.

• x: the x ordinate of the tile in the gridset space.

• y: the y ordinate of the tile in the gridset space.

• extension: the file extension associated to the tile format. E.g. png, jpeg, etc. (Note the extension is the same for the png and png8 formats, for example).

Configuration example:

<SwiftBlobStore default="true">
    <id>ObjectStorageCache</id>
    <enabled>true</enabled>
    <container>put-your-actual-container-name-here</container>
    <prefix>test-cache</prefix>
    <endpoint>endpoint</endpoint>
    <provider>openstack-swift</provider>
    <region>put-region-here</region>
    <keystoneVersion>3</keystoneVersion>
    <keystoneScope>project</keystoneScope>
    <keystoneDomainName>Default</keystoneDomainName>
    <identity>put-tenant-name-here:put-username-here</identity>
    <password>put-password-here</password>
</SwiftBlobStore>

Properties:

• container: Mandatory. The name of the Swift container where to store tiles.

• prefix: Optional. A prefix path to use as the “root folder” to store tiles at. For example, if the bucket is bucket.gwc.example and prefix is “mycache”, all tiles will be stored under bucket.gwc.example/mycache/{layer name} instead of bucket.gwc.example/{layer name}.

• endpoint: Manditory. Endpoint of the server

• provider: Mandatory. Jclouds provider (shouldn’t need modifying)

• region: Mandatory. Swift region for container.

• keystoneVersion: Mandatory. Keystone version

• keystoneScope: Optional. For scoped keystone authorization (project or domain scoped)

• keystoneDomainName: Optional. Keystone domain name (if different than the user domain)

• identity: Mandatory. Identity used to authenticate with the swift API (format - tenantName:username)

• password: Mandatory. Password used to authenticate with the swift API.