How Filestash handle configuration

Filestash stores its entire configuration in a single file named config.json, located by default under the state/config folder. Everything you see in the admin console under the settings and storage pages is a projection of this file, wrapped in a UI to make it easy to edit and understand. In this guide, we will peel back the layers so you can make sense of how configuration works and what are the customisation points.

Structure of the configuration file

We already alluded to it but it’s worth repeating: the /admin/storage and /admin/settings pages are nothing but direct projections of the config.json file. This config.json file naturally splits into two parts:

  1. the storage section, mapping to /admin/storage
  2. the settings section, mapping to /admin/settings

PART1 - The storage page

In the config.json file, these are the fields relevant to the storage page:

{
    ...
    "connections": [ // <- (1) storage backend
        {"type": "s3", "label": "mystorage"}
    ],
    ...
    "middleware": { // <- (2) authentication middleware
        "identity_provider": {
            "type": "ldap",
            "params": "..."
        },
        "attribute_mapping": {
            "related_backend": "mystorage", // <- foreign keys to the storage backend
            "params": "..." // <- mapping rules for the storages defined in related_backend
        }
    },
    ...
}

If you visit the /admin/storage page, you will see 2 blocks:

  1. Storage Backend: defines which storage you want to use, whether that’s S3, SFTP, IPFS, or anything else. In the config.json, this is the connections key which contains a list of all the storage backends you have enabled. Each entry has a label that acts as a primary key, referenced by attribute_mapping.related_backend to link authentication to that storage.
  2. Authentication Middleware: optional. Without it, users see the raw login form and need to know the technical details of your storage (server address, access keys, etc.). Enable it to authenticate users through an IDP and automatically connect them to the right storage. In the config.json, this lives in the middleware section which has 2 pieces to:
    1. Configure which identity_provider to use (typically “openid”, “saml”, or “ldap”) along with the parameters for your IDP.
    2. Define an attribute_mapping to generate storage connections from the user’s identity, referencing which storage backend it should apply to.

The attribute mapping params field is a JSON string following this structure:

{
    "mystorage": {"type":"s3","access_key_id":"...","secret_access_key":"..."},
    "other": {"type":"blackhole"}
}

For the full list of fields available for each storage backend, see this guide.

PART2 - The settings page

Everything in config.json that isn’t connections or middleware belongs to the settings section. The structure is not fixed as plugins can extend it with their own configuration keys. For example, the SFTP gateway plugin adds settings for the private key and other options specific to running an SFTP service. Most plugins follow the same pattern.

If you open the config.json alongside the settings page, you will see how the JSON keys map directly to what you see in the UI:

{
   "general": {        // <- that's your first level title
       "name": ...
       ...
   },
   "features": {       // <- next first level title
       "api": {        // <- second level title
          ...
       },
       ...
   },
   ...
}

Alternative Adapters

The plugin inventory lists two alternatives to the default filesystem storage:

  1. plg_config_s3: stores your configuration in an S3 bucket, which is useful for cluster wide deployments where every replica needs to share the same config.

  2. plg_config_env: makes the configuration read only and self contained in a single environment variable. Generate it from an existing config with: CONFIG_JSON=$(cat config.json | base64 -w0)

Note: both of these are compiled plugins and must be built directly into the Filestash binary.

Creating your own Adapter

In line with our plugin based architecture, you can create your own configuration adapter. The approach is to write a Go generator that overrides config_state.go and implements two functions:

func LoadConfig() ([]byte, error) {
    // return the configuration as JSON
}

func SaveConfig(v []byte) error {
    // persist the JSON configuration
}

This is exactly how the adapters listed above work. The same mechanism powers undocumented variants too. For example, the AWS Marketplace build uses a custom adapter that stores config in S3 with partitioning based on the account ID, along with a few convenience features specific to that environment.