Install and upgrade

On this page:

1. Requirements
2. Installation
3. Configuration
4. Plugins
5. Going to Prod
6. Upgrade

Requirements

All you need is a Linux server with these tools installed:

• docker
• docker-compose
• curl

Hardware requirement is minimal: 128MB of RAM and 1 core will suffice, provided you’re not doing intensive video transcoding and/or serving thousands of users simultaneously.

Installation

The official Docker image is available on Docker Hub

After installation, navigate to http://your_domain:8334 to configure your admin password:

Configuration

For a basic setup, simply choose the storage backends you need, and you’re all set:

This is the setup we use for the publicly available demo. If that’s all you need, you can jump straight to going to prod, as everything that follows is totally optional and involves the more advanced stuff of delegating authentication and authorisation.

Advanced Authentication Methods

Now that we’ve covered the basics, let’s dive into the cool stuff, connecting your storage to different authentication sources. There are plenty of valid strategies, but they all fall into one of two buckets:

• the passthrough pattern
• the facade pattern

Advanced Authentication - Passthrough Pattern

It is the right choice for you if:
you prefer the authentication to be handled by the storage layer, meaning the authentication details are passed directly to the storage.

What does that look like in practice?

Using this configuration, users will see a much simpler login page:

At this point, the user experience is much improved over our starting point, as users no longer need to remember details like the “hostname” and “port”. The connection combines static values, like the hardcoded “ftp.gnu.org” hostname from the admin configuration, with dynamic variables such as the “username” and “password” provided by the user.

Another common strategy is to create automatic login links. To set this up, simply change the strategy in the configuration to “direct” and configure everything statically.

Filestash has 3 plugins that support the passthrough pattern:

1. the “Passthrough” middleware plugin, showcased in this section
2. the “URL” middleware plugin*, which exposes URL parameters as dynamic variables in the configuration, and supporting signed URLs as an option. It’s ideal for generating secure deep links and QR codes.
3. the “Proxy” middleware plugin*, which is if you have a proxy sitting in between and forwarding headers like X-Remote-User or X-Remote-Group header.

Advanced Authentication - Facade Pattern

It is the right choice for you if:
You prefer users to authenticate through a separate system, with options to grant various type of access based on their identity.

This approach creates a “facade”, forcing users to authenticate through one of the following methods:

1. The Apache style htpasswd file
2. The Local Auth plugin to authenticate user locally with a nice management GUI if you think htpasswd is too “rough” and/or need to associated roles to local users.
3. Enterprise SSO plugins: OpenID, SAML, LDAP*
4. An external HTTP service*

Using the Apache style authentication middleware, it looks like this:

In this setup, we have 2 valid users, each with their own username and password. The path variable is dynamically generated based on the authenticated user. Concretely, this means the user “rick” has full access to all S3 buckets, while “jerry” is restricted to the “earth” bucket only.

Under the hood, these mappings are handled using the Go templating language, enhanced with custom functions to support more advanced use cases. For example, you can dynamically generate chroot paths based on AD groups by building queries like this: {{ if .memberOf | contains “ADMIN”}}/{{ else }}/{{ .department }}/{{ end }}

Authorisation

The enterprise build allows you to define and restrict the actions users are allowed to perform:

In this example, we enforce read only access to our storage. This effectively blocks users from creating new files or folders, uploading content, moving items, or deleting anything.

Similarly to what we did earlier, you can create rules based on any attribute returned by your IDP:

In this example, rick is allowed to perform all actions everywhere, while everyone else is limited to read only access, meaning they can list files within the “earth” bucket and download its contents.

If your project’s authorization requirements exceed the capabilities of the Go template mechanics, a custom “authorization plugin” is the way forward. An “authorization plugin” is a Filestash plugin implementing this interface.

To give a concrete example, one of our customers is this property management firm. Each property they manage is represented as a folder in their filesystem, with users assigned roles for each property, either as a “homeowner”, “board member”, “committee member”, “manager”, or “staff”. These roles grant users different access levels. For example, “homeowners” cannot view the accounting folder, “committee members” have read-only access to committee reports, and “board members” have write access to those reports. Only “managers” can delete files, while “staff” act as company administrators with full access across the entire portfolio of properties. For this highly customized use case, we implemented all their access control needs via an “authorization plugin”. Within a week of the initial call, they had a fully functional turnkey solution.

Plugins

So far, we’ve touched the 3 pillars of a solid bespoke solution:

1. Storage
2. Authentication Middleware
3. Authorization

To see all the plugins installed in your instance, visit the /about page:

Each plugin controls a specific aspect of how the software behaves. For example, “plg_starter_http” starts the server as an HTTP service, but there are many other options.

Fundamentally, Filestash is built as a framework for developing file management applications, and plugins are the lego blocks that enable customization of any of its aspects:

1. create a frontend override. Whenever you want to change something in the frontend, you can inject diffs into the core JavaScript code that will get applied at runtime. This technique is commonly used to customize branding for customers and create custom banners. An example of this can be found in the plg_override_download plugin, which adds an extra download button to the file list.
2. create another file viewer application. This approach allows you to add support for specific file types. For example, support for Microsoft office documents is implemented by the plg_editor_onlyoffice plugin, and there’s an alternative implementation that leverages the official Microsoft office viewer. We’ve used this technique to build fully custom viewers tailored to specific use cases.
3. enable various search mechanism. In Filestash, a search plugin implements this interface. The default build ships with this recursive search plugin, but we’ve also open-sourced a full-text search implementation and delivered custom plugins for customers integrating with their existing Solr or Elasticsearch infrastructure.
4. change how configuration is handled. By default, Filestash pulls its configuration from the filesystem, but customers have additional options to use environment variables or S3 as the source for configuration, making it simpler to operate in a cluster in a typical enterprise setup without having to sync config files around.
5. change how transcoding / thumbnailing work. By default, images are processed in real time to generate thumbnails via plg_image_c, and unsupported image formats (e.g., HEIF, RAW files) are transcoded to JPG. For those concerned about buffer overflows in C, a slower but safer implementation written entirely in Go is available. Additionally, video files can be transcoded into an HLS stream using the plg_video_transcoder plugin.
6. create additional endpoints. A lot of custom projects might require the creation of custom endpoints to handle all sorts of tasks. In the past, we’ve used this to generate QR codes, query an ERP for some information to be displayed in the frontend, serve as the target of an iframe, etc…
7. create a server override. Virtually any part of the code can be overriden and packaged as a “go generator” to override any part of the backend you don’t like.
8. build custom authorisation mechanisms. If you have a complex use case for how authorisation should be done, you can implement this interface to hook into all the API calls related to file management.
9. change how Filestash starts. We’ve previously referenced plg_starter_http, but other implementations exist on the open-source side, such as TOR

Those 9 very different type of plugin should cover 99% of the usage for plugin. To get the full picture and the remaining 1%, you will need to dig onto the source code, specially here and here.

Going to Prod

Enabling TLS

To deploy the base build in production, you will need a reverse proxy fronting the traffic and performing SSL termination there. That’s the setup powering the online demo, with nginx configured like this.

Customers have more flexibility in handling HTTPS, with additional options to expose Filestash as a native HTTPS service by either:

1. loading your existing certificates directly into Filestash
2. letting Filestash provision certificates at runtime via ACME and Let’s Encrypt. This method is “set and forget”, with no human intervention needed to renew certificates, no third party software, and no extra setup.

All those methods are documented here.

Mailing List

If you want to be in the loop about Filestash’s ongoing development, new releases and security updates, you might want to join the mailing list:

REGISTER

Enterprise Setup

Filestash is deployed in many sensitive environments with very stringent needs. If this sounds like what you are doing, you might want to read:

1. our hardening guide

2. some actual use cases we’ve worked on

3. some examples of architecture diagram that was made for a deployment in large pharma with requirements for high availability and scalability. You can read about that in this post from the AWS Blog:

Upgrade

To update Filestash: