Jul 10th 2015How to persist folders and files with dokku and docker-options

If you want to persist folders or files for an application deployed with dokku, here is how I got it to work. To take you through the steps, we will be creating a little node app that creates a file on each startup/deploy and displays all the files created when you hit the / route.

TL;DR: If you don't enjoy the handholding, you can skip straight to the part where we mount our volumes: Initialize the app.

First, let's create a simple package.json to get us started:

package.json

{
  "scripts": {
    "start": "node main"
  },
  "engines": {
    "iojs": "2.x"
  }
}

Install express

Then, let's install our only dependency, express:

$ npm i express -S

Also, let's make that our folder (to which we will write our files) exists:

$ mkdir -p storage

Now for our little application:

main.js

'use strict';

const fs = require('fs');
const express = require('express');
const path = require('path');

const port = process.env.PORT || 3000;
const dirName = process.env.STORAGE_DIR || path.join(__dirname, 'storage');
const app = express();

fs.writeFileSync(path.join(dirName, '' + Date.now()), '');
console.log('wrote to ' + dirName);

app.get('/', function(req, res) {
  fs.readdir(dirName, function(err, files) {
    if (err) {
      res.status(500).end('Unkown error');
      console.error(err.stack || err);
      process.exit(1);
    }

    files.forEach(function(file) {
      res.write(file + '\n');
    });

    res.end();
  });
});

app.listen(port);

Notice that I am using an environment variable STORAGE_DIR, because we will set this in our production environment.

If you run node main and visit localhost:3000 you should see one or more newline-separated timestamps. Ok, let's get this ready for deployment with dokku.

.env

BUILDPACK_URL=https://github.com/heroku/heroku-buildpack-nodejs
STORAGE_DIR=/storage

Our .env file sets up some important environment variables. First, we'll be using heroku's node.js buildpack that lets us use io.js. Also, we're setting our STORAGE_DIR environment variable as announced previously.

.gitignore

node_modules
storage
.DS_Store

Put the usual suspects in your .gitignore (don't forget the storage folder). Now initialize git:

Initialize git

$ git init
$ git add -a
$ git commit -m "initial commit"
$ git add remote dokku dokku@dokku:persistence

Initialize the app

$ ssh dokku apps:create persistence
$ ssh dokku docker-options:add persistence "run -v /home/apps/persistence/storage:/app/storage"
$ ssh dokku docker-options:add persistence "deploy -v /home/apps/persistence/storage:/app/storage"

Note that it is not a mistake that we are specifying /app/storage to be persisted although we specified STORAGE_DIR=/storage (without /app) in our .env file.

After that is done, we are ready to deploy:

Deploy the app

$ git push dokku master

When that is done, you can run

$ ssh dokku logs persistence

to see something like this:

Detected 512 MB available memory, 512 MB limit per process (WEB_MEMORY)
Recommending WEB_CONCURRENCY=1

> @ start /app
> node main

wrote to /app/storage

Note again that the console says we are writing to /app/storage and not /storage, even though STORAGE_DIR=/storage.

Visiting the app in your browser will give you a single timestamp.

Making sure everything works

First, you will want to see if your options are set correctly:

$ ssh dokku docker-options persistence

You should see something like this:

Deploy options:
    -v /home/apps/persistence/storage:/app/storage
Run options:
    -v /home/apps/persistence/storage:/app/storage

Next, you can ssh into your machine, look at the persisted directory

$ ls /home/apps/persistence/storage/

and you should see a single file with a timestamp as a name.

Ok, everything looks good. Now let's rebuild our app and see if we get a second timestamp added while keeping the first one we created:

$ ssh dokku ps:rebuild persistence

When the build is done, you should be able to revisit your site and see that a second timestamp was added. It works!

Destroying persisted apps

Please be aware that when you destroy your app by running

$ ssh dokku apps:destroy persistence

the persisted folders will stay on the remote system and you will need to delete them manually if you want them gone.