Introduction

This is part 2 about using an Ansible vault as a safe key store for sensitive variables for Docker compose files and maybe Dockhand.

In part 1 it is explained how the Ansible vault can be setup, using a file with the vault password to unlock the sensitive data.

In this article it is explored how the vault password can be cached for some time in memory and how it can be retrieved from memory and how this can be used for secrets in Docker compose files.

Wrapper Scripts

Verify that the following files are available via a bin directory, preferably in /usr/local/bin. If they're not available yet, download and install them. As docker-compose-vault-key is hardcoded called with /usr/local/bin from docker-compose-vault-password, the former (docker-compose-vault-key) must (for now) be installed in /usr/local/bin.

• docker-compose-vault
• docker-compose-vault-key
• docker-compose-vault-password

Aliases

To make life a bit easier aliases can be defined for convenience:

alias dc="docker-compose-vault"
alias dcc="docker-compose-vault config"
alias dcr="docker-compose-vault stop; docker-compose-vault up -d"
alias docker-vault="LC_ALL=C.UTF-8 ansible-vault"

Update Ansible_vault_password_file

Ensure that the variable ANSIBLE_VAULT_PASSWORD_FILE is set to:

ANSIBLE_VAULT_PASSWORD_FILE=/usr/local/bin/docker-compose-vault-password

Check with:

echo $ANSIBLE_VAULT_PASSWORD_FILE

Demo

Try it out.

Docker compose directory

whoami01 $ ls -l

-rw-rw-r-- 1 user user 1485 24 jul 21:55 compose.yaml
-rw------- 1 user user  484 24 jul 08:56 stack.env.vault

Example compose.yaml file:

services:
  whoami:
    image: traefik/whoami
    container_name: whoami${ID:?ID required variable}
    ports:
      - 8090:80
    restart: unless-stopped

    environment:
      - POSTGRES_USER=${VAULT_POSTGRES_USER:?POSTGRES_USER to be populated in dockhand}
      - POSTGRES_PASSWORD=${VAULT_POSTGRES_PASSWORD:?VAULT_POSTGRES_PASSWORD to be populated in dockhand}
      - POSTGRES_DB=dockhand
      - DCKR_TEST=${VAULT_DCKR_TEST:-unused}
      - DCKR_STND=${COMPOSE_PROJECT_NAME}

The file stack.env.vault contains sensitive variables.

The file with sensitive variables is encrypted, see part 1 how that is done.

Example unencrypted stack.env.vault file:

# Secret variables to be included in the Docker file
#
# Prepend the variables with VAULT_, so it is clear in the compose file, where the variable was defined
VAULT_TAG=v1.23
VAULT_VAR=ans-var
VAULT_POSTGRES_USER=pg_user
VAULT_POSTGRES_PASSWORD=pg_pass

Make sure the file is encrypted:

cat stack.env.vault

Response:

$ANSIBLE_VAULT;1.1;AES256
35303937313866363239643733306165636139393561386230303131343836326434643932353938
6435633465643733633865656165396631393534633234620a623239373533663935623666623561
61346162326133626334363634623864653731633430643764343465663239333631363137363633
3966633931643562640a393063656530626563626232613135353866313334636363316566626637
36356432323631313335313766623334383164333263393665626131366430393237353636393265
6263313537643363343738636234653936643764633534633038

The file with sensitive data can be stored safely in Git, as it is encrypted.

docker-vault view fails, as the vault password is not known:

docker-vault view stack.env.vault

Response:

[WARNING]: Error in vault password file loading (default): Vault password script /usr/local/bin/docker-compose-vault-password returned non-zero (1): None
[ERROR]: Vault password script /usr/local/bin/docker-compose-vault-password returned non-zero (1): None

docker-compose-vault config fails too, as the vault password is not known:

docker-compose-vault config

Response:

[WARNING]: Error in vault password file loading (default): Vault password script /usr/local/bin/docker-compose-vault-password returned non-zero (1): None
[ERROR]: Vault password script /usr/local/bin/docker-compose-vault-password returned non-zero (1): None
error while interpolating services.whoami.labels.example.variable.test: required variable VAULT_TAG is missing a value: VAULT_TAG must not be empty

The error is expected as no vault password is found nor provided.

Activate the vault password

Activate the vault password or key:

docker-compose-vault-key activate

Response:

Vault password:
Password cached for 900 seconds.

Request the status:

docker-compose-vault-key status

Response:

Docker compose vault status

Status      : cached
Key ID      : 88217422
Key name    : docker-compose-vault
Keyring     : @u
User        : user (1000)
Remaining   : 679s
Expires     : 12:40:51

Execute the following. Be carefull as the password will be shown, it all is working as expected.

$ANSIBLE_VAULT_PASSWORD_FILE

Response: vault key or password is shown.

As the vault key or password has been set, it can be used by docker-vault and docker-compose-vault.

docker-vault view stack.env.vault

Response:

...comments...
VAULT_TAG=v1.23
VAULT_VAR=ans-var
VAULT_POSTGRES_USER=pg_user
VAULT_POSTGRES_PASSWORD=pg_pass

And with:

docker-compose-vault config

Response:

name: whoami02
services:
  whoami:
    container_name: whoami02
    environment:
      DCKR_STND: whoami02
      DCKR_TEST: unused
      POSTGRES_DB: dockhand
      POSTGRES_PASSWORD: pg_pass
      POSTGRES_USER: pg_user
    image: traefik/whoami
...

The same command can be used to stop docker-config-vault stop and launch docker-config-vault up -d containers as well.

Summary

Possibilities to show the protected variables:

Using a prompt:

docker-vault view --ask-vault-password stack.env.vault

With a static (non-executable) password file:

docker-vault view --vault-password-file /srv/docker/swarm/stacks/conf/vault.pass stack.env.vault

With a dynamic (executable) password file:

Activate the password:

docker-compose-vault-key activate

Show the password:

docker-vault view --vault-password-file /usr/local/bin/docker-compose-vault-password stack.env.vault

In all cases the result is:

VAULT_TAG=v1.23
VAULT_VAR=ans-var

With the password activated, it can now be used by docker-compose-vault.

Dockhand

As the file with secrets stack.env.vault is encrypted it can be committed to Git safely, next to the compose file and other files that are needed by the containers in the stack. With the result that the encrypted file is available to Dockhand. If Dockhand can be configured with the vault password and Dockhand has the ability to decrypt stack.env.vault files then this could be a nice way to use exactly the same sensitive variables (no duplication) by Dockhand and on the command line. One thing left to do for Dockhand is to hide the vault variables (the ones prepended with VAULT_) in the GUI, as it already does for its own secret variables. This shouldn't be a too big change for Dockhand.

In the next post - part 3 it is explored how sensitive information from the stack.env.vault file could be hidden.

Decrypting the file should be possible with a (temporarily) password file, like this:

/usr/bin/ansible-vault view --vault-password-file /tmp/<random name> stack.env.vault

Note: ansible-vault is the original name, docker-vault is the aliased name that was used in the articles.

As Dockhand is already capable of storing variables in a secure way. The vault password can be stored in the same secure way.

One requirement, ansible-vault should be available on the Dockhand container image for Dockhand to be able to use it.

Please note that Hawser does not resolve nested variables (docker compose variables derived from host environment variables), like variables defined in for example the .env file and very likely in the stack.env.vault file that is used in this idea, see Dockhand issue 1307 (closed not planned) and the Dockhand manual - Global variables for stacks for more details.

Next?

What's next?

That depends, is this a method that can be implemented and supported by Dockhand? If this method to store sensitive data will be implemented by Dockhand, I think that support for different passwords on multiple levels could be looked into. The various levels could be:

• Global
• Environments
• Nodes (not available) and
• Stacks

Many different vault passwords may be difficult to maintain, but that is not a problem for Dockhand, but a challenge for the people that have to maintain the vault passwords. The only thing that Dockhand needs to do is, use the stored vault password to be able to use the secured sensitive data.

Perhaps Dockhand could be a vault password provider to docker-compose-vault on the command line, no idea if that can be done securely though or whether that makes sense.

Perhaps the vault can be unlocked during system boot/restart as well to make the sensitive variables available to during system startup, it does not seem impossible.

docker-compose-vault-key usage

docker-compose-vault-key

Response:

Usage:

    docker-compose-vault-key activate [--ttl TIME] [--replace]
    docker-compose-vault-key deactivate
    docker-compose-vault-key status
    docker-compose-vault-key password
    docker-compose-vault-key show

Options:

    --ttl TIME     Cache lifetime.
                   Supported suffixes:
                     s  seconds
                     m  minutes
                     h  hours
                   Without a suffix, the value is interpreted as seconds.

                   Maximum TTL TIME is approximately 12 hours.

    --replace      Replace an existing cached password.

Examples:

    docker-compose-vault-key activate
    docker-compose-vault-key activate --ttl 15m
    docker-compose-vault-key activate --ttl 2h
    docker-compose-vault-key activate --ttl 900
    docker-compose-vault-key activate --ttl 1h --replace

    # Display the cached password enclosed in angle brackets.
    # Useful to verify that the password command returns the expected value.
    printf '<%s>\n' "$(docker-compose-vault-key password)"

Environment:

    DOCKER_COMPOSE_VAULT_KEY_TTL
        Default: 900 seconds (15 minutes)

Part 3 - Hide sensitive variables in (GUI) applications

In part 3, the last of this series it is explored how sensitive information from the vault stack.env.vault can be hidden in applications.