Docker secrets met Ansible vault

Introduction

Explore a mechanism that makes it possible to store sensitive variables, like passwords, api-keys and the like in a vault. As the vault is nothing else then an encrypted file it be can committed to Git. This makes it possible that the same sensitive variables could be used by Dockhand and docker compose on the command line. One caveat Dockhand must be adapted to support this.

The nice thing is, that this is accomplished using known and well supported commands as Ansible Vault and Keyctl in combination with some scripts and aliases for convenience.

Ansible-vault

Check that ansible-vault is available with either:

• which ansible-vault
• command -v ansible-vault

If not available, install it.

Install Ansible

Install Ansible to make the script ansible-vault available. Use the package installer for your Linux distribution:

sudo apt install ansible-core

Response:

Installing:
  ansible

Installing dependencies:
  ansible-core       python3-argcomplete  python3-dnspython  python3-h2          python3-httplib2     python3-jmespath   python3-libcloud
  python3-pyparsing  python3-resolvelib   python3-winrm      ieee-data           python3-click        python3-gssapi     python3-hpack
  python3-httpx      python3-kerberos     python3-netaddr    python3-pyspnego    python3-selinux      python3-xmltodict  python3-anyio
  python3-decorator  python3-h11          python3-httpcore   python3-hyperframe  python3-legacycrypt  python3-passlib    python3-requests-ntlm
  python3-sniffio

Suggested packages:
  cowsay  sshpass  python3-trio  python3-aioquic  ipython3  python-netaddr-docs  python-pyparsing-doc

Summary:
  Upgrading: 0, Installing: 30, Removing: 0, Not Upgrading: 13
  Download size: 25,5 MB
  Space needed: 257 MB / 225 GB available

Check again that ansible-vault is available, using the commands provided earlier.

Note: This can be skipped, if the previous install command worked. As I later discovered that only the package ansible-core is needed to provide the executable ansible-vault: It is better to install only ansible-core with its dependencies, as that saves about 200MB of 250MB disk space. To accomplish that remove the package ansible.

LANGUAGE=C apt remove ansible
The following packages were automatically installed and are no longer required:
  ansible-core           python3-argcomplete  python3-dnspython   python3-h2        python3-httplib2   python3-jmespath   python3-netaddr
  python3-requests-ntlm  python3-sniffio      ieee-data           python3-click     python3-gssapi     python3-hpack      python3-httpx
  python3-kerberos       python3-pyparsing    python3-resolvelib  python3-winrm     python3-anyio      python3-decorator  python3-h11
  python3-httpcore       python3-hyperframe   python3-libcloud    python3-pyspnego  python3-selinux    python3-xmltodict
Use 'apt autoremove' to remove them.

REMOVING:
  ansible

Summary:
  Upgrading: 0, Installing: 0, Removing: 1, Not Upgrading: 9
  Freed space: 210 MB

Continue? [J/n]
(Reading database ... 171519 files and directories currently installed.)
Removing ansible (12.0.0+dfsg-0+deb13u1) ...
Processing triggers for man-db (2.13.1-1) ...

Only the package ansible, that bundles the official Ansible collections is removed.

Check that ansible-vault remains available:

command -v ansible-vault

/usr/bin/ansible-vault

Vault password file

Define the following variable in $HOME/.bash_aliases:

export ANSIBLE_VAULT_PASSWORD_FILE=$DOCKER_STACKS_DIR/conf/vault.pass

Define a location for the file that suites you, as long as it is never, never committed to Git. So, better to use a location outside Git repositories.

After saving the .bash_aliases file, read it in again by either:

• log out and in again
• log in from another terminal/console
• read the file in again in the same session by executing: . $HOME/.bash_aliases

Check that the variable is defined: echo $ANSIBLE_VAULT_PASSWORD_FILE

If it is continue, otherwise solve the issue and repeat.

Prepare the file to hold the password:

touch $ANSIBLE_VAULT_PASSWORD_FILE
chmod 600 $ANSIBLE_VAULT_PASSWORD_FILE
ls -l $ANSIBLE_VAULT_PASSWORD_FILE

Write your password into the $ANSIBLE_VAULT_PASSWORD_FILE.

vi $ANSIBLE_VAULT_PASSWORD_FILE
nano $ANSIBLE_VAULT_PASSWORD_FILE

Check the contents:

cat $ANSIBLE_VAULT_PASSWORD_FILE

Response:

Your password should be shown.

Alias ansible-vault to docker-vault

Add the following alias to the .bash_alias file:

alias docker-vault="ansible-vault"

Or in case ansible-vault/docker-vault complains that the locale setting is not ok create the following alias:

alias docker-vault="LC_ALL=C.UTF-8 ansible-vault"

Activate the setting by logging out and in again, as described earlier.

Check that the alias is active with:

alias docker-vault

Encrypt the stack.env.vault file

Go to a (test) directory with a compose file.

Create a stack.env.vault file, put a variable into the file and encrypt it.

A good practice is to prepend variables in the vault with VAULT so it is clear in the compose file where the variable is defined.

Example stack.env.vault:

VAULT_VAR=test

Encrypt command:

docker-vault encrypt --ask-vault-password stack.env.vault

Response:

New Vault password:
Confirm New Vault password:
Encryption successful

As the environment variable ANSIBLE_VAULT_PASSWORD_FILE is already defined, the file can be encrypted with the following command as well (ensure that you use a unencrypted file!):

docker-vault encrypt stack.env.vault

Response:

Encryption successful

View the encrypted file:

cat stack.env.vault

Response:

$ANSIBLE_VAULT;1.1;AES256
36353935653262373738653736616436633839613864616266363065386539303466363530633163
6635333535333162353733633164633839313662626661340a336138333732343266393530663037
35313631343934306165643235636531633436623239356166393838333437353435383236366136
6430383731366632330a616138663533633735303264346335663563356333656233333437326633
34386439653661343933623765653435303130383266373539316234316336616637

View the contents:

docker-vault view stack.env.vault

Response:

VAULT_VAR=test

docker-compose-vault

Get the script docker-compose-vault and save it to /usr/local/bin or another favorite bin directory.

The script looks for the file stack.env.vault if it exists it makes the variables defined in the file available to the docker compose command.

Test with a compose file

Excerpt from the test compose file:

...
  label:
    my.variable.test1: ${VAULT_VAR:?VAULT_VAR is required }
    my.variable.test2: ${VAULT_VAR:?VAULT_VAR must not be empty }
...

Execute:

docker-compose-vault config

Or use the alias dcc.

The response should contain:

...
  label:
    my.variable.test1: test
    my.variable.test2: test
...

If the variable(s) is not defined (because the stack.env.vault file is not decrypted), the response will be:

error while interpolating `services.whoami.labels.my.variable.test1`: required variable `VAULT_VAR` is missing a value: `VAULT_VAR` is required

Summary

There are multiple ways to decrypt the encrypted stack.env.vault file:

Using a prompt:

docker-vault view --ask-vault-password stack.env.vault

The password is asked for, if provided correctly the secret variables are revealed:

Vault password:
VAULT_TAG=v1.23
VAULT_VAR=ans-var

With a file holding the password:

docker-vault view --vault-password-file /srv/docker/swarm/stacks/conf/vault.pass stack.env.vault

Response:

VAULT_TAG=v1.23
VAULT_VAR=ans-var

In the next part part 2 it is explained how to use a password that is stored in memory for time, instead of using a file.

Attachments

• docker-compose-vault