HashiCorp Developer AI Private beta now available Sign up today
Vault
Custom secrets engine

Define credentials for the secrets engine

  • 12min
  • |
  • Vault
  • Video

In the Implement secrets for the secrets engine tutorial, you defined a new secret for your secrets engine.

In this tutorial, you will create a credentials schema for your secrets engine to renew and revoke secrets. You define this at the creds/* path of the secrets engine.

Step 5 creates secrets engine credentials at /secrets-engine/creds

To do this, you will:

  1. Set up your development environment.
    You will clone the HashiCups secrets engine repository. This contains many of the interfaces and objects you need to create a secrets engine.
  2. Define the fields for the secrets engine's credentials.
    You will set the schema for the role name in the credentials path.
  3. Implement read for the secrets engine's credentials.
    You will add code to generate new tokens for the HashiCups API and store them in the credentials path.
  4. Add the credentials path to the backend.
    You will update the backend to add a new API path for credentials.
  5. Explore acceptance tests that verify the credentials path.
    You will examine acceptance tests that check that Vault can create new HashiCups API tokens on-demand.
  6. Test the credentials path.
    You will set up and run acceptance tests to create new HashiCups API tokens for a local version of HashiCups.

Prerequisites

Note

Complete the tutorial to implement the secrets for the secrets engine.

Set up your development environment

Clone the learn-vault-plugin-secrets-hashicups repository.

$ git clone https://github.com/hashicorp-education/learn-vault-plugin-secrets-hashicups

Change into the repository directory.

$ cd vault-plugin-secrets-hashicups

Note

If you are stuck in this tutorial, refer to the vault-plugin-secrets-hashicups/solution directory.

Define the fields for the secrets engine's credentials

Open path_credentials.go. The file contains all of the objects and methods related to setting up the creds path for the secrets engine.

Note

Replace the methods and structs in the scaffold with the embedded code examples.

The pathCredentials method returns an object with empty fields. These attributes defined framework.Path extend the Vault API for the secrets engine's creds path.

Replace the pathCredentials method in the scaffold with a schema defining the name of the role for Fields.

path_credentials.go
1 2 3 4 5 6 7 8 9 101112131415func pathCredentials(b *hashiCupsBackend) *framework.Path {
    return &framework.Path{
        Pattern: "creds/" + framework.GenericNameRegex("name"),
        Fields: map[string]*framework.FieldSchema{
            "name": {
                Type:        framework.TypeLowerCaseString,
                Description: "Name of the role",
                Required:    true,
            },
        },
        Callbacks:       map[logical.Operation]framework.OperationFunc{},
        HelpSynopsis:    pathCredentialsHelpSyn,
        HelpDescription: pathCredentialsHelpDesc,
    }
}

Note

The /creds path for your secrets engine must always set a name field with the type of framework.TypeLowerCaseString.

The path includes attributes for help text, such as HelpSynopsis and HelpDescription. The help text describes the attributes someone needs to configure the secrets engine.

path_credentials.go
1 2 3 4 5 6 7 8 9 10111213141516171819202122232425func pathCredentials(b *hashiCupsBackend) *framework.Path {
    return &framework.Path{
        Pattern: "creds/" + framework.GenericNameRegex("name"),
        Fields: map[string]*framework.FieldSchema{
            "name": {
                Type:        framework.TypeLowerCaseString,
                Description: "Name of the role",
                Required:    true,
            },
        },
        Callbacks:       map[logical.Operation]framework.OperationFunc{},
        HelpSynopsis:    pathCredentialsHelpSyn,
        HelpDescription: pathCredentialsHelpDesc,
    }
}

const pathCredentialsHelpSyn = `
Generate a HashiCups API token from a specific Vault role.
`

const pathCredentialsHelpDesc = `
This path generates a HashiCups API user tokens
based on a particular role. A role can only represent a user token,
since HashiCups doesn't have other types of tokens.
`

Implement read for the secrets engine's credentials

Open path_credentials.go.

Note

Replace the methods and structs in the scaffold with the embedded code examples.

The Operations field in the pathCredentials method starts empty. You need to add methods to the Operations field to tell Vault how to handle creating, reading, updating, and deleting information at the creds path.

Add a method named createToken that extends the hashiCupsBackend. It creates the token based on the username in the role entry passed to the creds endpoint.

path_credentials.go
func (b *hashiCupsBackend) createToken(ctx context.Context, s logical.Storage, roleEntry *hashiCupsRoleEntry) (*hashiCupsToken, error) {
    client, err := b.getClient(ctx, s)
    if err != nil {
        return nil, err
    }

    var token *hashiCupsToken

    token, err = createToken(ctx, client, roleEntry.Username)
    if err != nil {
        return nil, fmt.Errorf("error creating HashiCups token: %w", err)
    }

    if token == nil {
        return nil, errors.New("error creating HashiCups token")
    }

    return token, nil
}

You need to parse the hashiCupsToken and map the fields to the response from the /creds endpoint.

Add a new method named createUserCreds in path_credentials.go. The method creates the HashiCups token and maps it to a response for the secrets engine backend to return. It also sets the time to live (TTL) and maximum TTL for the secret.

path_credentials.go
1 2 3 4 5 6 7 8 9 1011121314151617181920212223242526func (b *hashiCupsBackend) createUserCreds(ctx context.Context, req *logical.Request, role *hashiCupsRoleEntry) (*logical.Response, error) {
    token, err := b.createToken(ctx, req.Storage, role)
    if err != nil {
        return nil, err
    }

    resp := b.Secret(hashiCupsTokenType).Response(map[string]interface{}{
        "token":    token.Token,
        "token_id": token.TokenID,
        "user_id":  token.UserID,
        "username": token.Username,
    }, map[string]interface{}{
        "token": token.Token,
        "role":  role.Name,
    })

    if role.TTL > 0 {
        resp.Secret.TTL = role.TTL
    }

    if role.MaxTTL > 0 {
        resp.Secret.MaxTTL = role.MaxTTL
    }

    return resp, nil
}

Add a method named pathCredentialsRead. The method verifies the role exists for the secrets engine and creates a new HashiCups token based on the role entry.

path_credentials.go
1 2 3 4 5 6 7 8 9 1011121314func (b *hashiCupsBackend) pathCredentialsRead(ctx context.Context, req *logical.Request, d *framework.FieldData) (*logical.Response, error) {
    roleName := d.Get("name").(string)

    roleEntry, err := b.getRole(ctx, req.Storage, roleName)
    if err != nil {
        return nil, fmt.Errorf("error retrieving role: %w", err)
    }

    if roleEntry == nil {
        return nil, errors.New("error retrieving role: role is nil")
    }

    return b.createUserCreds(ctx, req, roleEntry)
}

Note

You want to output your secret in the response for your /creds endpoint. Otherwise, you can't use the dynamically generated secret!

Replace pathCredentials with an update to the list of Callbacks for operations on the /creds endpoint. Add logical.ReadOperation and logical.UpdateOperation and set the callback method to b.pathCredentialsRead. Both methods use the same callback to generate a new token each time you call the creds endpoint.

path_credentials.go
1 2 3 4 5 6 7 8 9 101112131415161718func pathCredentials(b *hashiCupsBackend) *framework.Path {
    return &framework.Path{
        Pattern: "creds/" + framework.GenericNameRegex("name"),
        Fields: map[string]*framework.FieldSchema{
            "name": {
                Type:        framework.TypeLowerCaseString,
                Description: "Name of the role",
                Required:    true,
            },
        },
        Callbacks: map[logical.Operation]framework.OperationFunc{
            logical.ReadOperation:   b.pathCredentialsRead,
            logical.UpdateOperation: b.pathCredentialsRead,
        },
        HelpSynopsis:    pathCredentialsHelpSyn,
        HelpDescription: pathCredentialsHelpDesc,
    }
}

Note

As a best practice for secrets engines, /creds should generate a new token and print out when you read from the endpoint. You do not need to implement logic to verify the content of the secret itself. Instead, you can create the secret each time the code references the pathCredentialsRead method.

Add the credentials path to the backend

For each API path you extend on the secrets engine, you must add it to the secrets engine backend.

Note

Replace the methods and structs in the scaffold with the embedded code examples.

Open backend.go and replace backend to add pathCredentials to the list of valid paths for the backend.

backend.go
1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627func backend() *hashiCupsBackend {
    var b = hashiCupsBackend{}

    b.Backend = &framework.Backend{
        Help: strings.TrimSpace(backendHelp),
        PathsSpecial: &logical.Paths{
            LocalStorage: []string{},
            SealWrapStorage: []string{
                "config",
                "role/*",
            },
        },
        Paths: framework.PathAppend(
            pathRole(&b),
            []*framework.Path{
                pathConfig(&b),
                pathCredentials(&b),
            },
        ),
        Secrets: []*framework.Secret{
            b.hashiCupsToken(),
        },
        BackendType: logical.TypeLogical,
        Invalidate:  b.invalidate,
    }
    return &b
}

Note

If you do not add your path to the backend object, you will get an error of unsupported path in your tests and compiled plugin.

Explore acceptance tests that verify the credentials path

The Vault Plugin SDK includes a testing framework for unit and acceptance tests.

  • Unit tests: Use mocks to verify the functionality of the secrets engine
  • Acceptance tests: Require a Vault instance, an active target API endpoint, and binary for the secrets engine.

You can write a set of acceptance tests to verify the secrets engine creates and updates the new HashiCups token.

Note

If you want to write unit tests for the /creds endpoint, you must mock the upstream target API. In this example, you write acceptance tests to make requests to the API.

Open backend_test.go. The file defines parameters related to managing and setting input values to tests.

Examine the constants for the names of environment variables. The tests retrieve environment variables for a few testing parameters, including:

  • VAULT_ACC: Set to 1 to run acceptance tests. Acceptance tests have external dependencies, so you may not want to run them all the time.
  • TEST_HASHICUPS_USERNAME: Username for HashiCups API. Make a request to /signup at the HashiCups API.
  • TEST_HASHICUPS_PASSWORD: Password for HashiCups API. Make a request to /signup at the HashiCups API.
  • TEST_HASHICUPS_URL: Testing endpoint for HashiCups API. Make a request to /signup at the HashiCups API.
backend_test.go
const (
    envVarRunAccTests       = "VAULT_ACC"
    envVarHashiCupsUsername = "TEST_HASHICUPS_USERNAME"
    envVarHashiCupsPassword = "TEST_HASHICUPS_PASSWORD"
    envVarHashiCupsURL      = "TEST_HASHICUPS_URL"
)

Open path_credentials_test.go.

Examine the method newAcceptanceTestEnv. It creates an acceptance testing environment that retrieves username and password information from environment variables to access HashiCups.

path_credentials_test.go
func newAcceptanceTestEnv() (*testEnv, error) {
    ctx := context.Background()

    maxLease, _ := time.ParseDuration("60s")
    defaultLease, _ := time.ParseDuration("30s")
    conf := &logical.BackendConfig{
        System: &logical.StaticSystemView{
            DefaultLeaseTTLVal: defaultLease,
            MaxLeaseTTLVal:     maxLease,
        },
        Logger: logging.NewVaultLogger(log.Debug),
    }
    b, err := Factory(ctx, conf)
    if err != nil {
        return nil, err
    }
    return &testEnv{
        Username: os.Getenv(envVarHashiCupsUsername),
        Password: os.Getenv(envVarHashiCupsPassword),
        URL:      os.Getenv(envVarHashiCupsURL),
        Backend:  b,
        Context:  ctx,
        Storage:  &logical.InmemStorage{},
    }, nil
}

Note

You can adapt newAcceptanceTestEnv for your own secrets engine. Setting your environment variables to retrieve test configuration for your secrets engine provides some flexibility in accessing your target API from pipelines or local development environments.

Examine the testing method, TestAcceptanceUserToken. The test checks if you want the test suite to include acceptance tests, initializes a new acceptance testing environment, and runs a series of commands to test that you generated a new HashiCups API token.

path_credentials_test.go
func TestAcceptanceUserToken(t *testing.T) {
    if !runAcceptanceTests {
        t.SkipNow()
    }

    acceptanceTestEnv, err := newAcceptanceTestEnv()
    if err != nil {
        t.Fatal(err)
    }

    t.Run("add config", acceptanceTestEnv.AddConfig)
    t.Run("add user token role", acceptanceTestEnv.AddUserTokenRole)
    t.Run("read user token cred", acceptanceTestEnv.ReadUserToken)
    t.Run("read user token cred", acceptanceTestEnv.ReadUserToken)
    t.Run("cleanup user tokens", acceptanceTestEnv.CleanupUserTokens)
}

TestAcceptanceUserToken runs the tests sequentially and uses a secrets engine backend. You need to set up the secrets engine configuration and create a role before you can generate new credentials. You should write your test sequence as follows:

  1. Add a configuration for your secrets engine to /config.
  2. Add a role for your secrets engine to /role/test.
  3. Read credentials from your secrets engine at /creds/test.
  4. Read another set of credentials from your secrets engine at /creds/test. They should not match the first set of credentials.
  5. Clean up the tokens by accessing the target API with its client and invalidating the credentials.

You will find the helper methods for the testing sequence in backend_test.go.

Examine ReadUserToken as an example. It calls the secrets engine backend with a ReadOperation at the path, creds/test-user-token. It checks that the response includes the JSON Web Token (JWT) from HashiCups. The method stores the tokens into a list so the test suite cleans them up after it finishes.

backend_test.go
1 2 3 4 5 6 7 8 9 101112131415161718192021222324func (e *testEnv) ReadUserToken(t *testing.T) {
    req := &logical.Request{
        Operation: logical.ReadOperation,
        Path:      "creds/test-user-token",
        Storage:   e.Storage,
    }
    resp, err := e.Backend.HandleRequest(e.Context, req)
    require.Nil(t, err)
    require.NotNil(t, resp)

    if t, ok := resp.Data["token"]; ok {
        e.Tokens = append(e.Tokens, t.(string))
    }
    require.NotEmpty(t, resp.Data["token"])

    if e.SecretToken != "" {
        require.NotEqual(t, e.SecretToken, resp.Data["token"])
    }

    require.NotNil(t, resp.Secret)
    if t, ok := resp.Secret.InternalData["token"]; ok {
        e.SecretToken = t.(string)
    }
}

Test the credentials path

Open a terminal and make sure your working directory uses the plugins/vault-plugin-secrets-hashicups.

$ pwd

${HOME}/hashicorp/vault-guides/plugins/vault-plugin-secrets-hashicups

The secrets engine directory includes files to create a local instance of HashiCups for acceptance tests.

Note

You will need to create a local instance of HashiCups to run the tests in this tutorial!

Use the terminal to create the database and API for HashiCups.

$ cd docker_compose && docker-compose up -d

Creating network "docker_compose_default" with the default driver
Creating docker_compose_db_1 ... done
Creating docker_compose_api_1 ... done

Verify with the terminal that Docker started HashiCups. Send a request to its health check endpoint.

$ cd .. && curl localhost:19090/health

ok

Set the username you will use to test the HashiCups API as the TEST_HASHICUPS_USERNAME environment variable.

$ export TEST_HASHICUPS_USERNAME='vault-plugin-testing'

Set the password you will use to test the HashiCups API as the TEST_HASHICUPS_PASSWORD environment variable.

$ export TEST_HASHICUPS_PASSWORD='Testing!123'

Set the URL you will use to test the HashiCups API as the TEST_HASHICUPS_URL environment variable. You should use localhost:19090 to access the HashiCups Docker containers.

$ export TEST_HASHICUPS_URL='http://localhost:19090'

Sign up for the HashiCups API with a username and password by calling the /signup API endpoint.

$ curl -X POST -H 'Content-Type:application/json' \
  ${TEST_HASHICUPS_URL}/signup \
  -d '{"username": "'${TEST_HASHICUPS_USERNAME}'", "password": "'${TEST_HASHICUPS_PASSWORD}'"}'

Output:

{"UserID":1,"Username":"vault-plugin-testing","token":"${TEST_HASHICUPS_JSON_WEB_TOKEN}"}

Run the acceptance tests for the credentials path in your terminal. You need to set the environment variable VAULT_ACC=1 to run them. The tests should pass.

$ VAULT_ACC=1 go test -v -run TestAcceptanceUserToken

Output:

=== RUN   TestAcceptanceUserToken
=== RUN   TestAcceptanceUserToken/add_config
=== RUN   TestAcceptanceUserToken/add_user_token_role
=== RUN   TestAcceptanceUserToken/read_user_token_cred
=== RUN   TestAcceptanceUserToken/read_user_token_cred#01
=== RUN   TestAcceptanceUserToken/cleanup_user_tokens
--- PASS: TestAcceptanceUserToken (0.03s)
    --- PASS: TestAcceptanceUserToken/add_config (0.00s)
    --- PASS: TestAcceptanceUserToken/add_user_token_role (0.00s)
    --- PASS: TestAcceptanceUserToken/read_user_token_cred (0.02s)
    --- PASS: TestAcceptanceUserToken/read_user_token_cred#01 (0.01s)
    --- PASS: TestAcceptanceUserToken/cleanup_user_tokens (0.01s)
PASS
ok      github.com/hashicorp/vault-guides/plugins/vault-plugin-secrets-hashicups    1.021s

Clean up

Use the terminal to remove the database and API for HashiCups.

$ cd docker_compose && docker-compose down

Next steps

Congratulations! You defined the /creds endpoint for your secrets engine.

If you are stuck in this tutorial, refer to the plugins/vault-plugin-secrets-hashicups/solution directory.

Previous

Implement secrets engine secrets

 Next

Test and build secrets engine