nginx add_header silently drops every inherited header

This one cost us an hour and it is in the nginx documentation, phrased in a way that is easy to read past.

add_header directives are inherited from the previous configuration level only if there are no add_header directives defined on the current level. Not merged. Replaced.

How it bites

Here is a configuration that looks obviously correct and is obviously wrong:

server {
    add_header Strict-Transport-Security "max-age=31536000" always;
    add_header X-Content-Type-Options   "nosniff" always;
    add_header X-Frame-Options          "DENY" always;

    location / {
        try_files $uri $uri/index.html @notfound;
    }

    location /assets/ {
        add_header Cache-Control "public, max-age=31536000, immutable";
    }
}

Requests to / get three security headers. Requests to /assets/ get one Cache-Control header and nothing else. The security headers are gone, because that location declared an add_header of its own.

Test it by curling the home page and it looks perfect. Every asset on the site is unprotected.

The pattern that works

Put the headers in a snippet and include it at every level that sets a header of its own:

# /etc/nginx/snippets/security.conf
add_header Strict-Transport-Security "max-age=31536000" always;
add_header X-Content-Type-Options   "nosniff" always;
add_header X-Frame-Options          "DENY" always;

# in the site config
server {
    include snippets/security.conf;

    location /assets/ {
        include snippets/security.conf;   # or the above is discarded
        add_header Cache-Control "public, max-age=31536000, immutable";
    }
}

Two things that go with it

Use always. Without it, add_header only applies to a specific list of response codes, so your 404 page ships bare. That also means a 403 from a directory listing carries no headers at all, which is a confusing thing to debug if you are checking a directory rather than a file.

And know where try_files resolves. If you serve prerendered pages with try_files $uri $uri/index.html, the found file is processed in the context of the location that ran it. A location matching a .html pattern is never entered, so a Cache-Control rule written there does precisely nothing.

The test that catches a misplaced include is not the home page. It is an asset: if it comes back without a security header, or with the wrong Cache-Control, an include landed in the wrong block.