TIL env-file means something different to docker-compose and docker

I’ve spent the last few days running head-first into a problem over and over, and I hope this might help someone else avoid this pain.

The command docker compose --env-file example.env up does not do the same thing as docker --env-file example.env run .

Docker --env-file

When you add the --env-file argument to the docker command, docker will load the variables in the env file and populate the environmental variables in the container.

This makes sense, and this is what I thought would happen with docker compose

Docker compose --env-file

When you use --env-file with docker compose, docker loads the environmental variables in the env file but, it only uses those env vars in the docker-compose.yaml file, and not the resulting containers.

Why this matters

I am writing a test that will “prove” the various components of our app work together as expected. This project is a mono-repo that has four Python projects that are deployed into three different containers.

We have tests that show that each part works as expected, but we now need to show that the parts talk to each other as we expect.

However, one of these containers is responsible for performing expensive (as in cost) and potentially long-running processes. For this test, I don’t want it to do those processes. I want it to fake those processes and return results.

As such, I enabled it to be configuredUsing the wonderful svcs package. with an environmental variable:

@svcs.fastapi.lifespan
async def lifespan(app: FastAPI, registry: svcs.Registry):
    registry.register(
        ExpensiveService,
        FakeExpensiveService if IS_TESTING else ExpensiveService
    )
    ...

I then created a test with the testcontainers package to spin up our images via docker compose, but it took far too long to run. I knew that something was wrong.

So I altered the test to test if the environmental variables were making it to the containerThey were not. :

from pathlib import Path

from testcontainers.compose import DockerCompose


def test__compose_environment__has_env_vars_from_file():
    compose_folder = Path(__file__).parents[1]
    compose = DockerCompose(
        compose_folder,
        env_file='tests/testing.env',
    )
    with compose:
        result = compose.exec_in_container(['env'])
        assert 'EXAMPLE' in result

This failed no matter how many different ways I tried to pass the environmental variables in.

How did you fix it?

After days of searching the web and asking co-workers, it finally hit us to tell the docker compose file to use the environmental variables and pass them into the containers that need them:

services:
  expensive_service:
    ...
    environment:
      - IS_TESTING='${IS_TESTING:-""}'
    ...

This will take the environmental variable from the docker compose command and pass it in to the container.