This incident initially looked like a plain DNS problem. The log kept showing:

[fetchSubscriptionAndCredits] failed: dial tcp: lookup cloudcode-pa.googleapis.com on 8.8.8.8:53: read udp ... i/o timeout

At first glance, that strongly suggests a resolver failure inside the container. But after digging further, I realized the pattern was broader: multiple Docker applications on the same VPS were showing similar “half-working” network behavior. Some requests succeeded, some timed out, and some containers could resolve domains while real connectivity still felt unstable.

That changed the direction of the investigation entirely.

Symptom

My first assumption was simple: container DNS must be broken because the error explicitly mentioned 8.8.8.8:53.

But three other clues quickly mattered more:

  • the problem was not limited to one project
  • switching to axios did not solve it
  • other Docker applications on the same host showed similar instability

That strongly suggested a host or Docker networking issue rather than an application-specific bug.

Step 1: Verify host DNS and container DNS first

I started from the host:

cat /etc/resolv.conf
resolvectl status

The host showed:

  • /etc/resolv.conf pointing to 127.0.0.53
  • an active upstream DNS of 168.63.129.16
  • a warp interface carrying DNS servers such as 1.1.1.1 and 8.8.8.8

Then I inspected a clean container:

docker run --rm alpine cat /etc/resolv.conf
docker run --rm busybox nslookup cloudcode-pa.googleapis.com

Inside the container, /etc/resolv.conf showed:

nameserver 10.0.2.3

And nslookup succeeded.

That immediately proved two things:

  1. Docker’s built-in DNS forwarder was reachable.
  2. cloudcode-pa.googleapis.com could be resolved from inside a container.

So my initial theory that “Docker DNS is completely broken” did not hold.

Step 2: Compare host, bridge, IPv4, and IPv6 behavior

Next, I tested connectivity across different network modes:

ip link show warp
ip link show docker0
docker run --rm --network bridge alpine ip route

docker run --rm --network host curlimages/curl:8.8.0 -I https://cloudcode-pa.googleapis.com
docker run --rm --network bridge curlimages/curl:8.8.0 -I https://cloudcode-pa.googleapis.com
docker run --rm --network bridge curlimages/curl:8.8.0 -4 -I https://cloudcode-pa.googleapis.com
docker run --rm --network bridge curlimages/curl:8.8.0 -6 -I https://cloudcode-pa.googleapis.com

The results were the key turning point:

  • --network host worked
  • --network bridge also worked for IPv4
  • --network bridge -4 worked
  • --network bridge -6 failed

That meant:

  • Docker’s basic IPv4 connectivity was not fundamentally broken
  • IPv6 from containers was clearly unhealthy
  • applications that tried IPv6 first could appear randomly slow or broken while waiting for fallback

This is exactly the kind of behavior that produces “sometimes it works, sometimes it times out”.

Step 3: Why MTU was still suspicious

The interface MTUs also exposed another strong risk:

warp    mtu 1420
docker0 mtu 1500

That does not automatically prove the root cause, but it is a classic WARP-related failure pattern:

  • the host egress path through warp uses a smaller MTU
  • Docker bridge still uses the default 1500
  • small packets may succeed
  • TLS handshakes, HTTP/2 traffic, or larger responses may become unstable

So my working theory was no longer “DNS only” and not “MTU only”. It was a combination:

  • the error message looked DNS-related
  • container DNS itself was mostly fine
  • IPv6 was definitely failing
  • Docker bridge MTU did not match the WARP path

Together, those factors were enough to create a half-broken network.

Step 4: Why application logs can mislead you

One confusing part was that the application log explicitly referenced 8.8.8.8:53.

That made it very tempting for me to blame Docker’s global DNS immediately. But application code may also have its own requester or resolver settings, including hard-coded DNS choices on specific request paths.

In practice that means:

  • system-level container DNS can be healthy
  • one application path can still force its own resolver behavior

That does not explain why every Docker app on the machine feels unstable, but it can absolutely distort the first round of debugging and pull attention away from the real host-level networking issue.

The most practical fix to try first

On this VPS, the most reasonable first fix for me was to align Docker MTU with WARP and pin Docker to the actually working upstream DNS:

sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json >/dev/null <<'EOF'
{
  "mtu": 1420,
  "dns": ["168.63.129.16"]
}
EOF

sudo systemctl restart docker

Then rebuild or restart the affected containers:

docker compose down
docker compose up -d --build

The goal here is simple:

  • make Docker bridge use the same MTU as the WARP egress path
  • avoid unpredictable resolver combinations inside containers
  • prefer the host’s currently working upstream DNS

If instability remains after that, the next two levers are:

  1. force critical applications to prefer IPv4
  2. temporarily switch sensitive services to network_mode: host to confirm whether bridge networking is the main factor

What this incident reinforced

1. A single error line is not the root cause

Seeing lookup ... on 8.8.8.8:53 does not automatically mean “Docker DNS is broken”.

Small validation steps matter more:

  • can a container nslookup the domain?
  • can a container curl it?
  • is host mode different from bridge mode?
  • is -4 different from -6?

Those checks narrow the problem space much faster than chasing one log line.

2. “Half-working” networks often point to MTU or IPv6 fallback issues

A pure DNS outage often looks more absolute.
But inconsistent success, random timeouts, and application-dependent symptoms are much more typical of:

  • MTU mismatches
  • broken IPv6 with slow fallback
  • routing side effects caused by tunnels layered on top of Docker bridge networking

3. WARP on the host does not mean containers inherit identical behavior

If the host works, the containers may still behave differently.
On Linux VPS setups, Docker bridge, NAT, IPv6, WARP, and MTU interactions frequently produce container-specific networking issues.

Conclusion

This issue did not end with “switch the HTTP client” or “change one DNS server”. A few small checks gradually narrowed it down to the network layer:

  • Docker DNS could resolve names
  • container IPv4 basically worked
  • container IPv6 clearly failed
  • WARP and Docker bridge had different MTUs

So the most practical first repair was to align Docker MTU with WARP and pin Docker to the host’s working DNS, then decide whether IPv4 preference or host networking is still needed afterward.

If you run into a situation where containers can resolve domains but real requests still timeout unpredictably, this troubleshooting order is often much more effective than guessing from the first error message alone.