Configuring NGINX as a reverse proxy to Amazon API Gateway

Use Case

If you host APIs through Amazon API Gateway, you may have noticed that API Gateway currently does not provide static IP addresses. This may cause a problem to your consumers that need a list of static IP addresses to whitelist in their firewall. To provide a list of static IP addresses to consumers, you can set up a NGINX server running on ECS behind a Network Load Balancer as a reverse proxy to Amazon API gateway.

You can learn more about NGINX on the Docker documentation or their official website.

Solution

Main AWS resources required to host this solution:

1. ECS cluster
   - to run a container task that will run the docker image of the NGINX server
2. ECR
   - to store the docker image of the NGINX server
3. Load Balancer (Network)
   - to route traffic to the ECS cluster
   - NLB provides static IP addresses, which can be whitelisted by consumers
4. IAM ECS task execution role
   - to allow the container task to run the docker image stored in ECR

nginx.conf.template (NGINX template):

user  nginx;
worker_processes  auto;
error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;
events {
  worker_connections  1024;
}

http {
  server {
    listen 8080;
    resolver 8.8.8.8 valid=10s;
    location / {
        set $backend "${PROXY_URL}";
        proxy_pass $backend;
        proxy_ssl_protocols TLSv1.2;
        proxy_ssl_server_name on;
        proxy_ssl_name ${HOST_URL};
    }
  }
  include       /etc/nginx/mime.types;
  default_type  application/octet-stream;
  log_format  main  '$remote_addr - $remote_user [$time_local] "$request" ''$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"' 'to: $upstream_addr: $request upstream_response_time $upstream_response_time msec $msec request_time $request_time';
  access_log  /var/log/nginx/access.log  main;
  sendfile        on;
  #tcp_nopush     on;
  keepalive_timeout  65;
  #gzip  on;
  include /etc/nginx/conf.d/*.conf;
}

Dockerfile:

FROM nginx:alpine
COPY nginx.conf.template /etc/nginx/conf.d/default.template
CMD sh -c "envsubst '\$PROXY_URL,\$HOST_URL' < /etc/nginx/conf.d/default.template > /etc/nginx/nginx.conf && nginx -g 'daemon off;'"

HOST_URL=www.example.com
PROXY_URL=https://www.example.com

Challenges encountered during implementation

Challenge 1
Custom domain names configured as “edge optimized” in Amazon API Gateway are served through Amazon CloudFront distribution, which supports TLS v1.1/v1.2. However, NGINX by default enables SSLv3, which is not supported by Amazon CloudFront. This causes SSL handshake errors when trying to forward requests.

Solution
To solve this problem, you need to specify which SSL protocol to use in the NGINX config file. The proxy_ssl_protocols directive in the NGINX template above forces the SSL negotiation to use TLS 1.2.

Challenge 2
Working examples of injecting environment variables in the NGINX config template from the Dockerfile were not easy to find.

Solution
Using the linux command envsubst worked, as shown in the CMD instruction of the Dockerfile. You need to pay attention to syntax for the environment variables in both the Dockerfile and NGINX template. In the NGINX template you must write your environment variables in the format of ${ENV_VAR}. You can replace mutliple environment variables by using a comma separated format in the envsubst command.

Challenge 3
By default, NGINX caches IP addresses of the backend URL upon startup of the NGINX server, which causes requests to fail once the IP addresses of the Amazon API Gateway endpoint change. This brings us back to the original problem we were trying to solve.

Solution
Using a variable in the proxy_pass directive forces re-resolution of the DNS names because NGINX treats variables differently to static configuration. The resolver directive can then be used to override the cache time of the resolved domain name. By default, NGINX sets the cache time to 5 minutes. The NGINX template above overrides this value to 10 seconds, so the domain name will be re-resolved every 10 seconds.

If you have any questions/comments regarding the work illustrated above, please feel free to reach out.