Digity Republic

Tech News, Reviews, Deals, and How-To's

How to Use Varnish Cache Server on Ubuntu 20.04 to Speed Up Static Web Pages

Varnish Cache Server

image of Varnish Cache Server

Introduction

Varnish Cache Server: Varnish is a flexible reverse HTTP proxy that stores results from backend servers in memory for rapid delivery upon subsequent requests. To decide whether to cache the answers to a specific request, it consults the HTTP headers. Because answers containing cookies are regarded as client-specific requests, it does not cache them by default; however, you may modify this option in the configuration file.

Varnish has additional uses outside serving as a cache server.

  • firewall for web applications
  • DoS assault defense
  • Balancer of loads
  • Immediately address unstable backends
  • Web router

The HTTP cache can be saved in one of three places:

  • Browser: Users’ browsers save this cache. Instead of making requests, it may use to offer material that is user-specific.
  • Proxy: Between users and servers, a proxy server acts as an intermediary. It may use to cache replies that will request by several users and often employ ISPs.
  • Reverse Proxy: Rather than directing requests to back-end servers, this type of proxy makes the administrator of the website and can use to deliver information from the network edge. You will build this sort of caching during this lesson.

You will configure Varnish as a caching reverse proxy server in this tutorial. The setup will then puts to the test using work and Varnish against a non-caching configuration.

Prerequisites

You’ll need these things to finish this tutorial:

one server running Ubuntu 20.04 with at least 2 GB of RAM
According to this first server setup tutorial for Ubuntu 20.04, a non-root user with sudo capabilities.

Installing Varnish and Apache in Step 1: Varnish Cache Server

Installing Apache and Varnish is the first step. These instructions will install Apache after updating apt-get:

sudo apt-get update
sudo apt-get install apache2 -y

You’ll see results demonstrating the installation of Apache.

Use the following command to install Varnish once Apache has been set up:

sudo apt-get install varnish -y

You’ll see results demonstrating the installation of Varnish.

Next, confirm that both packages successfully install. Use the following command to first determine Apache’s status:

sudo systemctl status apache2

The result will resemble this:

Output
root@ubuntu-s-1vcpu-2gb-fra1-01:~# sudo systemctl status apache2
● apache2.service - The Apache HTTP Server
     Loaded: loaded (/lib/systemd/system/apache2.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2021-08-04 18:58:39 UTC; 4min 10s ago
       Docs: https://httpd.apache.org/docs/2.4/
   Main PID: 2279 (apache2)
      Tasks: 55 (limit: 2344)
     Memory: 5.0M
     CGroup: /system.slice/apache2.service
             ├─2279 /usr/sbin/apache2 -k start
             ├─2281 /usr/sbin/apache2 -k start
             └─2282 /usr/sbin/apache2 -k start

Aug 04 18:58:39 ubuntu-s-1vcpu-2gb-fra1-01 systemd[1]: Starting The Apache HTTP Server...
Aug 04 18:58:39 ubuntu-s-1vcpu-2gb-fra1-01 apachectl[2278]: AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' di>
Aug 04 18:58:39 ubuntu-s-1vcpu-2gb-fra1-01 systemd[1]: Started The Apache HTTP Server.

To end the status command, use the Q key.

Next, use this command to verify Varnish’s status:

sudo systemctl status varnish

The output will look similar to this:

Output
root@ubuntu-s-1vcpu-2gb-fra1-01:~# sudo systemctl status apache2
● apache2.service - The Apache HTTP Server
     Loaded: loaded (/lib/systemd/system/apache2.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2021-08-04 18:58:39 UTC; 4min 10s ago
       Docs: https://httpd.apache.org/docs/2.4/
   Main PID: 2279 (apache2)
      Tasks: 55 (limit: 2344)
     Memory: 5.0M
     CGroup: /system.slice/apache2.service
             ├─2279 /usr/sbin/apache2 -k start
             ├─2281 /usr/sbin/apache2 -k start
             └─2282 /usr/sbin/apache2 -k start

Aug 04 18:58:39 ubuntu-s-1vcpu-2gb-fra1-01 systemd[1]: Starting The Apache HTTP Server...
Aug 04 18:58:39 ubuntu-s-1vcpu-2gb-fra1-01 apachectl[2278]: AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' di>
Aug 04 18:58:39 ubuntu-s-1vcpu-2gb-fra1-01 systemd[1]: Started The Apache HTTP Server.

To end the status command, use the Q key.

Next, use this command to verify Varnish’s status:

sudo systemctl status varnish

The result will resemble this:

Output
root@ubuntu-s-1vcpu-2gb-fra1-01:~# sudo systemctl status varnish
● varnish.service - Varnish HTTP accelerator
     Loaded: loaded (/lib/systemd/system/varnish.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2021-08-04 18:59:09 UTC; 4min 41s ago
       Docs: https://www.varnish-cache.org/docs/6.1/
             man:varnishd
   Main PID: 3423 (varnishd)
      Tasks: 217 (limit: 2344)
     Memory: 10.7M
     CGroup: /system.slice/varnish.service
             ├─3423 /usr/sbin/varnishd -j unix,user=vcache -F -a :6081 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m
             └─3447 /usr/sbin/varnishd -j unix,user=vcache -F -a :6081 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m

Aug 04 18:59:09 ubuntu-s-1vcpu-2gb-fra1-01 systemd[1]: Started Varnish HTTP accelerator.
Aug 04 18:59:10 ubuntu-s-1vcpu-2gb-fra1-01 varnishd[3423]: Debug: Version: varnish-6.2.1 revision 9f8588e4ab785244e06c3446fe09bf9db5dd8753
Aug 04 18:59:10 ubuntu-s-1vcpu-2gb-fra1-01 varnishd[3423]: Version: varnish-6.2.1 revision 9f8588e4ab785244e06c3446fe09bf9db5dd8753
Aug 04 18:59:10 ubuntu-s-1vcpu-2gb-fra1-01 varnishd[3423]: Debug: Platform: Linux,5.4.0-73-generic,x86_64,-junix,-smalloc,-sdefault,-hcritbit
Aug 04 18:59:10 ubuntu-s-1vcpu-2gb-fra1-01 varnishd[3423]: Platform: Linux,5.4.0-73-generic,x86_64,-junix,-smalloc,-sdefault,-hcritbit
Aug 04 18:59:10 ubuntu-s-1vcpu-2gb-fra1-01 varnishd[3423]: Debug: Child (3447) Started
Aug 04 18:59:10 ubuntu-s-1vcpu-2gb-fra1-01 varnishd[3423]: Child (3447) Started
Aug 04 18:59:10 ubuntu-s-1vcpu-2gb-fra1-01 varnishd[3423]: Info: Child (3447) said Child starts
Aug 04 18:59:10 ubuntu-s-1vcpu-2gb-fra1-01 varnishd[3423]: Child (3447) said Child starts

If you do not see both services active, give them a few minutes to fully load before continuing to run both of them.

You will now offer Apache2 Varnish something to serve, in this example Apache’s static web page, after Varnish is deploying.

Setting Up Varnish To Serve Apache’s Static Web Page in Step 2

Varnish was installed in the preceding stage, and the following step is to configure it. By default, Varnish connects to a local web server on port 8080 and listens on port 6081. To serve the Apache static site from the Apache server, you must alter it.

Varnish’s listening port must first be changed to 8080. Since Varnish and Apache are operating on the same server, you will use port 8080 for Varnish and port 80 for Apache. Normally, you would want the listening port to be 80.

Create the new directory using the mkdir command:

sudo mkdir /etc/systemd/system/varnish.service.d

Make a new file named customexec.conf in your preferred text editor:

Add the following text to customexec.conf:

/etc/systemd/system/varnish.service.d/customexec.conf file
[Service]
ExecStart=
ExecStart=/usr/sbin/varnishd -j unix,user=vcache -F -a :8080 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m

You modify the Varnish configuration’s Service section in this file. You assign a new value to the ExecStart option after removing the previous one.

When running Varnish with the following parameters, the new value indicates the binary file to use:

-j: Defines the jailing method to employ. Varnish prisons are using instead of different platform-specific techniques to decrease the permissions for the varnish process. To restrict the rights, in this case, you’re utilizing the Unix method and the user cache. On Ubuntu systems, the varnish is set to this by default.

F: The server should run in the foreground because the system anticipates that the primary process will continue to run so that it can track it, rather than forking a new process and exiting.

The IP address and port for accepting client connections specifying using the -a parameter. Since the IP in this instance is empty, the server will accept any IP. 8080 is the port’s set value.

-T: This flag designates localhost and port 6082 as the IP address and administration interface, respectively.

The default VCL file for the Varnish setup is specified by the -f parameter. Later on in this tutorial, you will modify this file to set up Varnish to connect to the Apache server.

The -S option designates a shared secret file that will be used to provide users access to the administration interface. On Ubuntu, Varnish’s secret setting defaults to /etc/varnish/secret. The secret file will not utilize in this lesson.

The -s option specifies the location and method of object storage. For Vanish, the default setting is malloc,256m. It refers to utilizing the malloc system function and a maximum size of 256 gigabytes to store different Varnish objects in memory. Other options include default, which uses umem in the absence of malloc, and file, which saves objects to a file on the hard drive.

Save the customexec.conf file, then exit. To reload the system services file from the disk, then run the following command:

sudo systemctl daemon-reload

Varnish must then be restarted for modifications to take effect.

sudo systemctl restart varnish

These latter two instructions have no output, so you won’t see anything. Use the netstat command to see all listening TCP sockets on the server to confirm that Varnish is now listening on port 8080.

sudo netstat -ltnp | grep 8080

The output will seem as follows:

Output
tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN      18689/varnishd
tcp6       0      0 :::8080                 :::*                    LISTEN      18689/varnishd

You must change the default configuration file at /etc/varnish/default.vcl now that Varnish is up and running and listening on port 8080:

sudo nano /etc/varnish/default.vcl

Change the. port set to 80 by navigating to the backend default block and doing so as follows:

default.vcl file

# Default backend definition. Set this to point to your content server.
backend default {
    .host = "127.0.0.1";
    .port = "80";
}

Save and exit the default.vcl file, then use the following command to restart Varnish:

sudo systemctl restart varnish

In a perfect world, there will no output. You may view the Apache static site by going to http://your server ip:8080 in your browser. Varnish is using it to open the static site.

As of right now, Apache is listening on port 80 while Varnish is listening on port 8080 on the same Droplet. The wrk program will then use to compare the response times of each server.

Step 3: Varnish Testing applying for work.

A contemporary HTTP benchmarking tool is wrk. It may use to load test web servers with several requests per second and is developing in C. Using wrk, you will run tests against Varnish and Apache in this stage, then compare the outcomes.

Installing work first requires compiling it from source code. Install some in the beginning.

sudo apt-get install build-essential libssl-dev git unzip -y

After that, clone the wrk directory’s git repository into it:

git clone https://github.com/wg/wrk.git wrk

Put that new directory in:

cd wrk

Using the make command, create the wrk executable:

make

In order to access wrk from anywhere in your directory tree, copy it to the /usr/local/bin directory:

sudo cp wrk /usr/local/bin

Now that wrk is set up, execute the following command to test Apache’s responsiveness:

wrk -t2 -c1000 -d30s --latency http://server_ip/

The following parameters are used with this command:

  • -t2 stands for “run two threads.”
  • -c1000: Maintain 1000 open HTTP connections.
  • Run the test for 30 seconds with the -d option.
  • —latency: Display latency information.

When the test finishes, wait 30 seconds, and the output will look something like this:

output
Running 30s test @ http://68.183.115.151/
  2 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    44.45ms  104.50ms   1.74s    91.20%
    Req/Sec     8.29k     1.07k   12.40k    71.00%
  Latency Distribution
     50%   11.59ms
     75%   22.73ms
     90%  116.16ms
     99%  494.90ms
  494677 requests in 30.04s, 5.15GB read
  Socket errors: connect 0, read 8369, write 0, timeout 69
Requests/sec:  16465.85
Transfer/sec:    175.45MB

In this test, there were 494,677 total requests, 8,369 read errors, and 69 timeout errors. The average latency was 44.45 milliseconds. The precise figures will vary depending on your installation.

Now use the following command to repeat the test on the Varnish server:

wrk -t2 -c1000 -d30s --latency http://server_ip:8080/

When the test finishes, wait 30 seconds, and the output will look something like this:

output
Running 30s test @ http://68.183.115.151:8080/
  2 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    14.41ms   13.70ms 602.49ms   90.05%
    Req/Sec     6.67k   401.10     8.74k    83.33%
  Latency Distribution
     50%   13.03ms
     75%   17.69ms
     90%   24.72ms
     99%   58.22ms
  398346 requests in 30.06s, 4.18GB read
  Socket errors: connect 0, read 19, write 0, timeout 0
Requests/sec:  13253.60
Transfer/sec:    142.48MB

Varnish will likely provide a somewhat different output than Apache, but it will foster overall. There were 398,346 queries in total, with an average latency of 14.41ms and no errors.

In these testing, the average response time for Apache was 44.45 milliseconds with 8,438 errors, whereas Varnish increased its response time to 14.41 milliseconds with no errors. As opposed to Apache, which must nearly always read from disk when a resource is requested, Varnish stored the response in memory and served it for subsequent requests.

Conclusion: Varnish Cache Server


Varnish was set up in this tutorial as a reverse proxy cache server for a static web page. You learned how to optimize speed with simple HTTP caching, and you used wrk to carry out load testing on the Varnish and Apache servers to compare the outcomes.

You’ve seen how the Varnish cache server speeds up your static website by providing material directly from the main memory rather than by making repeated requests to the back-end Apache server. Visit the official documentation for additional details on Varnish’s various applications.

Do You Need More Help?

Visit our forum, describe your question briefly and get answers