Mirror Upstream Clear Linux OS Update Server

For organizations that want to use the Clear Linux OS upstream updates, but want the benefits of a local mirror, this tutorial shows how to set up one and configure your Clear Linux OS clients to use it.

Prerequisites

  • The recommended disk space for the mirror server should have at least 100GB of disk space as each complete update content is approximately 45GB.

Install up Clear Linux OS server to host updates

  1. Follow the Install Clear Linux* OS from the live server guide to install Clear Linux OS server. Add a user with Administrator privilege.

  2. After installation is complete, boot it up.

  3. Add the wget bundle. This will be used to clone the upstream Clear Linux OS server.

    sudo swupd bundle-add wget

Clone the Clear Linux OS update content

Clear Linux OS periodically releases a “minversion”, which is a complete update. Then, subsequent releases are small updates until the next minversion.

Download a minversion to start your mirror.

  1. Determine a proper minversion by looking at a Manifest.MoM file for a particular release of Clear Linux OS.

    For example, if you look at https://cdn.download.clearlinux.org/update/33010/Manifest.MoM, you will see that the minversion is 32900. So clone this version as the starting point.

    MANIFEST        30
version:        33010
previous:       33000
minversion:     32900
filecount:      1131
timestamp:      1588358889
contentsize:    0

  2. Make a directory to store the cache.

    mkdir ~/mirror-download-clearlinux-org && cd $_

  3. Recursively download the update/0 folder.

    wget --no-verbose \
--no-parent --recursive \
--no-host-directories -erobots=off \
--reject "index.html" https://cdn.download.clearlinux.org/update/0/

  4. Recursively download the update/version folder.

    wget --no-verbose \
--no-parent --recursive \
--no-host-directories -erobots=off \
--reject "index.html" https://cdn.download.clearlinux.org/update/version/

  5. Now, recursively download the determined minversion, which for this example is 32900.

    wget --no-verbose \
--no-parent --recursive \
--no-host-directories -erobots=off \
--reject "index.html" https://cdn.download.clearlinux.org/update/32900/

    Note

    A minversion is pretty big, which is approximately 45GB. Depending on your proximity to the upstream server and your connection speed to the Internet, it may take up to a couple of days or more to complete the download. So be patient.

  6. Download later versions, up to the latest, if you like.

Setup a web server to host the mirrored content

By design, the Clear Linux OS swupd client communicates with the update server using HTTPS for security reasons. However, it can use HTTP by adding the --allow-insecure-http flag, if needed. Setting an HTTPS is a lot more involved. For this tutorial, we’ll just use an HTTP server for demonstration purpose.

  1. Install the nginx bundle.

    sudo swupd bundle-add nginx

  2. Configure the web server.

    1. Create a symbolic link to the mirrored update content directory.

      sudo mkdir -p /var/www && cd $_
sudo ln -sf $HOME/mirror-download-clearlinux-org mirror-download-clearlinux-org

    2. Set up nginx configuration files.

      sudo mkdir -p /etc/nginx/conf.d
sudo cp /usr/share/nginx/conf/nginx.conf.example /etc/nginx/nginx.conf

    3. Grant $USER permission to run the web server.

      sudo tee -a /etc/nginx/nginx.conf << EOF
user $USER;
EOF

    4. Configure the web server.

      sudo tee -a /etc/nginx/conf.d/mirror-download-clearlinux-org.conf << EOF
server {
  listen 80;
  listen [::]:80;
  server_name localhost;
  location / {
    root /var/www/mirror-download-clearlinux-org;
    autoindex on;
  }
}
EOF

    5. Set nginx to start automatically on boot and then start it.

      sudo systemctl enable nginx --now

Test your mirror

Now, try out your mirror by installing Clear Linux OS and adding bundles from it.

  1. Download either the live desktop or live server installer ISO of the same version as the mirrored version, which is 32900 for this tutorial. Go to https://cdn.download.clearlinux.org/releases/<release-version>/clear.

  2. Burn the ISO to a thumb drive. See Create a bootable USB drive using Etcher*.

  3. Boot it up and start the installer. Depending on which version of Clear Linux OS you want to install, follow one of these guides:

    In the Advanced options tab of the installer, select Swupd Mirror. See Figure 1.

    Advanced options > Swupd Mirror

    Figure 1: Advanced options > Swupd Mirror

    In the Mirror URL field, set it to the IP address of your mirror. It should be something like this: http://<IP address of mirror server>/update. And check the option Allow installation over insecure connections (http://). See Figure 2.

    Advanced options > Mirror URL setting

    Figure 2: Advanced options > Mirror URL setting

  4. After installation completes, boot up, and log in.

  5. Verify that the swupd client is pointing to your mirror.

    sudo swupd info

    Example output:

    Warning: This is an insecure connection
The --allow-insecure-http flag was used, be aware that this poses a threat the system

Distribution:      Clear Linux OS
Installed version: 32900
Version URL:       https://192.168.1.100/update
Content URL:       https://192.168.1.100/update

  6. Try listing available bundles on your mirror.

    sudo swupd bundle-list -a

  7. Add a bundle.

    sudo swupd bundle-add <bundle-name>

Keep your mirror in sync with upstream

Be sure to keep your mirror in sync with upstream so that your clients have the latest and greatest software and security updates. You can do that continuing to clone the newer upstream releases.