tech_documents:linux:repo_mirror

Repo Mirror Using Rocky Linux 8

Repo Mirror Using Rocky Linux 8

https://help.ubuntu.com/community/Rsyncmirror
https://computingforgeeks.com/create-local-centos-7-centos-6-mirrors/

Plan is to create a slimmed down mirror of CentOS 7, Rocky Linux 8, ELRepo, Remi PHP Repo, Zimbra Repo and others for internal systems (not as a public mirror or as an archive). Starting with a 500GB disk.

Base Install

Download and install Rocky Linux 8 minimal, setup FQDN and set static IP, enable NTP and set timezone, set root password, use LVM for partitioning, select minimal software.

Install Misc Tools, Create Sudo User and Reboot

dnf install unzip wget vim tar rsync
dnf update
useradd example_user && passwd example_user
usermod -aG wheel example_user
shutdown -r now

Disallow root login over SSH

sudo vim /etc/ssh/sshd_config

then set

PermitRootLogin no

Restart sshd

sudo systemctl restart sshd

Setup/Sync Repos CentOS/Rocky

Create Repo Dirs

sudo mkdir -p /var/mirror/pub/centos/7
sudo mkdir -p /var/mirror/pub/rocky/8
sudo mkdir -p /var/mirror/pub/epel/7
sudo mkdir -p /var/mirror/pub/epel/8
sudo mkdir -p /var/mirror/pub/remi/enterprise/7/php74
sudo mkdir -p /var/mirror/pub/remi/enterprise/8
sudo mkdir -p /var/mirror/pub/debian

Repo Mirror Script for CentOS

https://gist.github.com/didip/4506837

Create file centos.sh and chmod +x it

Note: there are excludes so that only the packages I want are synced, modify to your needs

#!/bin/bash
 
# Number of arguments should be at least 1
if [ $# -lt 1 ]; then
  echo "Usage: $0 centos-version-number"
  exit 1
fi
 
VERSION=$1
 
# Change these variables as appropriate
LOCK_FILE=/tmp/centos_mirror_rsync_updates
DOWNLOAD_LOCATION=rsync://mirrors.kernel.org/centos
TARGET_DIR=/var/mirror/pub/centos
RSYNC_RETVAL=1
 
if [ -f $LOCK_FILE ]; then
  echo "CentOS updates via rsync already running."
  exit 0
fi
 
echo "Starting rsync from $DOWNLOAD_LOCATION/$VERSION to $TARGET_DIR/$VERSION..."
mkdir -p $TARGET_DIR/$VERSION
 
touch $LOCK_FILE
 
# add --exclude as necessary
rsync  -avSHP --delete --delete-excluded --copy-links --exclude "atomic" --exclude "cloud" --exclude "configmanagement" --exclude "cr" --exclude "dotnet" --exclude "fasttrack" --exclude "infra" --exclude "messaging" --exclude "nfv" --exclude "opstools" --exclude "paas" --exclude "rt" --exclude "sclo"  --exclude "storage"  --exclude "virt" $DOWNLOAD_LOCATION/$VERSION $TARGET_DIR/
RSYNC_RETVAL=$?
 
rsync -avSHP $DOWNLOAD_LOCATION/RPM-GPG-KEY-CentOS-$VERSION  $TARGET_DIR/RPM-GPG-KEY-CentOS-$VERSION 
 
/bin/rm -f $LOCK_FILE
 
if [ $RSYNC_RETVAL -eq 0 ]; then
  echo "Finished rsync from $DOWNLOAD_LOCATION/$VERSION to $TARGET_DIR/$VERSION."
fi
 
exit 0

Run the script for CentOS 7:

sudo ./centos.sh 7

Repo Mirror Script for Rocky Linux

https://gist.github.com/didip/4506837

Create file rocky.sh and chmod +x it

Note: there are excludes so that only the packages I want are synced, modify to your needs

#!/bin/bash
 
# Number of arguments should be at least 1
if [ $# -lt 1 ]; then
  echo "Usage: $0 rocky-version-number"
  exit 1
fi
 
VERSION=$1
 
# Change these variables as appropriate
LOCK_FILE=/tmp/rocky_mirror_rsync_update
DOWNLOAD_LOCATION=rsync://mirrors.vcea.wsu.edu/rocky
TARGET_DIR=/var/mirror/pub/rocky
RSYNC_RETVAL=1
 
if [ -f $LOCK_FILE ]; then
  echo "Rocky updates via rsync already running."
  exit 0
fi
 
echo "Starting rsync from $DOWNLOAD_LOCATION/$VERSION to $TARGET_DIR/$VERSION..."
mkdir -p $TARGET_DIR/$VERSION
 
touch $LOCK_FILE
 
# add --exclude as necessary
rsync  -avSHP --delete --delete-excluded --copy-links --exclude "AppStream/aarch64" --exclude "AppStream/source" --exclude "AppStream/x86_64/debug" --exclude "AppStream/x86_64/kickstart" --exclude "BaseOS/aarch64" --exclude "BaseOS/source" --exclude "BaseOS/x86_64/iso" --exclude "BaseOS/x86_64/debug" --exclude "BaseOS/x86_64/kickstart" --exclude "Devel" --exclude "HighAvailability" --exclude "isos/aarch64" --exclude "isos/x86_64" --exclude "live/x86_64" --exclude "Live" --exclude "Minimal" --exclude "PowerTools" --exclude "RT" --exclude "ResilientStorage" --exclude "extras/aarch64" --exclude "extras/source" --exclude "extras/x86_64/debug" --exclude "plus/aarch64" --exclude "plus/source" --exclude "plus/x86_64/debug" --exclude "images"  --exclude "nfv"  --exclude "rockypi" $DOWNLOAD_LOCATION/$VERSION $TARGET_DIR/
RSYNC_RETVAL=$?
 
rsync -avSHP $DOWNLOAD_LOCATION/RPM-GPG-KEY-rockyofficial  $TARGET_DIR/RPM-GPG-KEY-rockyofficial
 
/bin/rm -f $LOCK_FILE
 
if [ $RSYNC_RETVAL -eq 0 ]; then
  echo "Finished rsync from $DOWNLOAD_LOCATION/$VERSION to $TARGET_DIR/$VERSION."
fi
 
exit 0

Run the script for Rocky Linux 8:

sudo ./rocky.sh 8

EPEL

Same as Rocky but modified for EPEL. Pick an appropriate mirror closer to you and modify the exclusions as necessary

#!/bin/bash
 
# Number of arguments should be at least 1
if [ $# -lt 1 ]; then
  echo "Usage: $0 epel-version-number"
  exit 1
fi
 
VERSION=$1
 
# Change these variables as appropriate
LOCK_FILE=/tmp/epel_mirror_rsync_update
DOWNLOAD_LOCATION=rsync://dfw.mirror.rackspace.com/epel
TARGET_DIR=/var/mirror/pub/epel
RSYNC_RETVAL=1
 
if [ -f $LOCK_FILE ]; then
  echo "epel updates via rsync already running."
  exit 0
fi
 
echo "Starting rsync from $DOWNLOAD_LOCATION/$VERSION to $TARGET_DIR/$VERSION..."
mkdir -p $TARGET_DIR/$VERSION
 
touch $LOCK_FILE
 
# add --exclude as necessary
rsync  -avSHP --delete --delete-excluded --copy-links --exclude "Everything/SRPMS/" --exclude "Everything/aarch64/" --exclude "Everything/ppc64le" --exclude "Everything/s390x/" --exclude "Everything/source/" --exclude "Everything/x86_64/debug/" --exclude "Modular/SRPMS/" --exclude "Modular/aarch64/" --exclude "Modular/ppc64le" --exclude "Modular/s390x/" --exclude "Modular/source/" --exclude "Modular/x86_64/debug/" $DOWNLOAD_LOCATION/$VERSION $TARGET_DIR/
RSYNC_RETVAL=$?
 
rsync -avSHP $DOWNLOAD_LOCATION/RPM-GPG-KEY-EPEL-$VERSION  $TARGET_DIR/RPM-GPG-KEY-EPEL-$VERSION
 
/bin/rm -f $LOCK_FILE
 
if [ $RSYNC_RETVAL -eq 0 ]; then
  echo "Finished rsync from $DOWNLOAD_LOCATION/$VERSION to $TARGET_DIR/$VERSION."
fi
 
exit 0

Setup/Sync Repos Debian

We're going to use debmirror to create a private mirror: https://linux.die.net/man/1/debmirror

Install EPEL release and debmirror:

sudo dnf install epel-release
sudo dnf install debmirror

Edit the config file:

sudo vim /etc/debmirror.conf

Set the variables as such:

# Location of the local mirror (use with care)
$mirrordir="/var/mirror/debian";
 
# Output options
$verbose=1;
$progress=1;
$debug=0;
 
# Download options
$host="ftp.us.debian.org";
$user="anonymous";
$passwd="anonymous@";
$remoteroot="debian";
$download_method="rsync";
@dists="bullseye";
@sections="main,main/debian-installer,contrib,non-free";
@arches="amd64";

debian-archive-keyring.gpg

I couldn't find a simple way to get this file on a non-debian system so I ended up grabbing it from another Debian Bullseye/11 install.

Create dir for gpg file:

sudo mkdir -p /usr/share/keyring

Copy the debian-archive-keyring.gpg file to /usr/share/keyring

Import the key:

gpg --keyring /usr/share/keyrings/debian-archive-keyring.gpg --export | gpg --no-default-keyring --keyring trustedkeys.gpg --import

Run the mirror sync

sudo ./debmirror

Install Apache

Configure hosts

sudo vim /etc/hosts

Add a line for your FQDN

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.10.30  mirror.domainname.com mirror

Install Apache and Configure Firewall

sudo dnf install httpd httpd-tools
sudo systemctl enable httpd
sudo systemctl start httpd
sudo firewall-cmd --permanent --zone=public --add-service=http
sudo firewall-cmd --permanent --zone=public --add-service=https
sudo firewall-cmd --reload

Create Virtual Hosts

Setup virtual host config files

sudo vim /etc/httpd/conf.d/mirror.example.conf

Add the following content. Make sure the website directory path, server name, and PHP version match your setup:

<Directory /var/mirror>
        Require all granted
        Options +Indexes
</Directory>
<VirtualHost *:80>
     ServerAdmin webmaster@example.com
     ServerName mirror.example.com
     DocumentRoot /var/mirror
     ErrorLog /var/log/httpd/mirror.example.com_error.log
     CustomLog /var/log/httpd/mirror.example.com_access.log combined
</VirtualHost>
<IfModule mod_autoindex.c>
  IndexOptions NameWidth=*
</ifModule>

Save and close the file when you are finished. Then check the Apache configuration file for any syntax errors:

sudo httpd -t

You’ll see the following output:

Output
Syntax OK

Disable Exposing Apache Server Info

https://www.if-not-true-then-false.com/2009/howto-hide-and-modify-apache-server-information-serversignature-and-servertokens-and-hide-php-version-x-powered-by/
This will help to prevent being the target of a an attack if you're running an out of date or vunerable release

sudo vim /etc/httpd/conf/httpd.conf

Add the following:

ServerSignature Off
ServerTokens Prod

delete the welconf.conf to disable the default apache welcome page.

sudo rm /etc/httpd/conf.d/welcome.conf

Remove Default Access from httpd.conf

This is important as it would allow access to the /var/www/html path via HTTP.

sudo vim /etc/httpd/conf/httpd.conf

Modify the following statements to mirror the examples below:

DocumentRoot "/var/www/html"

<Directory "/var/www">
    AllowOverride None
    Require all denied
    Options None
</Directory>

<Directory "/var/www/html">
    AllowOverride None
    Require all denied
    Options None
</Directory>

<Directory "/var/www/cgi-bin">
    AllowOverride None
    Require all denied
    Options None
</Directory>

Set Owner and File Permissons

sudo chown -R apache:apache /var/mirror
sudo chmod -R 755 /var/mirror

Set SElinux Permissions

https://www.redhat.com/sysadmin/apache-yum-dnf-repo

sudo chcon -Rt httpd_sys_content_t /var/mirror/

Finally, restart the Apache service to implement your changes:

sudo systemctl restart httpd

Automatic Updates for Rocky Linux

https://www.tecmint.com/dnf-automatic-install-security-updates-automatically-in-centos-8/

sudo dnf install dnf-automatic
sudo vim /etc/dnf/automatic.conf

Set:

upgrade_type = security
download_updates = yes
apply_updates = yes
system_name = (your system name)
emit_via = motd

Enable the auto-update timer

sudo systemctl enable --now dnf-automatic.timer

fail2ban

https://idroot.us/install-fail2ban-centos-8/ https://www.digitalocean.com/community/tutorials/how-to-protect-an-apache-server-with-fail2ban-on-ubuntu-14-04

sudo dnf install epel-release
sudo dnf install fail2ban

Create a Jail for SSHd

sudo vim /etc/fail2ban/jail.d/sshd.local

Add the following:

[sshd]
enabled = true
port    = ssh
logpath = %(sshd_log)s
backend = %(sshd_backend)s

[selinux-ssh]
enabled  = true
port     = ssh
logpath  = %(auditd_log)s

Create a Jail for Apache

sudo vim /etc/fail2ban/jail.d/apache.local

Add the following:

[apache-auth]
enabled  = true
port     = http,https
logpath  = %(apache_error_log)s

[apache-badbots]
enabled  = true
port     = http,https
logpath  = %(apache_access_log)s
bantime  = 48h
maxretry = 1

[apache-noscript]
enabled  = true
port     = http,https
logpath  = %(apache_error_log)s

[apache-overflows]
enabled  = true
port     = http,https
logpath  = %(apache_error_log)s
maxretry = 2

[apache-nohome]
enabled  = true
port     = http,https
logpath  = %(apache_error_log)s
maxretry = 2

[apache-botsearch]
enabled  = true
port     = http,https
logpath  = %(apache_error_log)s
maxretry = 2

[apache-fakegooglebot]
enabled  = true
port     = http,https
logpath  = %(apache_access_log)s
maxretry = 1
ignorecommand = %(ignorecommands_dir)s/apache-fakegooglebot <ip>

[apache-modsecurity]
enabled  = true
port     = http,https
logpath  = %(apache_error_log)s
maxretry = 2

[apache-shellshock]
enabled = true
port    = http,https
logpath = %(apache_error_log)s
maxretry = 1

sudo systemctl start fail2ban
sudo systemctl enable fail2ban
sudo fail2ban-client status sshd

Configure Client Repos

CentOS 7

Edit the Repo Config

sudo vim /etc/yum.repos.d/CentOS-Base.repo

Comment out any line like

mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os&infra=$infra

Uncomment the baseurl lines and change the FQDN to your FQDN or IP of the repo server

baseurl=http://10.222.0.21/centos/$releasever/os/$basearch/

Now you should be able to do a yum update.

Rocky 8

sudo vim /etc/yum.repos.d/Rocky-AppStream.repo

Modify as such, commenting out mirror and uncommenting baseurl then modifying the url:

# Rocky-AppStream.repo
#
# The mirrorlist system uses the connecting IP address of the client and the
# update status of each mirror to pick current mirrors that are geographically
# close to the client.  You should use this for Rocky updates unless you are
# manually picking other mirrors.
#
# If the mirrorlist does not work for you, you can try the commented out
# baseurl line instead.

[appstream]
name=Rocky Linux $releasever - AppStream
#mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=$basearch&repo=AppStream-$releasever
baseurl=http://10.222.0.21/$contentdir/$releasever/AppStream/$basearch/os/
gpgcheck=1
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rockyofficial

Repeat for /etc/yum.repos.d/Rocky-BaseOS.repo and any other branches you've included and will use from your local mirror.

Table of Contents

Repo Mirror Using Rocky Linux 8

Base Install

Install Misc Tools, Create Sudo User and Reboot

Disallow root login over SSH

Setup/Sync Repos CentOS/Rocky

Repo Mirror Script for CentOS

Repo Mirror Script for Rocky Linux

EPEL

Setup/Sync Repos Debian

debian-archive-keyring.gpg

Install Apache

Configure hosts

Install Apache and Configure Firewall

Create Virtual Hosts

Setup virtual host config files

Disable Exposing Apache Server Info

Remove Default Access from httpd.conf

Set Owner and File Permissons

Set SElinux Permissions

Automatic Updates for Rocky Linux

fail2ban

Create a Jail for SSHd

Create a Jail for Apache

Configure Client Repos

CentOS 7