Categories
Applications AWS EC2 Firewall Linux NethServer OpenVPN Roundcube Security/Vulnerability

NethServer on AWS

If you ever wanted to setup a VPN server, mail server, mail box, web server, or nextcloud (an open source google drive alternative), then you can go through this blog and set it up in minutes using NethServer.

NethServer new management web UI at 9090 port

Since AWS does not provide a NethServer AMI by default via MarketPlace or Community AMI, I had to make a VM image in my local server, modify it, import it to AWS and then modify it again and make it working for AWS.

I have made that image public and it is available in the Mumbai region. If you need it in any other region, contact me or you can make an AMI after launching the server in the Mumbai region and transfer it to whichever region you like.

So to setup a NethServer in AWS, below are the steps you have to follow

Setting up EC2

The instance

Go to the AWS EC2 console and click on Launch Instance

There go to Community AMIs and search for NethServer or you can search the below AMI ID. Do note that this AMI ID might change if I update the AMI

ami-0c0dc2c9b42edfa60

Security group rules

The security group should allow the following as per requirement for accessing the server

For the latest management web UI, open the port 9090 inbound

For old management web UI, open port 980 inbound

For SSH Access, open port 22 inbound

You can keep outbound access as full open for all traffic.

User Access

The NethServer AMI by default comes with a user. The details are as follows

Username: admin
Password: Nethserver@123

Domain/Other details

By default, this AMI comes with the FQDN ami.vigneshn.in

This can be changed and there are also my contact details in the Company name space so you can change it according to your requirements.

Also note that you have to change the FQDN before you install LDAP in account providers (Users and Groups).

You can also use Active directory (external) for user and group management.

Also you need to setup LDAP to change the password of the default admin user

Known caveats

Currently there is an issue with Nethserver where it requrires a green interface at any cost, without which it will throw and error at ipconf step.

To fix this, create a network interface in AWS and attach it to the instance.

Make this a green network and if your ip is 172.20.20.20 for the network interface, then give that as static in for green and the subnet mask as 255.255.255.0 and the gateway as 172.20.20.1

Documentation

For more details on how to set up the individual services in NethServer, you can visit the NethServer documentation here.

Categories
Applications Cluster Linux MySQL

MySQL-Cluster group replication

Follow this to setup a MySQL database cluster with group replication. Group replication is where the database across the cluster is constantly synced, so you can write or read from any database and it will be replicated across the other ones.

This increases the redundancy and read throughput.

Assumptions

Following are the assumptions that are made.

Hostname of service DB server 1: service-db1
Hostname of service DB server 2: service-db2
Hostname of service DB server 3: service-db3

IP of service DB server 1: 172.32.0.10
IP of service DB server 2: 172.32.0.20
IP of service DB server 3: 172.32.0.30

MySQL port: 3306
MySQL cluster port: 33061

Install MySQL on all three servers

Add the MySQL community repo

Download the deb file

curl -OL https://repo.mysql.com//mysql-apt-config_0.8.15-1_all.deb

Install using dpkg

sudo dpkg -i mysql-apt-config_0.8.15-1_all.deb

Select mysql-5.7

Update the apt repo

sudo apt update

Install MySQL server

sudo apt install mysql-server

Start MySQL service

sudo systemctl start mysql && sudo systemctl enable mysql

Do secure installation

sudo mysql_secure_installation

Open configuration

sudo vim /etc/mysql/mysql.conf.d/mysqld.cnf

Edit configuration

OptionOld valueNew value
bind-address127.0.0.10.0.0.0

Add /etc/hosts entries to make hostnames resolvable

Open hosts file

sudo vim /etc/hosts

Add the following configuration after changing the IPs appropriately

172.32.0.10 service-db1
172.32.0.20 service-db2
172.32.0.30 service-db3

Add firewall configuration

sudo ufw allow from 172.32.0.54 to any port 3306 proto tcp
sudo ufw allow from 172.32.0.175 to any port 3306 proto tcp
sudo ufw allow from 172.32.0.166 to any port 3306 proto tcp

sudo ufw allow out to 172.32.0.54 port 3306 proto tcp
sudo ufw allow out to 172.32.0.175 port 3306 proto tcp
sudo ufw allow out to 172.32.0.166 port 3306 proto tcp

sudo ufw allow out to 172.32.0.54 port 33061 proto tcp
sudo ufw allow out to 172.32.0.175 port 33061 proto tcp
sudo ufw allow out to 172.32.0.166 port 33061 proto tcp

sudo ufw allow from 172.32.0.54 to any port 33061 proto tcp
sudo ufw allow from 172.32.0.175 to any port 33061 proto tcp
sudo ufw allow from 172.32.0.166 to any port 33061 proto tcp

Prepare first MySQL server for group replication

Edit configuration

Open the file

sudo vim /etc/mysql/mysql.conf.d/mysqld.cnf

Add the following content

disabled_storage_engines=”MyISAM,BLACKHOLE,FEDERATED,ARCHIVE,MEMORY”
server_id=1
gtid_mode=ON
enforce_gtid_consistency=ON
master_info_repository=TABLE
relay_log_info_repository=TABLE
binlog_checksum=NONE
log_slave_updates=ON
log_bin=binlog
binlog_format=ROW
plugin_load_add=’group_replication.so’
transaction_write_set_extraction=XXHASH64
group_replication_group_name=”40d2393e-a59b-11ea-8acd-0242ac110003″
group_replication_start_on_boot=off
group_replication_local_address= “service-db1:33061″
group_replication_group_seeds= “service-db1:33061,service-db2:33061,service-db3:33061″
group_replication_bootstrap_group=off
group_replication_ip_whitelist=”service-db1,service-db2,service-db3”

The server_id changes to 2 and 3 and group_replication_local_address changes to the hostname and make it resolvable using /etc/hosts file of the current instance for all the servers in mysql cluster, and also change the whitelist IP range.

Login to mysql

sudo mysql -u root

Get UUID

SELECT UUID();

Replace UUID in the configuration file

sudo vim /etc/mysql/mysql.conf.d/mysqld.cnf

The UUID to be changed is at group_replication_group_name section

Restart MySQL server

sudo systemctl restart mysql

Login to mysql

sudo mysql -u root

Run the following commands

Turn off bin log

SET SQL_LOG_BIN=0;

Create replication user

CREATE USER rpl_user@’%’ IDENTIFIED BY ‘123456789’;

Give appropriate permission

GRANT REPLICATION SLAVE ON *.* TO rpl_user@’%’;

Flush permissions

FLUSH PRIVILEGES;

Turn on bin log

SET SQL_LOG_BIN=1;

Enable user for replication

CHANGE MASTER TO MASTER_USER=’rpl_user’, MASTER_PASSWORD=’123456789′ FOR CHANNEL ‘group_replication_recovery’;

Bootstrap the replication group

SET GLOBAL group_replication_bootstrap_group=ON;

Start the replication

START GROUP_REPLICATION;

Stop the bootstrap

SET GLOBAL group_replication_bootstrap_group=OFF;

Check if the node is online

SELECT * FROM performance_schema.replication_group_members;

Change the mysqld.cnf file group_replication_start_on_boot to on

Cluster second MySQL server to the group

Edit configuration

Open the file

sudo vim /etc/mysql/mysql.conf.d/mysqld.cnf

Add the following content

disabled_storage_engines=”MyISAM,BLACKHOLE,FEDERATED,ARCHIVE,MEMORY”
server_id=2
gtid_mode=ON
enforce_gtid_consistency=ON
master_info_repository=TABLE
relay_log_info_repository=TABLE
binlog_checksum=NONE
log_slave_updates=ON
log_bin=binlog
binlog_format=ROW
plugin_load_add=’group_replication.so’
transaction_write_set_extraction=XXHASH64
group_replication_group_name=”40d2393e-a59b-11ea-8acd-0242ac110003″
group_replication_start_on_boot=off
group_replication_local_address= “service-db2:33061″
group_replication_group_seeds= “service-db1:33061,service-db2:33061,service-db3:33061″
group_replication_bootstrap_group=off
group_replication_ip_whitelist=”service-db1,service-db2,service-db3”

The server_id changes to 2 and 3 and group_replication_local_address changes to the hostname and make it resolvable using /etc/hosts file of the current instance for all the servers in mysql cluster, and also change the whitelist IP range. Change the UUID to be the same as the first server

Restart MySQL server

sudo systemctl restart mysql

Login to mysql on master server (first server)

sudo mysql -u root

Run the following commands

Turn off bin log

SET SQL_LOG_BIN=0;

Create replication user

CREATE USER rpl_user@’%’ IDENTIFIED BY ‘123456789’;

Give appropriate permission

GRANT REPLICATION SLAVE ON *.* TO rpl_user@’%’;

Flush permissions

FLUSH PRIVILEGES;

Turn on bin log

SET SQL_LOG_BIN=1;

Enable user for replication

CHANGE MASTER TO MASTER_USER=’rpl_user’, MASTER_PASSWORD=’123456789′ FOR CHANNEL ‘group_replication_recovery’;

Start the replication

START GROUP_REPLICATION;

Disable read only mode

SET GLOBAL super_read_only=0;

Check if the node is online

SELECT * FROM performance_schema.replication_group_members;

Cluster third MySQL server to the group

Edit configuration

Open the file

sudo vim /etc/mysql/mysql.conf.d/mysqld.cnf

Add the following content

disabled_storage_engines=”MyISAM,BLACKHOLE,FEDERATED,ARCHIVE,MEMORY”
server_id=3
gtid_mode=ON
enforce_gtid_consistency=ON
master_info_repository=TABLE
relay_log_info_repository=TABLE
binlog_checksum=NONE
log_slave_updates=ON
log_bin=binlog
binlog_format=ROW
plugin_load_add=’group_replication.so’
transaction_write_set_extraction=XXHASH64
group_replication_group_name=”40d2393e-a59b-11ea-8acd-0242ac110003″
group_replication_start_on_boot=off
group_replication_local_address= “service-db3:33061″
group_replication_group_seeds= “service-db1:33061,service-db2:33061,service-db3:33061″
group_replication_bootstrap_group=off
group_replication_ip_whitelist=”service-db1,service-db2,service-db3”

The server_id changes to 2 and 3 and group_replication_local_address changes to a resolvable url or IP address of the current instance for all the servers in mysql cluster, and also change the whitelist IP range. Change the UUID to be the same as the first server

Login to mysql on master server (first server)

sudo mysql -u root

Run the following commands

Turn off bin log

SET SQL_LOG_BIN=0;

Create replication user

CREATE USER rpl_user@’%’ IDENTIFIED BY ‘123456789’;

Give appropriate permission

GRANT REPLICATION SLAVE ON *.* TO rpl_user@’%’;

Flush permissions

FLUSH PRIVILEGES;

Turn on bin log

SET SQL_LOG_BIN=1;

Enable user for replication

CHANGE MASTER TO MASTER_USER=’rpl_user’, MASTER_PASSWORD=’123456789′ FOR CHANNEL ‘group_replication_recovery’;

Start the replication

START GROUP_REPLICATION;

Disable read only mode

SET GLOBAL super_read_only=0;

Check if the node is online

SELECT * FROM performance_schema.replication_group_members;

That’s it. Your MySQL servers are now clustered with redundancy.