Matrix, the new era of open-source instant messaging
Speaking of instant messaging, you may think about WhatsApp, Telegram, Signal, etc. But those apps are stealing your privacy for business reasons, which is sad, especially when you are discussing some indescribable topics with your friends. You may run into trouble at any time. So that’s when Matrix comes in!
According to their website,
Matrix is an open-source project that publishes the
Matrix open standard for secure, decentralized, real-time communication, and its Apache licensed
reference implementations.
Maintained by the non-profit Matrix.org Foundation, we aim
to create an open platform that is as independent, vibrant, and evolving as the Web itself… but for communication.
As of June 2019, Matrix is out of beta, and the protocol is entirely suitable for production usage.
So today, I am going to teach you how to self-host your Matrix server.
We are going to use Synapse as our server and use Coturn as our TURN server. (a TURN server is used to overcome NAT restrictions and do P2P Audio/Video calls.
What you will need
-
An amd64/arm64-based server, 1 vCPU, 1 GB RAM, and as much bigger disk as possible since you’ll need to store all your chat history and files by yourself. In this case, we will use Ubuntu 22.04 LTS Jammy. (any OS with apt should be ok, if you are using yum, Pacman, yay, DNF, etc., you’ll need to figure out which command you’ll need to use by yourself.
-
A domain name, we will need 3 subdomains later.
-
A working PC/Tablet/Mobile, for us to SSH into the server
Steps
SSH into your server, and use the following command to install Docker.
curl -fsSL https://get.docker.com | bash
sudo apt update
sudo apt install docker-compose
If you are not using root account, then please use the following command to configure rootless.
sudo apt update
sudo apt install -y uidmap
dockerd-rootless-setup.sh install
Now, open your Shell RC file with your favorite editors like Vim or Nano. (Usually ~/.bashrc or ~/.profile) and insert these three lines at the end of the file.
export PATH=/usr/bin:$PATH
export PATH=/home/$(whoami)/.local/bin:$PATH
export DOCKER_HOST=unix:///run/user/1001/docker.sock
Congrats! Now Docker is installed, now we can install Synapse.
Use the following command to create a folder.
mkdir -p synapse
cd synapse
Okay, now use your favorite text editor to make a new file, named docker-compose.yml. Replace the critical arguments and paste them into your new file.
version: "3.4"
services:
synapse:
hostname: matrix
image: matrixdotorg/synapse:latest
restart: always
container_name: matrix_server
depends_on:
- db
- redis
ports:
- "127.0.0.1:8001:8008"
volumes:
- ./synapse/data:/data
networks:
- synapse_network
- external_network
healthcheck:
test: ["CMD-SHELL", "curl -s localhost:8008/health || exit 1"]
db:
image: postgres
restart: always
container_name: matrix_db
volumes:
- ./synapse/db:/var/lib/postgresql/data
environment:
POSTGRES_USER: synapse
POSTGRES_PASSWORD: DATABASE_PASSWORD_HERE # Set a strong password.
POSTGRES_DB: synapse
POSTGRES_INITDB_ARGS: "--encoding='UTF8' --lc-collate='C' --lc-ctype='C'"
networks:
- synapse_network
healthcheck:
test: ["CMD", "pg_isready", "-U", "synapse"]
redis:
image: redis:6.0-alpine
restart: always
container_name: matrix_redis
volumes:
- ./synapse/redis:/data
networks:
- synapse_network
healthcheck:
test: ["CMD", "redis-cli", "ping"]
networks:
synapse_network:
internal: true
external_network:
Now point your domain to your server’s IP. We will need 3 subdomains. We will make contoso.com our example domain.
Point chat.contoso.com to your server, this will be the front end (FRONT_END_URL). Point matrix.contoso.com to your server, this will be the back end (BACK_END_URL). Point turn.contoso.com to your server, this will be the TURN server domain(TURN_SERVER_DOMAIN).
If you are using Cloudflare, the front end and back end need to be ‘Proxied’ and the TURN needs to be ‘DNS Only’.
Run the following command to initialize. Replace BACK_END_URL as your back-end URL
docker-compose run --rm -e SYNAPSE_SERVER_NAME=BACK_END_URL -e SYNAPSE_REPORT_STATS=yes synapse generate
Now check if there’s a file called homeserver.yaml at ./synapse/data/
Open the file, we will change some critical arguments, and add some critical parameters as well.
Refer to my example configuration:
# Configuration file for Synapse.
#
# This is a YAML file: see [1] for a quick introduction. Note in particular
# that *indentation is important*: all the elements of a list or dictionary
# should have the same indentation.
#
# [1] https://docs.ansible.com/ansible/latest/reference_appendices/YAMLSyntax.html
#
# For more information on how to configure Synapse, including a complete accounting of
# each option, go to docs/usage/configuration/config_documentation.md or
# https://matrix-org.github.io/synapse/latest/usage/configuration/config_documentation.html
server_name: "FRONT_END_URL"
public_baseurl: https://BACK_END_URL
pid_file: /data/homeserver.pid
serve_server_wellknown: true
listeners:
- port: 8008
tls: false
type: http
x_forwarded: true
resources:
- names: [client, federation]
compress: false
database:
name: psycopg2
txn_limit: 10000
args:
user: synapse
password: DATABASE_PASSWORD_PLEASE_CHANGE
database: synapse
host: db
port: 5432
cp_min: 5
cp_max: 10
redis:
enabled: true
host: redis
port: 6379
log_config: LOG_CONFIG_DONT_CHANGE
media_store_path: /data/media_store
registration_shared_secret: REGISTRATION_SHARED_SECRET_DONT_CHANGE
report_stats: true
macaroon_secret_key: MACAROON_SECRET_KEY_DONT_CHANGE
form_secret: FORM_SECRET_DONT_CHANGE
signing_key_path: SIGNING_KEY_PATH_DONT_CHANGE
trusted_key_servers:
- server_name: "matrix.org"
enable_registration: true
recaptcha_public_key: "RECAPTCHA_V2_PUBLIC_KEY"
recaptcha_private_key: "RECAPTCHA_V2_PRIVATE_KEY"
enable_registration_captcha: true
# vim:ft=yaml
DO NOT COPY MY CONFIG TO REPLACE YOUR CONFIG!!! YOU SHOULD NEVER DO THIS!!!
Here are a few notes, just in case you didn’t notice my remarks in the config.
A few parameters you need to change and do not need to change but I have remarked. server_name, default is BACK_END, replace it to FRONT_END. public_baseurl, replace BACK_END_URL, but please add a ‘https://’ before it. password UNDER database, replace DATABASE_PASSWORD to the password you have set in docker-compose.yml log_config, registration_shared_secret, macaroon_secret_key, form_secret, signing_key_path: all of these should be generated automatically, no need to change, don’t copy.
i.e. Lines you need to change: 12, 13, 29, 50, 51
Then you will need to configure reCAPTCHA.
Go to https://www.google.com/recaptcha/admin/create and create a reCAPTCHA v2. Add three domains. Your front end, your back end, and app.element.io, which act as our client. Uncheck the checkbox of ‘Verify the source of the reCAPTCHA solution’. And click create.
You will get two strings, replace ‘RECAPTCHA_V2_PUBLIC_KEY’ and ‘RECAPTCHA_V2_PRIVATE_KEY’ with your Public Key and Private Key you have just got.
Now it’s time to configure our domain reverse proxy. Use the following command to install Nginx.
sudo apt update
sudo apt install nginx
cd /etc/nginx/conf.d
We suggest you use Cloudflare Origin CA (Origin Server) as your certificate and change your SSL/TLS mode to ‘Full (Strict)’.
You can use ‘Flexible’, but you will need to delete the TLS-related lines by yourself.
We will need two configs. One is named ‘frontend.conf’ and the other is named ‘backend.conf’
Use the following command to create a directory for your certificate.
sudo mkdir -p /etc/nginx/cert
And then copy your certificate to /etc/nginx/cert, the certificate must be renamed to ‘cert.crt’ and the private key must be renamed to ‘cert.key’.
It doesn’t matter if they are all ‘.pem’ files before, you can just rename them by using ‘mv’.
Copy my configs to your two files and replace some necessary parameters.
frontend.conf: (need to be changed: Line 5, 28, 34)
server
{
listen 80;
listen 443 ssl;
server_name FRONT_END_URL;
#SSL-START SSL related configuration, do NOT delete or modify the next line of commented-out 404 rules
#error_page 404/404.html;
ssl_certificate /etc/nginx/cert/cert.crt;
ssl_certificate_key /etc/nginx/cert/cert.key;
ssl_protocols TLSv1.1 TLSv1.2 TLSv1.3;
ssl_ciphers EECDH+CHACHA20:EECDH+CHACHA20-draft:EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:EECDH+3DES:RSA+3DES:!MD5;
ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;
add_header Strict-Transport-Security "max-age=31536000";
error_page 497 https://$host$request_uri;
#SSL-END
# Forbidden files or directories
location ~ ^/(.user.ini|.htaccess|.git|.svn|.project|LICENSE|README.md)
{
return 404;
}
location /.well-known/matrix/client {
return 200 '{"m.homeserver": {"base_url": "BACK_END_URL"}}';
default_type application/json;
add_header Access-Control-Allow-Origin *;
}
location /.well-known/matrix/server {
return 200 '{"m.server": "BACK_END_URL:443"}';
default_type application/json;
add_header Access-Control-Allow-Origin *;
}
}
backend.conf: (need to be changed: Line 5)
server
{
listen 80;
listen 443 ssl;
server_name BACK_END_URL;
#SSL-START SSL related configuration, do NOT delete or modify the next line of commented-out 404 rules
#error_page 404/404.html;
ssl_certificate /etc/nginx/cert/cert.crt;
ssl_certificate_key /etc/nginx/cert/cert.key;
ssl_protocols TLSv1.1 TLSv1.2 TLSv1.3;
ssl_ciphers EECDH+CHACHA20:EECDH+CHACHA20-draft:EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:EECDH+3DES:RSA+3DES:!MD5;
ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;
add_header Strict-Transport-Security "max-age=31536000";
error_page 497 https://$host$request_uri;
#SSL-END
#ERROR-PAGE-START Error page configuration, allowed to be commented, deleted or modified
#error_page 404 /404.html;
#error_page 502 /502.html;
#ERROR-PAGE-END
# Forbidden files or directories
location ~ ^/(.user.ini|.htaccess|.git|.svn|.project|LICENSE|README.md)
{
return 404;
}
location ~ ^(/_matrix|/_synapse/client) {
# note: do not add a path (even a single /) after the port in `proxy_pass`,
# otherwise nginx will canonicalise the URI and cause signature verification
# errors.
proxy_pass http://127.0.0.1:8001;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $host;
# Nginx by default only allows file uploads up to 1M in size
# Increase client_max_body_size to match max_upload_size defined in homeserver.yaml
client_max_body_size 500M;
}
}
Then use the following command to reload and enable Nginx on startup.
sudo nginx -s reload
sudo systemctl enable nginx
sudo systemctl start nginx
Then go to https://federationtester.matrix.org/ and test if your Synapse server is working. (Enter the FRONT-END URL)
Now go back to your SSH session, and change the directory back to the Synapse folder, there should be one folder and one file in it. Named ‘synapse’ and ‘docker-compose.yml’. Execute the following command to get into the Synapse Container Shell.
docker-compose exec synapse /bin/bash
Now we are going to create a new user. Execute the following command in the container to create your first user.
cd data
register_new_matrix_user -c homeserver.yaml http://localhost:8008
exit
Remember to choose ‘y’ when they ask you to Make admin.
Hooray!
Now your Synapse server is up and running. You can go to https://app.element.io to log in to your server. Remember to change the default matrix.org server to your own server. The server domain is your BACK_END DOMAIN, NOT FRONT_END!!!
About TURN server: It took me three hours to write this article and I don’t think I have the patience to write the guide on how to set up the TURN server. I will post the guide later.