wiki/howto/matrix-element

[TOC]

Presentation

• Matrix server: now synapse, comming soon dendrite

• How it works?

• Status of the project. Extra

• Based on #channels, and @people attached to them. Its access: Guest (Read-only, Read + Write), Login (Local OR LDAP), Encrypted channels through Double Ratchet algorithm - device based encryption (encrypted-devices.png). History control (no, public, semipublic, private)

  • @user:<server>, #channel:<server>
  • public channels
    • Who can access this room?
    • Anyone who knows the room's link, including guests
    • List this room 's room directory?

• Clients:

  • There are a lot
  • Recommended: element.io/app HTML5 App, Desktop (Electron), AppStore, Google Play, F-Droid
    • Custom server config (login-element-guifi.png)
    • Room directory (public channels)
    • Start chat

• Bridge with other networks. Thought, response from Matrix

• element Integrations: IRC networks (TODO: upload integrations-irc.png), jitsi (TODO: upload jitsi.png), more

• Locations: Spanish 79%, Catalan 0%

Install on Debian 9 stretch 2017-10-06

This guide helps you to install a matrix server using authentication of a particular LDAP (guifi.net) with a postgresql database. Hope it helps you to be inspired on your particular needs.

matrix-element homeservers up & running that used this howto:

• https://element.guifi.net
• https://riot.musaik.net

First steps

If you want to automate this steps (terraform, ansible, docker) you would like to use advice that to debian with the following variable

export DEBIAN_FRONTEND="noninteractive"

Run all following commands as root user

Add repository

cat <<EOF > /etc/apt/sources.list.d/synapse.list
deb http://packages.matrix.org/debian/ buster main
deb-src http://packages.matrix.org/debian/ buster main
EOF

Add repo key

curl -L -s https://packages.matrix.org/debian/repo-key.asc | apt-key add -

Install synapse matrix server

apt-get install matrix-synapse-py3

The two asked options are stored here:

• /etc/matrix-synapse/conf.d/report_stats.yaml
• /etc/matrix-synapse/conf.d/server_name.yaml

Config at /etc/matrix-synapse/homeserver.yaml is overridden by config in /etc/matrix-synapse/conf.d. Let's add all stuff at /etc/matrix-synapse/conf.d/guifi.yaml:

cat <<EOF > /etc/matrix-synapse/conf.d/guifi.yaml

 # from synapse version 1 the TLS must be certified (letsencrypt is enough)
tls_certificate_path: "/etc/letsencrypt/live/matrix.example.com/fullchain.pem"
tls_private_key_path: "/etc/letsencrypt/live/matrix.example.com/privkey.pem"

 # overridden: default is sqlite
database:
  name: psycopg2
  args:
    user: synapse_user
    password: synapse_user
    database: synapse
    host: localhost
    cp_min: 5
    cp_max: 10

 # LDAP from guifi

password_providers:
 - module: "ldap_auth_provider.LdapAuthProvider"
   config:
     enabled: true
     uri: "ldaps://ldap.guifi.net"
     start_tls: true
     base: "o=webusers,dc=guifi,dc=net"
     attributes:
        uid: "uid"
        mail: "mail"
        name: "uid"

 # overridden: default is false
allow_guest_access: True

 # reverse proxy -> https://github.com/matrix-org/synapse#using-a-reverse-proxy-with-synapse
 # just adds on port 8008:
 #  + bind_addresses: ['127.0.0.1']
 #  + x_forwarded: true

listeners:
  # Main HTTPS listener
  # For when matrix traffic is sent directly to synapse.
  -
    # The port to listen for HTTPS requests on.
    port: 8448

    # Local interface to listen on.
    # The empty string will cause synapse to listen on all interfaces.
    #bind_address: ''
    # includes IPv6 -> src https://github.com/matrix-org/synapse/issues/1886
    bind_address: '::'

    # This is a 'http' listener, allows us to specify 'resources'.
    type: http

    tls: true

    # Use the X-Forwarded-For (XFF) header as the client IP and not the
    # actual client IP.
    x_forwarded: false

    # List of HTTP resources to serve on this listener.
    resources:
      -
        # List of resources to host on this listener.
        names:
          - client     # The client-server APIs, both v1 and v2

        # Should synapse compress HTTP responses to clients that support it?
        # This should be disabled if running synapse behind a load balancer
        # that can do automatic compression.
        compress: true

      - names: [federation]  # Federation APIs
        compress: false

  # Unsecure HTTP listener,
  # For when matrix traffic passes through loadbalancer that unwraps TLS.
  - port: 8008
    tls: false
    bind_address: '127.0.0.1'
    type: http

    x_forwarded: true

    resources:
      - names: [client]
        compress: true
      - names: [federation]
        compress: false

 # enable communities feature
enable_group_creation: True

# Retention policy limits. If set, a user won't be able to send a
# 'm.room.retention' event which features a 'min_lifetime' or a 'max_lifetime'
# that's not within this range. This is especially useful in closed federations,
# in which server admins can make sure every federating server applies the same
# rules.
#

retention:
  enabled: true
  default_policy:
    min_lifetime: 1d
    max_lifetime: 1y

  # If no configuration is provided, a single job will be set up to delete expired
  #   # events in every room daily.
  purge_jobs:
    - longest_max_lifetime: 1y
      interval: 1w


# Resource-constrained homeserver Settings
#
# If limit_remote_rooms.enabled is True, the room complexity will be
# checked before a user joins a new remote room. If it is above
# limit_remote_rooms.complexity, it will disallow joining or
# instantly leave.
#
# limit_remote_rooms.complexity_error can be set to customise the text
# displayed to the user when a room above the complexity threshold has
# its join cancelled.
#
# Uncomment the below lines to enable:
limit_remote_rooms:
  enabled: true
  complexity: 1.0
  complexity_error: "limit_remote_rooms is enabled in synapse with complexity=1 - contact administrator or complain publicly"
EOF

it could be interesting a welcome room for the community, let's suppose is #welcome:matrix.example.com

# welcome room were people autojoin
auto_join_rooms:
  - "#xrcb:xrcb.cat"
# without this, guest account automatically join the auto_join_rooms
auto_join_rooms_for_guests: false
# to show the directory exploration of public rooms in your server
allow_guest_access: true

update from 2021-1-20, recently in matrix.guifi.net thanks to avoiding complex rooms and to the retention of one year, we could lowered down the RAM usage from 4 GB to 1 GB and from 40+ GB (always growing) to nearly constant ~12 GB (according to the today's usage)

/etc/matrix-synapse/conf.d/extra.yaml:

 # The public-facing base URL for the client API (not including _matrix/...)
 #  $ curl -k https://matrix.example.com:8448/_matrix/client/r0/
 # {
 #     "errcode": "M_UNRECOGNIZED",
 #     "error": "Unrecognized request"
 # }
public_baseurl: https://matrix.example.com:8448/

Set up requirements for guifi LDAP

apt-get install matrix-synapse-ldap3

Set up requirements

apt-get install postgresql

create user

su -s /bin/bash postgres -c "createuser synapse_user"

enter postresql CLI:

su -s /bin/bash postgres -c psql

put password to user

ALTER USER "synapse_user" WITH PASSWORD 'synapse_user';

and set up database

CREATE DATABASE synapse
 ENCODING 'UTF8'
 LC_COLLATE='C'
 LC_CTYPE='C'
 template=template0
 OWNER synapse_user;

Set up client in Debian/Ubuntu

apt-get install libpq-dev python3-pip python3-psycopg2

note: synapse currently assumes python 2.7 by default

Start or restart matrix service

service matrix-synapse restart

To check if is running:

service matrix-synapse status

TLS renewal

In guifi.yaml we already added the tls files, extra work is required to access letsencrypt files and to do renewall succesful.

Considering you created a user for the letsencrypt renewal process with letsencrypt user. Add matrix-synapse in the group of letsencrypt

gpassswd -a matrix-synapse letsencrypt

And change permissions of /etc/letsencrypt directory to be able to access through matrix reference

chgrp -R letsencrypt /etc/letsencrypt
chmod -R g=rX /etc/letsencrypt

One time each month a renewal is performed:

cat <<EOF > /etc/cron.monthly/matrix-synapse
#!/bin/bash
service matrix-synapse restart
EOF
chmod +x /etc/cron.monthly/matrix-synapse

DNS configuration

This DNS configuration is required to see federation working in your matrix server

More info setting up federation

matrix.example.com IN A <IP>
element.example.com IN A <IP>
_matrix._tcp.example.com. 3600 IN SRV 10 0 8448 matrix.example.com.

Accessibility to the server

Requirements:

apt-get install certbot nginx-full

reverse proxy server with nginx

matrix_domain="matrix.example.com"
cat <<EOF > /etc/nginx/sites-available/${matrix_domain}
server {
    listen 80;
    listen [::]:80;
    server_name ${matrix_domain};

    location /.well-known {
            default_type "text/plain";
            allow all;
            root /var/www/html;
    }

    return 301 https://\$host\$request_uri;
}

server {
    listen 443 ssl;
    listen [::]:443 ssl;
    server_name ${matrix_domain};

    ssl_certificate /etc/letsencrypt/live/${matrix_domain}/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/${matrix_domain}/privkey.pem;

    location /.well-known {
            default_type "text/plain";
            allow all;
            root /var/www/html;
    }

    # static front page to anounce how works the service
    # example: https://github.com/guifi-exo/public/tree/master/web/matrix.guifi.net
    location / {
        root /var/www/html;
        try_files /matrix.html /matrix.html;
    }

    location ~ ^(/_matrix|/_synapse/client) {
        proxy_pass http://127.0.0.1:8008;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Host $host;

        # Nginx by default only allows file uploads up to 1M in size
        # Increase client_max_body_size to match max_upload_size defined in homeserver.yaml
        client_max_body_size 50M;
    }
    # Synapse responses may be chunked, which is an HTTP/1.1 feature.
    proxy_http_version 1.1;
}
EOF

ln -s /etc/nginx/sites-available/${matrix_domain}.conf /etc/nginx/sites-enabled/${matrix_domain}.conf
certbot certonly -n --keep --agree-tos --email ${matrix_email} --webroot -w /var/www/html/ -d ${matrix_domain}
service nginx reload

src https://matrix-org.github.io/synapse/latest/reverse_proxy.html

use reverse proxy server (nginx) as the endpoint for federation

you need to specify where is the federation port, by default seems to be looking for the 8448 matrix server port, to override it, add this (with your matrix URL):

mkdir -p /var/www/html/.well-known/
cat > /var/www/html/.well-known/matrix/server <<EOF
{"m.server": "matrix.example.com:443"}
EOF

web static client

element_domain="element.example.com"
element_email="info@example.com"
cat <<EOF > /etc/nginx/sites-available/${element_domain}.conf
server {
    listen 80;
    listen [::]:80;
    server_name ${element_domain};

    location /.well-known {
            default_type "text/plain";
            allow all;
            root /var/www/html;
    }

    return 301 https://\$host\$request_uri;
}

server {
    listen 443 ssl;
    listen [::]:443 ssl;
    server_name ${element_domain};

    ssl_certificate /etc/letsencrypt/live/${element_domain}/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/${element_domain}/privkey.pem;

    location /.well-known {
            default_type "text/plain";
            allow all;
            root /var/www/html;
    }

    root /var/www/html/element-web;
}
EOF

ln -s /etc/nginx/sites-available/${element_domain}.conf /etc/nginx/sites-enabled/${element_domain}.conf

certbot certonly -n --keep --agree-tos --email ${element_email} --webroot -w /var/www/html/ -d ${element_domain}

service nginx reload

script to upgrade static element

Requirements:

apt-get install jq

take file element-update.sh and put it in /var/www/html/element-web

add a config file for your site like config.element.example.com.json inside element-web directory /var/www/html/element-web

{
    "default_server_config": {
        "m.homeserver": {
            "base_url": "https://matrix.example.com",
            "server_name": "matrix.example.com"
        }
    },
    "disable_identity_server": true,
    "disable_custom_urls": true,
    "disable_guests": false,
    "disable_login_language_selector": false,
    "disable_3pid_login": false,
    "brand": "matrix.example.com",
    "integrations_ui_url": "https://scalar.vector.im/",
    "integrations_rest_url": "https://scalar.vector.im/api",
    "integrations_widgets_urls": [
        "https://scalar.vector.im/_matrix/integrations/v1",
        "https://scalar.vector.im/api",
        "https://scalar-staging.vector.im/_matrix/integrations/v1",
        "https://scalar-staging.vector.im/api",
        "https://scalar-staging.riot.im/scalar/api"
    ],
    "integrations_jitsi_widget_url": "https://scalar.vector.im/api/widgets/jitsi.html",
    "bug_report_endpoint_url": "https://element.io/bugreports/submit",
    "defaultCountryCode": "ES",
    "showLabsSettings": false,
    "features": {
        "feature_pinning": "labs",
        "feature_custom_status": "labs",
        "feature_custom_tags": "labs",
        "feature_state_counters": "labs"
    },
    "default_federate": true,
    "default_theme": "light",
    "roomDirectory": {
        "servers": [
            "matrix.example.com"
        ]
    },
    "piwik": {},
    "enable_presence_by_hs_url": {
        "https://matrix.example.com": true
    },
    "settingDefaults": {
        "breadcrumbs": true
    }
}

edit file vi /etc/cron.d/updateelement, add following content:

SHELL=/bin/bash

40 6 * * * root /var/www/html/element-web/bin/element-update.sh

new element releases fill directory /var/www/html/element-web/bundles/. Check it out from time to time

Data

Data grows here:

• /var/lib/postgresql
• /var/lib/matrix-synapse/media
• /var/log/matrix-synapse/ - warning, uses up to 1 GB, change behavior in /etc/matrix-synapse/log.yaml

I symlink this directories to specific volume

learn more about how data grows https://matrix.org/docs/projects/other/hdd-space-calc-for-synapse.html

Test federation

To test federation you can use this service: https://matrix.org/federationtester/api/report?server_name=matrix.example.com

with more visuals: https://matrix.org/federationtester/ or https://arewereadyyet.com/

source: https://github.com/matrix-org/matrix-federation-tester

other installation guides

https://matrix.org/docs/guides/installing-synapse

https://matrix.org/docs/guides/#installing-synapse

src https://matrix.org/blog/2019/01/17/bens-favourite-projects-2018/

A walk through of installing Synapse+Riot+Jitsi from scratch on Debian https://matrix.org/blog/2020/04/06/running-your-own-secure-communication-service-with-matrix-and-jitsi

known problems

notifications

some people do not receive notifications in its smartphone