nos_oignons (0.0.1.dev)
mail
prawn
+ rest-client
safe_yaml
GEM
remote: https://rubygems.org/
specs:
+ addressable (2.8.4)
+ public_suffix (>= 2.0.2, < 6.0)
aruba (2.1.0)
bundler (>= 1.17, < 3.0)
childprocess (>= 2.0, < 5.0)
builder (3.2.4)
childprocess (4.1.0)
contracts (0.16.1)
+ crack (0.4.5)
+ rexml
cucumber (8.0.0)
builder (~> 3.2, >= 3.2.4)
cucumber-ci-environment (~> 9.0, >= 9.0.4)
cucumber-tag-expressions (4.1.0)
date (3.3.3)
diff-lcs (1.5.0)
+ domain_name (0.5.20190701)
+ unf (>= 0.0.5, < 1.0.0)
ffi (1.15.5)
+ hashdiff (1.0.1)
+ http-accept (1.7.0)
+ http-cookie (1.0.5)
+ domain_name (~> 0.5)
json (2.6.2)
mail (2.8.1)
mini_mime (>= 0.1.1)
timeout
net-smtp (0.3.3)
net-protocol
+ netrc (0.11.0)
pdf-core (0.9.0)
prawn (2.4.0)
pdf-core (~> 0.9.0)
ttfunk (~> 1.7)
+ public_suffix (5.0.1)
+ rest-client (2.1.0)
+ http-accept (>= 1.7.0, < 2.0)
+ http-cookie (>= 1.0.2, < 2.0)
+ mime-types (>= 1.16, < 4.0)
+ netrc (~> 0.8)
+ rexml (3.2.5)
rspec-expectations (3.11.0)
diff-lcs (>= 1.2.0, < 2.0)
rspec-support (~> 3.11.0)
timecop (0.9.5)
timeout (0.3.2)
ttfunk (1.7.0)
+ unf (0.1.4)
+ unf_ext
+ unf_ext (0.0.8.2)
+ webmock (3.18.1)
+ addressable (>= 2.8.0)
+ crack (>= 0.3.2)
+ hashdiff (>= 0.4.0, < 2.0.0)
PLATFORMS
ruby
matrix (~> 0.4.2)
nos_oignons!
timecop
+ webmock
BUNDLED WITH
2.2.5
Met à jour la liste des adresses email inscrites à la liste *ag@* par rapport
aux membres à jour de cotisation. À exécuter à travers un *cron*. A besoin de
-pouvoir lancer les commandes `list_members`, `add_members` et `remove_members`
-via `sudo` sur le compte `list`.
+pouvoir se connecter à l’API REST de Mailman 3.
+
+La variable d’environnement suivante doit être
+définie `NOS_OIGNONS_MAILMAN_REST_API_URL` avec par exemple :
+`https://user:password@mailman.example.org`
+
+À défaut, cette dernière sera lue depuis le fichier `mailman-rest-url`
+dans le dossier indiqué par la variable d’environnemnet `CREDENTIALS_DIRECTORY`
+(voir [System and Service Credentials](https://systemd.io/CREDENTIALS/) et
+la procédure de déploiement indiqué plus bas).
`send-membership-reminders`
---------------------------
Ne pas oublier de le rendre exécutable.
-Pour permettre au script `update-ag-subscribers` de fonctionner, il est
-nécessaire de l'autoriser à exécuter certaines commandes de Mailman.
-Pour cela, on va créer un fichier dans `/etc/sudoers.d` :
+On va utiliser des
+[services](https://www.freedesktop.org/software/systemd/man/systemd.service.html)
+et des [timers](https://www.freedesktop.org/software/systemd/man/systemd.timer.html)
+systemd pour les scripts à utiliser régulièrement.
+
+Dans `/etc/systemd/system/gestion-adh-update-ag-subscribers.service` :
+
+ [Unit]
+ Description=Update subscribers of the `ag` mailing list according to memberships
+
+ [Service]
+ Type=oneshot
+ WorkingDirectory=/srv/ikiwiki/wiki-ca/gestion-adh
+ User=wiki-ca
+ ExecStart=/srv/ikiwiki/wiki-ca/gestion-adh/bin/update-ag-subscribers
+ Environment=BUNDLE_GEMFILE=/srv/ikiwiki/wiki-ca/gestion-adh/Gemfile NOS_OIGNONS_BOARD_WIKI_PATH=/srv/ikiwiki/wiki-ca/src
+ SyslogIdentifier=update-ag-subscribers
+ ProtectSystem=strict
+ ProtectHome=true
+ PrivateTmp=yes
+ PrivateDevices=yes
+
+Dans `/etc/systemd/system/gestion-adh-update-ag-subscribers.timer` :
+
+ [Unit]
+ Description=Run update-ag-subscribers every hour
+
+ [Timer]
+ RandomizedDelaySec=30min
+ OnCalendar=hourly
+
+ [Install]
+ WantedBy=timers.target
+
+Il est également nécessaire de configurer le *credential* avec les informations
+de connexion à Mailman :
+
+ echo -n 'https://USER:SECRET_PASSWORD@localhost:8001' | sudo systemd-creds encrypt --name=mailman-rest-url -p - - | sudo tee -a /etc/systemd/system/gestion-adh-update-subscribers.service.d/overrides.conf
+
+Dans `/etc/systemd/system/gestion-adh-send-membership-reminders.service` :
+
+ [Unit]
+ Description=Send reminders to renew membership
+
+ [Service]
+ Type=oneshot
+ WorkingDirectory=/srv/ikiwiki/wiki-ca/gestion-adh
+ User=wiki-ca
+ ExecStart=/srv/ikiwiki/wiki-ca/gestion-adh/bin/send-membership-reminders
+ Environment=BUNDLE_GEMFILE=/srv/ikiwiki/wiki-ca/gestion-adh/Gemfile NOS_OIGNONS_BOARD_WIKI_PATH=/srv/ikiwiki/wiki-ca/src
+ SyslogIdentifier=send-membership-reminders
+ ProtectSystem=strict
+ ProtectHome=true
+ PrivateTmp=yes
+ PrivateDevices=yes
+ ReadWritePaths=/srv/ikiwiki/wiki-ca/gestion-adh/var
+
+Dans `/etc/systemd/system/gestion-adh-send-membership-reminders.timer` :
+
+ [Unit]
+ Description=Run send-membership-reminders every day
+
+ [Timer]
+ OnCalendar=06:42
+
+ [Install]
+ WantedBy=timers.target
+
+Dans `/etc/systemd/system/gestion-adh-send-member-emails-to-advisors.service` :
+
+ [Unit]
+ Description=Send member emails to the advisory board
- Defaults:wiki-ca !requiretty
+ [Service]
+ Type=oneshot
+ WorkingDirectory=/srv/ikiwiki/wiki-ca/gestion-adh
+ User=wiki-ca
+ ExecStart=/srv/ikiwiki/wiki-ca/gestion-adh/bin/send-member-emails-to-advisors
+ Environment=BUNDLE_GEMFILE=/srv/ikiwiki/wiki-ca/gestion-adh/Gemfile NOS_OIGNONS_BOARD_WIKI_PATH=/srv/ikiwiki/wiki-ca/src
+ SyslogIdentifier=send-member-emails-to-advisors
+ ProtectSystem=strict
+ ProtectHome=true
+ PrivateTmp=yes
+ PrivateDevices=yes
- Cmnd_Alias AG_MANAGEMENT = /usr/sbin/list_members ag,\
- /usr/sbin/add_members -r - ag,\
- /usr/sbin/remove_members -f - ag
+Dans `/etc/systemd/system/gestion-adh-send-member-emails-to-advisors.timer` :
- wiki-ca ALL = (list) NOPASSWD: AG_MANAGEMENT
+ [Unit]
+ Description=Run send-member-emails-to-advisors monthly
-Ensuite, pour exécuter régulièrement les scripts via le *crontab* du compte
-`wiki-ca`, il faut y ajouter :
+ [Timer]
+ OnCalendar=monthly
- 42 * * * * BUNDLE_GEMFILE=/srv/ikiwiki/wiki-ca/gestion-adh/Gemfile NOS_OIGNONS_BOARD_WIKI_PATH=/srv/ikiwiki/wiki-ca/src /srv/ikiwiki/wiki-ca/gestion-adh/bin/update-ag-subscribers
- 42 6 * * * BUNDLE_GEMFILE=/srv/ikiwiki/wiki-ca/gestion-adh/Gemfile NOS_OIGNONS_BOARD_WIKI_PATH=/srv/ikiwiki/wiki-ca/src /srv/ikiwiki/wiki-ca/gestion-adh/bin/send-membership-reminders
- 21 0 1 * * BUNDLE_GEMFILE=/srv/ikiwiki/wiki-ca/gestion-adh/Gemfile NOS_OIGNONS_BOARD_WIKI_PATH=/srv/ikiwiki/wiki-ca/src /srv/ikiwiki/wiki-ca/gestion-adh/bin/send-member-emails-to-advisors
+ [Install]
+ WantedBy=timers.target
Pour installer le `pre-commit` *hook* sur le dépôt utilisé par
Ikiwiki, on met dans `/srv/ikiwiki/wiki-ca/src/.git/hooks/pre-commit` :
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
+require 'webmock'
require 'json'
+require 'digest'
Before('@mailman') do
- @mock_mailman_db = expand_path('mock_mailman.json')
- init_mailman_mock_db({})
- ENV['MOCK_MAILMAN_DB'] = @mock_mailman_db
- @orig_path = ENV['PATH']
- ENV['PATH'] = "#{File.expand_path('../../support/mock_mailman', __FILE__)}:#{@orig_path}"
+ @ag_emails = []
+ stub_request(:any, /mailman.example.org/).
+ with(basic_auth: ['mailman_rest_user', 'mailman_rest_pass'])
+ stub_request(:get, 'https://mailman.example.org/3.0/lists/ag@nos-oignons.net/roster/member').
+ with(basic_auth: ['mailman_rest_user', 'mailman_rest_pass']).
+ to_return { |request| {body: roster_member_payload } }
end
-After('@mailman') do
- ENV['PATH'] = @orig_path
- FileUtils.remove_entry_secure @mock_mailman_db
+def member_id_for_email(email)
+ Digest::SHA1.hexdigest(email)
end
-def init_mailman_mock_db(dict)
- File.open(@mock_mailman_db, 'w') { |f| f.write(JSON.dump(dict)) }
+def roster_member_payload
+ # This is an approximation of the actual payload from Mailman, but good enough for our needs
+ payload = {
+ "total_size": @ag_emails.count,
+ "start": 0,
+ "entries": @ag_emails.map { |email| { "email": email, "member_id": member_id_for_email(email) } },
+ }
+ payload.to_json
end
-def mailman_mock_db
- JSON.load(File.open(@mock_mailman_db))
+Given /^une liste ag@ avec comme emails inscrits:$/ do |subscriber_list|
+ @ag_emails = subscriber_list.strip.split
end
-Given /^une liste ag@ avec comme emails inscrits:$/ do |subscriber_list|
- emails = subscriber_list.strip.split
- init_mailman_mock_db('ag' => emails)
+Then 'la liste ag@ doit avoir reçu l’inscription de {string}' do |email|
+ expect(
+ a_request(:post, "https://mailman.example.org/3.0/members").
+ with(headers: {'Content-Type': 'application/json'},
+ body: hash_including(
+ {"list_id": "ag.nos-oignons.net",
+ "subscriber": email,
+ "pre_verified": true,
+ "pre_confirmed": true,
+ "pre_approved": true,
+ }
+ ),
+ )).to have_been_made
+end
+
+Then 'la liste ag@ ne doit pas avoir reçu d’inscription' do
+ expect(
+ a_request(:post, "https://mailman.example.org/3.0/members")
+ ).not_to have_been_made
end
-Then /^la liste ag@ doit avoir comme emails inscrits:$/ do |expected|
- emails = expected.strip.split.sort
- expect(mailman_mock_db['ag'].sort).to eql(emails)
+Then 'la liste ag@ doit avoir reçu la désinscription de {string}' do |email|
+ expect(
+ a_request(:delete, "https://mailman.example.org/3.0/members/#{member_id_for_email(email)}")
+ ).to have_been_made
end
NosOignons::ReminderDb.instance.reload!
ENV['NOS_OIGNONS_RECEIPTS_DIR'] = expand_path('receipts')
FileUtils.mkdir(ENV['NOS_OIGNONS_RECEIPTS_DIR'])
+ ENV['NOS_OIGNONS_MAILMAN_REST_API_URL'] = 'https://mailman_rest_user:mailman_rest_pass@mailman.example.org'
Mail.defaults do
delivery_method :test
end
+++ /dev/null
-#!/usr/bin/ruby
-#-*- coding: utf-8 -*-
-#
-# Système de gestion des adhésions de Nos oignons
-# Copyright © 2013-2014 Nos oignons <contact@nos-oignons.net>
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Affero General Public License as
-# published by the Free Software Foundation, either version 3 of the
-# License, or (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU Affero General Public License for more details.
-#
-# You should have received a copy of the GNU Affero General Public License
-# along with this program. If not, see <http://www.gnu.org/licenses/>.
-
-require 'json'
-
-db = ENV['MOCK_MAILMAN_DB']
-
-lists = JSON.load(File.read(db))
-if ARGV[0] != '-r'
- $stderr.puts "Bad call"
- exit 1
-else
- emails = (ARGV[1] == '-' ? $stdin : File.open(ARGV[1])).read.split
- lists[ARGV[2]] = (lists[ARGV[2]] || []) + emails
-end
-File.open(db, 'w') { |f| f.write(JSON.dump(lists)) }
+++ /dev/null
-#!/usr/bin/ruby
-#-*- coding: utf-8 -*-
-#
-# Système de gestion des adhésions de Nos oignons
-# Copyright © 2013-2014 Nos oignons <contact@nos-oignons.net>
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Affero General Public License as
-# published by the Free Software Foundation, either version 3 of the
-# License, or (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU Affero General Public License for more details.
-#
-# You should have received a copy of the GNU Affero General Public License
-# along with this program. If not, see <http://www.gnu.org/licenses/>.
-
-require 'json'
-
-db = ENV['MOCK_MAILMAN_DB']
-
-lists = JSON.load(File.read(db))
-(lists[ARGV[0]] || []).each do |email|
- puts email
-end
+++ /dev/null
-#!/usr/bin/ruby
-#-*- coding: utf-8 -*-
-#
-# Système de gestion des adhésions de Nos oignons
-# Copyright © 2013-2014 Nos oignons <contact@nos-oignons.net>
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Affero General Public License as
-# published by the Free Software Foundation, either version 3 of the
-# License, or (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU Affero General Public License for more details.
-#
-# You should have received a copy of the GNU Affero General Public License
-# along with this program. If not, see <http://www.gnu.org/licenses/>.
-
-require 'json'
-
-db = ENV['MOCK_MAILMAN_DB']
-
-lists = JSON.load(File.read(db))
-if ARGV[0] != '-f'
- $stderr.puts "Bad call"
- exit 1
-else
- emails = (ARGV[1] == '-' ? $stdin : File.open(ARGV[1])).read.split
- lists[ARGV[2]] = (lists[ARGV[2]] || []) - emails
-end
-File.open(db, 'w') { |f| f.write(JSON.dump(lists)) }
+++ /dev/null
-#!/bin/sh
-#
-# Système de gestion des adhésions de Nos oignons
-# Copyright © 2013-2014 Nos oignons <contact@nos-oignons.net>
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Affero General Public License as
-# published by the Free Software Foundation, either version 3 of the
-# License, or (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU Affero General Public License for more details.
-#
-# You should have received a copy of the GNU Affero General Public License
-# along with this program. If not, see <http://www.gnu.org/licenses/>.
-
-if ! [ "-u" = "$1" ] && ! [ "list" = "$2" ]; then
- echo "Bad call" >&2
- exit 1
-fi
-
-shift 2
-
-exec "$@"
Et avec Jane, à jour de cotisation
Et avec Fatima, à jour de cotisation
Lorsque j'exécute update-ag-subscribers
- Alors la liste ag@ doit avoir comme emails inscrits:
- """
- pierre@example.org
- jane@example.org
- fatima@example.org
- """
+ Alors la liste ag@ ne doit pas avoir reçu d’inscription
Scénario: Nouvelle adhésion
Soit une liste ag@ avec comme emails inscrits:
Et avec Fatima, à jour de cotisation
Et une nouvelle adhésion de Sean
Lorsque j'exécute update-ag-subscribers
- Alors la liste ag@ doit avoir comme emails inscrits:
- """
- pierre@example.org
- jane@example.org
- fatima@example.org
- sean@example.org
- """
+ Alors la liste ag@ doit avoir reçu l’inscription de "sean@example.org"
Scénario: Non renouvellement de la cotisation
Soit une liste ag@ avec comme emails inscrits:
Et avec Jane qui n'a pas payé sa cotisation cette année
Et avec Fatima, à jour de cotisation
Lorsque j'exécute update-ag-subscribers
- Alors la liste ag@ doit avoir comme emails inscrits:
- """
- pierre@example.org
- fatima@example.org
- """
+ Alors la liste ag@ doit avoir reçu la désinscription de "jane@example.org"
Scénario: Un ajout et une suppression
Soit une liste ag@ avec comme emails inscrits:
Et avec Jane qui n'a pas payé sa cotisation cette année
Et une nouvelle adhésion de Sean
Lorsque j'exécute update-ag-subscribers
- Alors la liste ag@ doit avoir comme emails inscrits:
- """
- pierre@example.org
- sean@example.org
- """
+ Alors la liste ag@ doit avoir reçu l’inscription de "sean@example.org"
+ Et la liste ag@ doit avoir reçu la désinscription de "jane@example.org"
module NosOignons
BOARD_EMAIL = 'ca@nos-oignons.net'
ADVISORS_EMAIL = 'deontologie@nos-oignons.net'
- MEMBER_MAILING_LIST = 'ag'
+ MEMBER_MAILING_LIST = 'ag@nos-oignons.net'
CONTACT_INFO = <<-EOT.gsub(/^ /, '')
Identifiant SIREN 842 479 313
https://nos-oignons.net/
def update_ag_subscribers!
list = NosOignons::MEMBER_MAILING_LIST
- current_emails = NosOignons::Mailman.list_members(list)
+ current_members = NosOignons::Mailman.list_members(list)
uptodate_emails = NosOignons::Member.all.select(&:up_to_date?).collect(&:email)
- emails_to_add = uptodate_emails - current_emails
- NosOignons::Mailman.add_members(list, emails_to_add) unless emails_to_add.empty?
- emails_to_remove = current_emails - uptodate_emails
- NosOignons::Mailman.remove_members(list, emails_to_remove) unless emails_to_remove.empty?
+ # Who is not subscribed yet?
+ emails_to_add = uptodate_emails - current_members.collect(&:email)
+ emails_to_add.each do |email|
+ NosOignons::Mailman.susbcribe_email(list, email)
+ end
+
+ # Who should not be subscribed anymore?
+ current_members.each do |list_member|
+ unless uptodate_emails.include?(list_member.email)
+ list_member.unsubscribe!
+ end
+ end
end
def send_membership_reminders!
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-require 'shellwords'
+require 'json'
+require 'rest-client'
module NosOignons
module Mailman
+ class MailmanMember < Struct.new(:member_id, :email, keyword_init: true)
+ def unsubscribe!
+ Mailman::request(:delete, "/3.0/members/#{URI::encode_www_form_component(member_id)}")
+ end
+ end
+
class << self
def list_members(list)
- `sudo -u list list_members #{Shellwords.escape(list)}`.strip.split
+ s = request(:get, "/3.0/lists/#{URI::encode_www_form_component(list)}/roster/member")
+ r = JSON.parse(s)
+ r["entries"].collect { |entry|
+ MailmanMember.new(member_id: entry["member_id"], email: entry["email"])
+ }
end
- def add_member(list, email)
- add_members(list, [email])
+ def susbcribe_email(list, email)
+ list_id = list.gsub(/@/, '.')
+ request(:post, "/3.0/members", payload: {
+ list_id: list_id,
+ subscriber: email,
+ pre_verified: true,
+ pre_confirmed: true,
+ pre_approved: true,
+ }.to_json,
+ headers: { content_type: :json },
+ )
end
- def add_members(list, emails)
- IO.popen(['sudo', '-u', 'list', 'add_members', '-r', '-', list], 'w') do |io|
- emails.each do |email|
- io.puts email
- end
- end
+ def request(method, path, **args)
+ RestClient::Request.execute(
+ **args,
+ method: method,
+ url: "#{rest_url}#{path}"
+ )
end
- def remove_member(list, email)
- remove_members(list, [email])
- end
+ private
+
+ def rest_url
+ @rest_url if defined?(@rest_url)
- def remove_members(list, emails)
- IO.popen(['sudo', '-u', 'list', 'remove_members', '-f', '-', list], 'w') do |io|
- emails.each do |email|
- io.puts email
+ @rest_url = ENV["NOS_OIGNONS_MAILMAN_REST_API_URL"]
+ if @rest_url.nil?
+ cred_path = File.join("#{ENV["CREDENTIALS_DIRECTORY"]}", "mailman-rest-url")
+ begin
+ @rest_url = File.open(cred_path).read
+ rescue Errno::ENOENT
+ raise ArgumentError.new("Environment variable `NOS_OIGNONS_MAILMAN_REST_API_URL` not defined, and unable to read the file `$CREDENTIALS_DIRECTORY/mailman-rest-url` either")
end
end
+ @rest_url
end
end
end
s.add_development_dependency 'aruba'
s.add_development_dependency 'json'
s.add_development_dependency 'timecop'
+ s.add_development_dependency 'webmock'
s.add_runtime_dependency 'safe_yaml'
s.add_runtime_dependency 'mail'
s.add_runtime_dependency 'prawn'
+ s.add_runtime_dependency 'rest-client'
end
projects / gestion-adh.git / commitdiff