Switch to Mailman 3 REST API for “ag” subscriptions

We used to control subscriptions to the “ag“ mailing list
using Mailman 2 command line interface.

Now that we use Mailman 3, we need to switch to its REST API to do the
same. Most of what is useful to us is documented at:
https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/rest/docs/membership.html

The `rest-client` gem has been added to ease with implementing REST
requests.

A key difference from the old interface is that a subscription
creates a `member_id`, specific to the list and different from
the `user_id` or email address. Unsubscribing a user requires to
this `member_id`.

It is not a big issue as our process to update the list subscriptions
always start by listing all current subscriptions. We retrieve both
the `member_id` and email address at this stage. After comparing
to the list of known members, we can use the `member_id` to remove
outdated subscriptions.

The relevant Gherkin scenarios have been updated to be easier to test
with these new process but are functionally equivalent.

We also take the opportunity to move from crontab(5) to systemd.timer(5)
and systemd.service(5) to use systemd service credentials
to protect the password to Mailman REST API.

diff --git a/Gemfile.lock b/Gemfile.lock
index 6bf5cf2a8b28972db923a9d5e36e8347fcb6ea10..742e6fa779406ab2f6e9d77ee290ecdda48db164 100644 (file)
--- a/Gemfile.lock
+++ b/Gemfile.lock
@@ -4,11 +4,14 @@ PATH
     nos_oignons (0.0.1.dev)
       mail
       prawn
     nos_oignons (0.0.1.dev)
       mail
       prawn

+      rest-client

       safe_yaml
 
 GEM
   remote: https://rubygems.org/
   specs:
       safe_yaml
 
 GEM
   remote: https://rubygems.org/
   specs:

+    addressable (2.8.4)
+      public_suffix (>= 2.0.2, < 6.0)

     aruba (2.1.0)
       bundler (>= 1.17, < 3.0)
       childprocess (>= 2.0, < 5.0)
     aruba (2.1.0)
       bundler (>= 1.17, < 3.0)
       childprocess (>= 2.0, < 5.0)


@@ -19,6 +22,8 @@ GEM
     builder (3.2.4)
     childprocess (4.1.0)
     contracts (0.16.1)
     builder (3.2.4)
     childprocess (4.1.0)
     contracts (0.16.1)

+    crack (0.4.5)
+      rexml

     cucumber (8.0.0)
       builder (~> 3.2, >= 3.2.4)
       cucumber-ci-environment (~> 9.0, >= 9.0.4)
     cucumber (8.0.0)
       builder (~> 3.2, >= 3.2.4)
       cucumber-ci-environment (~> 9.0, >= 9.0.4)


@@ -45,7 +50,13 @@ GEM
     cucumber-tag-expressions (4.1.0)
     date (3.3.3)
     diff-lcs (1.5.0)
     cucumber-tag-expressions (4.1.0)
     date (3.3.3)
     diff-lcs (1.5.0)

+    domain_name (0.5.20190701)
+      unf (>= 0.0.5, < 1.0.0)

     ffi (1.15.5)
     ffi (1.15.5)

+    hashdiff (1.0.1)
+    http-accept (1.7.0)
+    http-cookie (1.0.5)
+      domain_name (~> 0.5)

     json (2.6.2)
     mail (2.8.1)
       mini_mime (>= 0.1.1)
     json (2.6.2)
     mail (2.8.1)
       mini_mime (>= 0.1.1)


@@ -67,10 +78,18 @@ GEM
       timeout
     net-smtp (0.3.3)
       net-protocol
       timeout
     net-smtp (0.3.3)
       net-protocol

+    netrc (0.11.0)

     pdf-core (0.9.0)
     prawn (2.4.0)
       pdf-core (~> 0.9.0)
       ttfunk (~> 1.7)
     pdf-core (0.9.0)
     prawn (2.4.0)
       pdf-core (~> 0.9.0)
       ttfunk (~> 1.7)

+    public_suffix (5.0.1)
+    rest-client (2.1.0)
+      http-accept (>= 1.7.0, < 2.0)
+      http-cookie (>= 1.0.2, < 2.0)
+      mime-types (>= 1.16, < 4.0)
+      netrc (~> 0.8)
+    rexml (3.2.5)

     rspec-expectations (3.11.0)
       diff-lcs (>= 1.2.0, < 2.0)
       rspec-support (~> 3.11.0)
     rspec-expectations (3.11.0)
       diff-lcs (>= 1.2.0, < 2.0)
       rspec-support (~> 3.11.0)


@@ -82,6 +101,13 @@ GEM
     timecop (0.9.5)
     timeout (0.3.2)
     ttfunk (1.7.0)
     timecop (0.9.5)
     timeout (0.3.2)
     ttfunk (1.7.0)

+    unf (0.1.4)
+      unf_ext
+    unf_ext (0.0.8.2)
+    webmock (3.18.1)
+      addressable (>= 2.8.0)
+      crack (>= 0.3.2)
+      hashdiff (>= 0.4.0, < 2.0.0)

 
 PLATFORMS
   ruby
 
 PLATFORMS
   ruby


@@ -93,6 +119,7 @@ DEPENDENCIES
   matrix (~> 0.4.2)
   nos_oignons!
   timecop
   matrix (~> 0.4.2)
   nos_oignons!
   timecop

+  webmock

 
 BUNDLED WITH
    2.2.5
 
 BUNDLED WITH
    2.2.5

diff --git a/README b/README
index 71c969fd061d39aa0e09b1451ed7ba1ccdc9d667..123a38ad591bc0d6749230ce815fb0efcd8fec1b 100644 (file)
--- a/README
+++ b/README
@@ -100,8 +100,16 @@ le dépôt central du wiki du C.A. (via un lien symbolique dans
 
 Met à jour la liste des adresses email inscrites à la liste *ag@* par rapport
 aux membres à jour de cotisation. À exécuter à travers un *cron*. A besoin de
 
 Met à jour la liste des adresses email inscrites à la liste *ag@* par rapport
 aux membres à jour de cotisation. À exécuter à travers un *cron*. A besoin de

-pouvoir lancer les commandes `list_members`, `add_members` et `remove_members`
-via `sudo` sur le compte `list`.
+pouvoir se connecter à l’API REST de Mailman 3.
+
+La variable d’environnement suivante doit être
+définie `NOS_OIGNONS_MAILMAN_REST_API_URL` avec par exemple :
+`https://user:password@mailman.example.org`
+
+À défaut, cette dernière sera lue depuis le fichier `mailman-rest-url`
+dans le dossier indiqué par la variable d’environnemnet `CREDENTIALS_DIRECTORY`
+(voir [System and Service Credentials](https://systemd.io/CREDENTIALS/) et
+la procédure de déploiement indiqué plus bas).

 
 `send-membership-reminders`
 ---------------------------
 
 `send-membership-reminders`
 ---------------------------


@@ -176,24 +184,101 @@ Pour rendre facilement accessible `list-members-emails`, on peut ajouter dans
 
 Ne pas oublier de le rendre exécutable.
 
 
 Ne pas oublier de le rendre exécutable.
 

-Pour permettre au script `update-ag-subscribers` de fonctionner, il est
-nécessaire de l'autoriser à exécuter certaines commandes de Mailman.
-Pour cela, on va créer un fichier dans `/etc/sudoers.d` :
+On va utiliser des
+[services](https://www.freedesktop.org/software/systemd/man/systemd.service.html)
+et des [timers](https://www.freedesktop.org/software/systemd/man/systemd.timer.html)
+systemd pour les scripts à utiliser régulièrement.
+
+Dans `/etc/systemd/system/gestion-adh-update-ag-subscribers.service` :
+
+    [Unit]
+    Description=Update subscribers of the `ag` mailing list according to memberships
+
+    [Service]
+    Type=oneshot
+    WorkingDirectory=/srv/ikiwiki/wiki-ca/gestion-adh
+    User=wiki-ca
+    ExecStart=/srv/ikiwiki/wiki-ca/gestion-adh/bin/update-ag-subscribers
+    Environment=BUNDLE_GEMFILE=/srv/ikiwiki/wiki-ca/gestion-adh/Gemfile NOS_OIGNONS_BOARD_WIKI_PATH=/srv/ikiwiki/wiki-ca/src
+    SyslogIdentifier=update-ag-subscribers
+    ProtectSystem=strict
+    ProtectHome=true
+    PrivateTmp=yes
+    PrivateDevices=yes
+
+Dans `/etc/systemd/system/gestion-adh-update-ag-subscribers.timer` :
+
+    [Unit]
+    Description=Run update-ag-subscribers every hour
+
+    [Timer]
+    RandomizedDelaySec=30min
+    OnCalendar=hourly
+
+    [Install]
+    WantedBy=timers.target
+
+Il est également nécessaire de configurer le *credential* avec les informations
+de connexion à Mailman :
+
+    echo -n 'https://USER:SECRET_PASSWORD@localhost:8001' | sudo systemd-creds encrypt --name=mailman-rest-url -p - - | sudo tee -a /etc/systemd/system/gestion-adh-update-subscribers.service.d/overrides.conf
+
+Dans `/etc/systemd/system/gestion-adh-send-membership-reminders.service` :
+
+    [Unit]
+    Description=Send reminders to renew membership
+
+    [Service]
+    Type=oneshot
+    WorkingDirectory=/srv/ikiwiki/wiki-ca/gestion-adh
+    User=wiki-ca
+    ExecStart=/srv/ikiwiki/wiki-ca/gestion-adh/bin/send-membership-reminders
+    Environment=BUNDLE_GEMFILE=/srv/ikiwiki/wiki-ca/gestion-adh/Gemfile NOS_OIGNONS_BOARD_WIKI_PATH=/srv/ikiwiki/wiki-ca/src
+    SyslogIdentifier=send-membership-reminders
+    ProtectSystem=strict
+    ProtectHome=true
+    PrivateTmp=yes
+    PrivateDevices=yes
+    ReadWritePaths=/srv/ikiwiki/wiki-ca/gestion-adh/var
+
+Dans `/etc/systemd/system/gestion-adh-send-membership-reminders.timer` :
+
+    [Unit]
+    Description=Run send-membership-reminders every day
+
+    [Timer]
+    OnCalendar=06:42
+
+    [Install]
+    WantedBy=timers.target
+
+Dans `/etc/systemd/system/gestion-adh-send-member-emails-to-advisors.service` :
+
+    [Unit]
+    Description=Send member emails to the advisory board

 
 

-    Defaults:wiki-ca !requiretty
+    [Service]
+    Type=oneshot
+    WorkingDirectory=/srv/ikiwiki/wiki-ca/gestion-adh
+    User=wiki-ca
+    ExecStart=/srv/ikiwiki/wiki-ca/gestion-adh/bin/send-member-emails-to-advisors
+    Environment=BUNDLE_GEMFILE=/srv/ikiwiki/wiki-ca/gestion-adh/Gemfile NOS_OIGNONS_BOARD_WIKI_PATH=/srv/ikiwiki/wiki-ca/src
+    SyslogIdentifier=send-member-emails-to-advisors
+    ProtectSystem=strict
+    ProtectHome=true
+    PrivateTmp=yes
+    PrivateDevices=yes

 
 

-    Cmnd_Alias AG_MANAGEMENT = /usr/sbin/list_members ag,\
-                               /usr/sbin/add_members -r - ag,\
-                               /usr/sbin/remove_members -f - ag
+Dans `/etc/systemd/system/gestion-adh-send-member-emails-to-advisors.timer` :

 
 

-    wiki-ca ALL = (list) NOPASSWD: AG_MANAGEMENT
+    [Unit]
+    Description=Run send-member-emails-to-advisors monthly

 
 

-Ensuite, pour exécuter régulièrement les scripts via le *crontab* du compte
-`wiki-ca`, il faut y ajouter :
+    [Timer]
+    OnCalendar=monthly

 
 

-    42 * * * * BUNDLE_GEMFILE=/srv/ikiwiki/wiki-ca/gestion-adh/Gemfile NOS_OIGNONS_BOARD_WIKI_PATH=/srv/ikiwiki/wiki-ca/src /srv/ikiwiki/wiki-ca/gestion-adh/bin/update-ag-subscribers
-    42 6 * * * BUNDLE_GEMFILE=/srv/ikiwiki/wiki-ca/gestion-adh/Gemfile NOS_OIGNONS_BOARD_WIKI_PATH=/srv/ikiwiki/wiki-ca/src /srv/ikiwiki/wiki-ca/gestion-adh/bin/send-membership-reminders
-    21 0 1 * * BUNDLE_GEMFILE=/srv/ikiwiki/wiki-ca/gestion-adh/Gemfile NOS_OIGNONS_BOARD_WIKI_PATH=/srv/ikiwiki/wiki-ca/src /srv/ikiwiki/wiki-ca/gestion-adh/bin/send-member-emails-to-advisors
+    [Install]
+    WantedBy=timers.target

 
 Pour installer le `pre-commit` *hook* sur le dépôt utilisé par
 Ikiwiki, on met dans `/srv/ikiwiki/wiki-ca/src/.git/hooks/pre-commit` :
 
 Pour installer le `pre-commit` *hook* sur le dépôt utilisé par
 Ikiwiki, on met dans `/srv/ikiwiki/wiki-ca/src/.git/hooks/pre-commit` :

diff --git a/features/step_definitions/mailman.rb b/features/step_definitions/mailman.rb
index e201bab96bc9591f8a6f306db9a242846387702a..c3c36cf3ba59be95bc0047f7b5e1a3fb8bf87e04 100644 (file)
--- a/features/step_definitions/mailman.rb
+++ b/features/step_definitions/mailman.rb
@@ -16,36 +16,61 @@
 # You should have received a copy of the GNU Affero General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 # You should have received a copy of the GNU Affero General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 

+require 'webmock'

 require 'json'
 require 'json'

+require 'digest'

 
 Before('@mailman') do
 
 Before('@mailman') do

-  @mock_mailman_db = expand_path('mock_mailman.json')
-  init_mailman_mock_db({})
-  ENV['MOCK_MAILMAN_DB'] = @mock_mailman_db
-  @orig_path = ENV['PATH']
-  ENV['PATH'] = "#{File.expand_path('../../support/mock_mailman', __FILE__)}:#{@orig_path}"
+  @ag_emails = []
+  stub_request(:any, /mailman.example.org/).
+    with(basic_auth: ['mailman_rest_user', 'mailman_rest_pass'])
+  stub_request(:get, 'https://mailman.example.org/3.0/lists/ag@nos-oignons.net/roster/member').
+    with(basic_auth: ['mailman_rest_user', 'mailman_rest_pass']).
+    to_return { |request| {body: roster_member_payload } }

 end
 
 end
 

-After('@mailman') do
-  ENV['PATH'] = @orig_path
-  FileUtils.remove_entry_secure @mock_mailman_db
+def member_id_for_email(email)
+  Digest::SHA1.hexdigest(email)

 end
 
 end
 

-def init_mailman_mock_db(dict)
-  File.open(@mock_mailman_db, 'w') { |f| f.write(JSON.dump(dict)) }
+def roster_member_payload
+  # This is an approximation of the actual payload from Mailman, but good enough for our needs
+  payload = {
+    "total_size": @ag_emails.count,
+    "start": 0,
+    "entries": @ag_emails.map { |email| { "email": email, "member_id": member_id_for_email(email) } },
+  }
+  payload.to_json

 end
 
 end
 

-def mailman_mock_db
-  JSON.load(File.open(@mock_mailman_db))
+Given /^une liste ag@ avec comme emails inscrits:$/ do |subscriber_list|
+  @ag_emails = subscriber_list.strip.split

 end
 
 end
 

-Given /^une liste ag@ avec comme emails inscrits:$/ do |subscriber_list|
-  emails = subscriber_list.strip.split
-  init_mailman_mock_db('ag' => emails)
+Then 'la liste ag@ doit avoir reçu l’inscription de {string}' do |email|
+  expect(
+    a_request(:post, "https://mailman.example.org/3.0/members").
+      with(headers: {'Content-Type': 'application/json'},
+           body: hash_including(
+             {"list_id": "ag.nos-oignons.net",
+              "subscriber": email,
+              "pre_verified": true,
+              "pre_confirmed": true,
+              "pre_approved": true,
+             }
+           ),
+          )).to have_been_made
+end
+
+Then 'la liste ag@ ne doit pas avoir reçu d’inscription' do
+  expect(
+    a_request(:post, "https://mailman.example.org/3.0/members")
+  ).not_to have_been_made

 end
 
 end
 

-Then /^la liste ag@ doit avoir comme emails inscrits:$/ do |expected|
-  emails = expected.strip.split.sort
-  expect(mailman_mock_db['ag'].sort).to eql(emails)
+Then 'la liste ag@ doit avoir reçu la désinscription de {string}' do |email|
+  expect(
+    a_request(:delete, "https://mailman.example.org/3.0/members/#{member_id_for_email(email)}")
+  ).to have_been_made

 end
 
 end
 

diff --git a/features/support/env.rb b/features/support/env.rb
index ae8c5c3a1fe9a49ae58437715edf06db6631f5b2..4893603c31f28098e12196fa3a0648cae3977a88 100644 (file)
--- a/features/support/env.rb
+++ b/features/support/env.rb
@@ -45,6 +45,7 @@ Before do
   NosOignons::ReminderDb.instance.reload!
   ENV['NOS_OIGNONS_RECEIPTS_DIR'] = expand_path('receipts')
   FileUtils.mkdir(ENV['NOS_OIGNONS_RECEIPTS_DIR'])
   NosOignons::ReminderDb.instance.reload!
   ENV['NOS_OIGNONS_RECEIPTS_DIR'] = expand_path('receipts')
   FileUtils.mkdir(ENV['NOS_OIGNONS_RECEIPTS_DIR'])

+  ENV['NOS_OIGNONS_MAILMAN_REST_API_URL'] = 'https://mailman_rest_user:mailman_rest_pass@mailman.example.org'

   Mail.defaults do
     delivery_method :test
   end
   Mail.defaults do
     delivery_method :test
   end

diff --git a/features/support/mock_mailman/add_members b/features/support/mock_mailman/add_members
deleted file mode 100755 (executable)
index d6dd3ba..0000000
--- a/features/support/mock_mailman/add_members
+++ /dev/null
@@ -1,32 +0,0 @@
-#!/usr/bin/ruby
-#-*- coding: utf-8 -*-
-#
-# Système de gestion des adhésions de Nos oignons
-# Copyright © 2013-2014 Nos oignons <contact@nos-oignons.net>
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Affero General Public License as
-# published by the Free Software Foundation, either version 3 of the
-# License, or (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Affero General Public License for more details.
-#
-# You should have received a copy of the GNU Affero General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
-
-require 'json'
-
-db = ENV['MOCK_MAILMAN_DB']
-
-lists = JSON.load(File.read(db))
-if ARGV[0] != '-r'
-  $stderr.puts "Bad call"
-  exit 1
-else
-  emails = (ARGV[1] == '-' ? $stdin : File.open(ARGV[1])).read.split
-  lists[ARGV[2]] = (lists[ARGV[2]] || []) + emails
-end
-File.open(db, 'w') { |f| f.write(JSON.dump(lists)) }

diff --git a/features/support/mock_mailman/list_members b/features/support/mock_mailman/list_members
deleted file mode 100755 (executable)
index 7550f78..0000000
--- a/features/support/mock_mailman/list_members
+++ /dev/null
@@ -1,27 +0,0 @@
-#!/usr/bin/ruby
-#-*- coding: utf-8 -*-
-#
-# Système de gestion des adhésions de Nos oignons
-# Copyright © 2013-2014 Nos oignons <contact@nos-oignons.net>
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Affero General Public License as
-# published by the Free Software Foundation, either version 3 of the
-# License, or (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Affero General Public License for more details.
-#
-# You should have received a copy of the GNU Affero General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
-
-require 'json'
-
-db = ENV['MOCK_MAILMAN_DB']
-
-lists = JSON.load(File.read(db))
-(lists[ARGV[0]] || []).each do |email|
-  puts email
-end

diff --git a/features/support/mock_mailman/remove_members b/features/support/mock_mailman/remove_members
deleted file mode 100755 (executable)
index 063d7b3..0000000
--- a/features/support/mock_mailman/remove_members
+++ /dev/null
@@ -1,32 +0,0 @@
-#!/usr/bin/ruby
-#-*- coding: utf-8 -*-
-#
-# Système de gestion des adhésions de Nos oignons
-# Copyright © 2013-2014 Nos oignons <contact@nos-oignons.net>
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Affero General Public License as
-# published by the Free Software Foundation, either version 3 of the
-# License, or (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Affero General Public License for more details.
-#
-# You should have received a copy of the GNU Affero General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
-
-require 'json'
-
-db = ENV['MOCK_MAILMAN_DB']
-
-lists = JSON.load(File.read(db))
-if ARGV[0] != '-f'
-  $stderr.puts "Bad call"
-  exit 1
-else
-  emails = (ARGV[1] == '-' ? $stdin : File.open(ARGV[1])).read.split
-  lists[ARGV[2]] = (lists[ARGV[2]] || []) - emails
-end
-File.open(db, 'w') { |f| f.write(JSON.dump(lists)) }

diff --git a/features/support/mock_mailman/sudo b/features/support/mock_mailman/sudo
deleted file mode 100755 (executable)
index 8fcbc72..0000000
--- a/features/support/mock_mailman/sudo
+++ /dev/null
@@ -1,26 +0,0 @@
-#!/bin/sh
-#
-# Système de gestion des adhésions de Nos oignons
-# Copyright © 2013-2014 Nos oignons <contact@nos-oignons.net>
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Affero General Public License as
-# published by the Free Software Foundation, either version 3 of the
-# License, or (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Affero General Public License for more details.
-#
-# You should have received a copy of the GNU Affero General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
-
-if ! [ "-u" = "$1" ] && ! [ "list" = "$2" ]; then
-       echo "Bad call" >&2
-       exit 1
-fi
-
-shift 2
-
-exec "$@"

diff --git a/features/update-ag-subscribers.feature b/features/update-ag-subscribers.feature
index 4ee9725232d2145d032ef8e22db0b191cd0a7cd3..86f74c103bdab9bbb1cc167647ee2af3d8506ac9 100644 (file)
--- a/features/update-ag-subscribers.feature
+++ b/features/update-ag-subscribers.feature
@@ -17,12 +17,7 @@ Fonctionnalité: mettre à jour les emails inscrites à la liste ag@
     Et avec Jane, à jour de cotisation
     Et avec Fatima, à jour de cotisation
     Lorsque j'exécute update-ag-subscribers
     Et avec Jane, à jour de cotisation
     Et avec Fatima, à jour de cotisation
     Lorsque j'exécute update-ag-subscribers

-    Alors la liste ag@ doit avoir comme emails inscrits:
-      """
-      pierre@example.org
-      jane@example.org
-      fatima@example.org
-      """
+    Alors la liste ag@ ne doit pas avoir reçu d’inscription

 
   Scénario: Nouvelle adhésion
     Soit une liste ag@ avec comme emails inscrits:
 
   Scénario: Nouvelle adhésion
     Soit une liste ag@ avec comme emails inscrits:


@@ -36,13 +31,7 @@ Fonctionnalité: mettre à jour les emails inscrites à la liste ag@
     Et avec Fatima, à jour de cotisation
     Et une nouvelle adhésion de Sean
     Lorsque j'exécute update-ag-subscribers
     Et avec Fatima, à jour de cotisation
     Et une nouvelle adhésion de Sean
     Lorsque j'exécute update-ag-subscribers

-    Alors la liste ag@ doit avoir comme emails inscrits:
-      """
-      pierre@example.org
-      jane@example.org
-      fatima@example.org
-      sean@example.org
-      """
+    Alors la liste ag@ doit avoir reçu l’inscription de "sean@example.org"

 
   Scénario: Non renouvellement de la cotisation
     Soit une liste ag@ avec comme emails inscrits:
 
   Scénario: Non renouvellement de la cotisation
     Soit une liste ag@ avec comme emails inscrits:


@@ -55,11 +44,7 @@ Fonctionnalité: mettre à jour les emails inscrites à la liste ag@
     Et avec Jane qui n'a pas payé sa cotisation cette année
     Et avec Fatima, à jour de cotisation
     Lorsque j'exécute update-ag-subscribers
     Et avec Jane qui n'a pas payé sa cotisation cette année
     Et avec Fatima, à jour de cotisation
     Lorsque j'exécute update-ag-subscribers

-    Alors la liste ag@ doit avoir comme emails inscrits:
-      """
-      pierre@example.org
-      fatima@example.org
-      """
+    Alors la liste ag@ doit avoir reçu la désinscription de "jane@example.org"

 
   Scénario: Un ajout et une suppression
     Soit une liste ag@ avec comme emails inscrits:
 
   Scénario: Un ajout et une suppression
     Soit une liste ag@ avec comme emails inscrits:


@@ -71,8 +56,5 @@ Fonctionnalité: mettre à jour les emails inscrites à la liste ag@
     Et avec Jane qui n'a pas payé sa cotisation cette année
     Et une nouvelle adhésion de Sean
     Lorsque j'exécute update-ag-subscribers
     Et avec Jane qui n'a pas payé sa cotisation cette année
     Et une nouvelle adhésion de Sean
     Lorsque j'exécute update-ag-subscribers

-    Alors la liste ag@ doit avoir comme emails inscrits:
-      """
-      pierre@example.org
-      sean@example.org
-      """
+    Alors la liste ag@ doit avoir reçu l’inscription de "sean@example.org"
+    Et la liste ag@ doit avoir reçu la désinscription de "jane@example.org"

diff --git a/lib/nos_oignons.rb b/lib/nos_oignons.rb
index ef1ac5eed892449be0367bd66d300a873ace762c..ee6fe7ebd2fc0a9ae8af14d6021f40f7c345f0bf 100644 (file)
--- a/lib/nos_oignons.rb
+++ b/lib/nos_oignons.rb
@@ -25,7 +25,7 @@ require 'nos_oignons/reminder_db'
 module NosOignons
   BOARD_EMAIL = 'ca@nos-oignons.net'
   ADVISORS_EMAIL = 'deontologie@nos-oignons.net'
 module NosOignons
   BOARD_EMAIL = 'ca@nos-oignons.net'
   ADVISORS_EMAIL = 'deontologie@nos-oignons.net'

-  MEMBER_MAILING_LIST = 'ag'
+  MEMBER_MAILING_LIST = 'ag@nos-oignons.net'

   CONTACT_INFO = <<-EOT.gsub(/^    /, '')
     Identifiant SIREN 842 479 313
     https://nos-oignons.net/
   CONTACT_INFO = <<-EOT.gsub(/^    /, '')
     Identifiant SIREN 842 479 313
     https://nos-oignons.net/


@@ -58,13 +58,21 @@ module NosOignons
     def update_ag_subscribers!
       list = NosOignons::MEMBER_MAILING_LIST
 
     def update_ag_subscribers!
       list = NosOignons::MEMBER_MAILING_LIST
 

-      current_emails = NosOignons::Mailman.list_members(list)
+      current_members = NosOignons::Mailman.list_members(list)

       uptodate_emails = NosOignons::Member.all.select(&:up_to_date?).collect(&:email)
 
       uptodate_emails = NosOignons::Member.all.select(&:up_to_date?).collect(&:email)
 

-      emails_to_add = uptodate_emails - current_emails
-      NosOignons::Mailman.add_members(list, emails_to_add) unless emails_to_add.empty?
-      emails_to_remove = current_emails - uptodate_emails
-      NosOignons::Mailman.remove_members(list, emails_to_remove) unless emails_to_remove.empty?
+      # Who is not subscribed yet?
+      emails_to_add = uptodate_emails - current_members.collect(&:email)
+      emails_to_add.each do |email|
+        NosOignons::Mailman.susbcribe_email(list, email)
+      end
+
+      # Who should not be subscribed anymore?
+      current_members.each do |list_member|
+        unless uptodate_emails.include?(list_member.email)
+          list_member.unsubscribe!
+        end
+      end

     end
 
     def send_membership_reminders!
     end
 
     def send_membership_reminders!

diff --git a/lib/nos_oignons/mailman.rb b/lib/nos_oignons/mailman.rb
index 352b9689bbf258071b386866280d3afb945d05e9..007fd337183bb581c6508b467b4629d03581df39 100644 (file)
--- a/lib/nos_oignons/mailman.rb
+++ b/lib/nos_oignons/mailman.rb
@@ -16,37 +16,62 @@
 # You should have received a copy of the GNU Affero General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 # You should have received a copy of the GNU Affero General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 

-require 'shellwords'
+require 'json'
+require 'rest-client'

 
 module NosOignons
   module Mailman
 
 module NosOignons
   module Mailman

+    class MailmanMember < Struct.new(:member_id, :email, keyword_init: true)
+      def unsubscribe!
+        Mailman::request(:delete, "/3.0/members/#{URI::encode_www_form_component(member_id)}")
+      end
+    end
+

     class << self
       def list_members(list)
     class << self
       def list_members(list)

-        `sudo -u list list_members #{Shellwords.escape(list)}`.strip.split
+        s = request(:get, "/3.0/lists/#{URI::encode_www_form_component(list)}/roster/member")
+        r = JSON.parse(s)
+        r["entries"].collect { |entry|
+            MailmanMember.new(member_id: entry["member_id"], email: entry["email"])
+          }

       end
 
       end
 

-      def add_member(list, email)
-        add_members(list, [email])
+      def susbcribe_email(list, email)
+        list_id = list.gsub(/@/, '.')
+        request(:post, "/3.0/members", payload: {
+              list_id: list_id,
+              subscriber: email,
+              pre_verified: true,
+              pre_confirmed: true,
+              pre_approved: true,
+            }.to_json,
+            headers: { content_type: :json },
+          )

       end
 
       end
 

-      def add_members(list, emails)
-        IO.popen(['sudo', '-u', 'list', 'add_members', '-r', '-', list], 'w') do |io|
-          emails.each do |email|
-            io.puts email
-          end
-        end
+      def request(method, path, **args)
+        RestClient::Request.execute(
+          **args,
+          method: method,
+          url: "#{rest_url}#{path}"
+        )

       end
 
       end
 

-      def remove_member(list, email)
-        remove_members(list, [email])
-      end
+      private
+
+      def rest_url
+        @rest_url if defined?(@rest_url)

 
 

-      def remove_members(list, emails)
-        IO.popen(['sudo', '-u', 'list', 'remove_members', '-f', '-', list], 'w') do |io|
-          emails.each do |email|
-            io.puts email
+        @rest_url = ENV["NOS_OIGNONS_MAILMAN_REST_API_URL"]
+        if @rest_url.nil?
+          cred_path = File.join("#{ENV["CREDENTIALS_DIRECTORY"]}", "mailman-rest-url")
+          begin
+            @rest_url = File.open(cred_path).read
+          rescue Errno::ENOENT
+            raise ArgumentError.new("Environment variable `NOS_OIGNONS_MAILMAN_REST_API_URL` not defined, and unable to read the file `$CREDENTIALS_DIRECTORY/mailman-rest-url` either")

           end
         end
           end
         end

+        @rest_url

       end
     end
   end
       end
     end
   end

diff --git a/nos_oignons.gemspec b/nos_oignons.gemspec
index c174de22a76b18f260cbbf577a793f76b33b212c..f8cc0184e393f21f20bc719db58d1c2db5dae615 100644 (file)
--- a/nos_oignons.gemspec
+++ b/nos_oignons.gemspec
@@ -28,7 +28,9 @@ Gem::Specification.new do |s|
   s.add_development_dependency 'aruba'
   s.add_development_dependency 'json'
   s.add_development_dependency 'timecop'
   s.add_development_dependency 'aruba'
   s.add_development_dependency 'json'
   s.add_development_dependency 'timecop'

+  s.add_development_dependency 'webmock'

   s.add_runtime_dependency 'safe_yaml'
   s.add_runtime_dependency 'mail'
   s.add_runtime_dependency 'prawn'
   s.add_runtime_dependency 'safe_yaml'
   s.add_runtime_dependency 'mail'
   s.add_runtime_dependency 'prawn'

+  s.add_runtime_dependency 'rest-client'

 end
 end