SvnReporter: a Subversion commit reporter

Project:SvnReporter
Version:0.4 (2005.12.03)
Author:Remy Blank
License:GPLv2
URL:http://c-space.org/software/SvnReporter.html
Download:http://c-space.org/download/SvnReporter/SvnReporter-0.4.tar.gz

Description

SvnReporter generates various reports in response to commits happening in a Subversion repository. It is intended to be called from the post-commit hook.

Two types of reports are supported: single-commit and commit list reports. The former generate reports relative to a specific revision only, and are typically used to generate post-commit mails. The latter generate reports relative to a list of commits, e.g. an RSS feed or a web page showing the latest commits.

Reports can be restricted to commits whose objects match certain criteria, specified by a list of regular expressions. The format of the reports can be defined freely and flexibly using a very simple template format. A few example templates are included in the distribution.

Installation

SvnReporter is released under the GNU General Public License, version 2

The latest version can be downloaded here. It has been tested on Linux, but AFAIK there's not reason why it shouldn't run on other platforms.

SvnReporter requires at least Python 2.4, and provides an installer based on distutils. This means that installation to the default location (/usr) is as simple as:

python setup.py install

To install SvnReporter to another location, e.g. /usr/local, enter:

python setup.py install --prefix=/usr/local

The folder doc/examples contains a sample configuration file with its associated Python module and three templates.

SvnReporter.ini Configuration file
Imports.py Function importer module
mail.txt multipart/alternative e-mail template
atom.xml Atom feed template
rss.xml RSS feed template

Usage

Basic usage information can be displayed with SvnReporter --help:

$ python SvnReporter --help
usage: SvnReporter.py [options] config repos_path [revision]

SvnReporter generates various reports in response to commits happening in a
Subversion repository. It is intended to be called from the  post-commit hook.

options:
  --version   show program's version number and exit
  -h, --help  show this help message and exit

The command line arguments are as follows:

config Path to the configuration file
repos_path Path to the Subversion repository
revision Revision for which reports must be generated, or youngest revision if missing

SvnReporter is meant to be called from a post-commit hook, which gets both repos_path and revision as command-line arguments. A typical post-commit hook on a *nix system could look as follows (it assumes that the configuration file is located in a SvnReporter sub-folder of the hooks folder):

#!/bin/sh
REPOS="$1"
REV="$2"
cd "${REPOS}/hooks/SvnReporter"
SvnReporter SvnReporter.conf "${REPOS}" "${REV}"    

Configuration file format

The configuration file defines an arbitrary number of sections, each specifying the reports to generate for commits matching certain conditions. A section starts with a name between square brackets (e.g. [mySection]) followed by any number of key / value pairs (variable definitions) separated by an equal sign or a colon (e.g. entries=10 or entries: 10). Values can span multiple lines, where the following lines must be indented by whitespace. A complete description of the configuration file structure can be found in the ConfigParser documentation.

Every section should define at least the following special variables, either directly or as default values through the [DEFAULT] section:

exclude

Defines an optional list of regular expressions to exclude commit objects. Every regular expression must be on a separate line. For a section to match, at least one of the commit objects must match one of the match regular expressions and not match any of the exclude regular expressions. Only commits matching the section are processed by the generators.

Excluding temporary files

exclude = .*\.tmp
generators

Defines a list of report generators for the section. Every generator will create one report. Each generator gets its parameters enclosed in parentheses after the generator name. The generators are separated by commas. (Actually, the whole expression is placed between square brackets [] and evaluated as a Python expression.)

One mail and one RSS feed generator

generators = 
    Mail("mail.txt", "host.domain", "svn@host.domain", "joe@nospam.com"),
    Feed("rss.xml", "/var/www/htdocs/svn.rss", 10, 30, 200),
match

Defines a list of regular expressions to match commit objects. Every regular expression must be on a separate line. For a section to match, at least one of the commit objects must match one of the match regular expressions and not match any of the exclude regular expressions. Only commits matching the section are processed by the generators.

Match everything below Projects/MyProject/ and Libraries/MyLib/

match =
    Projects/MyProject/.*
    Libraries/MyLib/.*
module

Defines an optional Python module to import for expression name resolution. This allows importing and/or adding new functions to be used in string substitution expressions. Every section gets its own copy of the module.

Import module Import.py into locals

module = Import.py

Moreover, the [DEFAULT] section can define the following variable:

cache

Defines a location where SvnReporter will store cached data extracted from Subversion to persist across runs, so that it doesn't have to access the repository for data accessed in a previous run. If this variable is not specified, caching will still be done on a per-run basis, but will not be persisted across runs.

Store cache in /var/cache/SvnReporter/cache.pickle

cache = /var/cache/SvnReporter/cache.pickle

A sample configuration file (a commented version is included in doc/examples)

[DEFAULT]
host = server.mydomain.com
path = rc-feeds/%(__name__)s
viewVcBase = http://%(host)s/rc
viewVcRoot = svn

cache = cache.pickle
module = Imports.py
generators =
        Mail("mail.txt", %(repr(host))s, %(repr(mailFrom))s, %(repr(mailTo))s),
        Feed("rss.xml", "/var/www/localhost/htdocs/%(path)s.rss", %(entries)s, %(maxDays)s, %(maxDepth)s),
mailFrom = svn@%(host)s
entries = 10
maxDays = None
maxDepth = None

feedTitle = Commits on %(__name__)s
feedLink = http://%(host)s/%(path)s
entryTitle = Revision %(revision.number)d
entryLink = %(viewVcBase)s?root=%(quote_plus(viewVcRoot))s&rev=%(revision.number)d&view=rev

revLink = %(viewVcRevisionLink(viewVcBase, revision.number, viewVcRoot))s
objectLink = %(viewVcObjectLink(viewVcBase, revision.number, change, viewVcRoot))s
diffLink = %(viewVcDiffLink(viewVcBase, revision.number, change, "(diff to previous)", viewVcRoot))s

[All]
feedTitle = All commits except private
match = .*
exclude = private/.*
mailTo =
        admin@mydomain.com
        manager@mydomain.com

[SvnReporter]
match = dev/SvnReporter/.*
mailTo =
        user2@mydomain.com
        client@mydomain.com

Generators

Generators use a template to process data coming from Subversion, generate an output report, and send it somewhere. There are currently two types of generators:

Mail generator

The Mail generator is a single-commit generator that sends its output to an SMTP server. The template is expected to generate an RFC 2822-compliant message.

The constructor signature is the following:

Mail(template, mailer, mailFrom, recipients)

Where:

template Path to the template file
mailer Host name or IP address of the SMTP server
mailFrom Address of the sender of the message, will be used in the MAIL FROM command to the SMTP server
recipients List of recipients, separated by newlines

Mail generators with identical parameters (same template, mailer and mailFrom) for different sections combine their recipient lists. This prevents receiving a commit mail more than once if a commit matches several sections and a particular recipient is specified in more than one of the matching sections.

Mail generators provice an additional variable __names__, a Python list containing the names of the sections for which a mail is generated.

Feed generator

The Feed generator is a generic commit list generator that writes its output to a file. The format of the template is completely free. This allows generating a wide variety of reports: Atom feeds, RSS feeds, commit summary web pages, text files, etc.

The constructor signature is the following:

Feed(template, destination, entries=None, maxDays=None, maxDepth=None)

Where:

template Path to the template file
destination Path to the output file
entries Maximum number of entries in the generated report
maxDays Stop iteration over the revision loop if revisions become older than maxDays days
maxDepth Stop iteration over the revision loop at revision (youngest - maxDepth).

Template file format

Every generator takes a template file name as its first argument. The template describes what must be generated. Two mechanisms allow customizing what is generated: string substitutions and foreach loops. Everything else is taken as-is into the output file.

String substitutions

The string substitution mechanism is based on Python's string formatting operator % using mapping keys, but extends it to allow any Python expression as the mapping key. The identifiers used in the expression are looked up in the following order:

  1. The content structs provided by Subversion data (repository, revision, change), and foreach loop counters
  2. The section of the configuration file for which content is being generated
  3. The [DEFAULT] section in the configuration file
  4. The module imported with the module variable in the current section

The variable __name__ always contains the name of the current configuration file section, i.e. the section for which output is currently being generated.

Substitute the value of the name myVar defined in the configuration file

%(myVar)s

Substitute the current revision number minus one

%(revision.number - 1)d

Substitute the escaped content of htmlContent (escape must have been imported from xml.sax.saxutils)

%(escape(htmlContent))s

Substitute a star (*) if the content of an object has changed, and nothing if not

%({False: "", True: "*"}[change.textChanged])s

foreach loops

Every generator defines a certain number of loops related to lists of data coming from Subversion. A foreach loop defines a portion of the template that will be repeated once for every item in the list, and inserted at the location where the loop is defined. A foreach loop starts with <?foreach name?> and ends with <?end name?>, where "name" is the name of the loop.

Loops can be nested, and can appear more than once in a template. However, some nestings are rigidly defined. For example, a "change" loop of a Feed template must be contained within a revision loop, as every revision defines a new list of changes.

Every loop defines a struct variable with the same name as the loop, and containing data for the current loop item. The data can be accessed by specifying the loop variable name and the attribute separated by a period (.). Moreover, the loop defines a loop counter whose name is {name}Count (e.g. revisionCount or changeCount). The following variables are available:

repository

This variable is always available, and provides data about the Subversion repository.

uuid [str]
UUID of the repository
youngest [int]
Latest revision number
revision

This variable is always available for single-commit generators, and in revision loops for commit list generators. It provides data about a revision. The generators loop over revisions in reverse order, i.e. from youngest to oldest revision.

number [int]
Number of the revision
author [str]
Commit author of the revision
date [str]
Date when the revision was committed
log [str]
Log message of the revision
change

This variable is available in change loops, and provides data about individual changes in a revision. The generators loop over changes in alphabetical order of path.

path [str]
Path of the changed object
action [str]
Action performed, an be one of "added", "copied", "deleted" or "modified"
propChanged [bool]
True iff properties of the object have changed
textChanged [bool]
True iff the content of the object has changed
changedStr [str]
String showing a summary of the change types, similar to what is shown by svn status
basePath [str]
Path to the source if the object was copied
baseRevision [int]
Revision number of the source if the object was copied

Generate a list of all commit object paths, each on a separate line

The following objects have changed:
<?foreach change?>  %(change.path)s
<?end change?>

This template might generate the following output:

The following objects have changed:
  Project/file1.c
  Project/file2.c
  Project/ChangelLog

Generate a list of revisions with the paths that have been changed, prefixed with the change action (added, copied, deleted, modified)

Latest revisions:
-----------------
<?foreach revision?>Revision %(revision.number)d
<?foreach change?>  %(change.action)8s  %(change.path)s
<?end change?>
<?end revision?>

This template might generate the following output:

Latest revisions:
-----------------
Revision 72
  modified  Project/file1.c
  added     Project/file2.c
  
Revision 71
  copied    Project/file1.c
  deleted   Project/oldfile.c

Frequently asked questions

To do

The following features will be added to SvnReporter as time permits.

Feedback

If you are using or trying to use SvnReporter, I would be happy to hear from you! I'm especially interested in the following:

In any case, just drop me an e-mail.

Links