Initial commit.

This commit is contained in:
2021-05-24 22:18:33 +03:00
commit e2954d55f4
3701 changed files with 330017 additions and 0 deletions

View File

@@ -0,0 +1,3 @@
ChangeLog
build
dist

View File

@@ -0,0 +1,340 @@
GNU GENERAL PUBLIC LICENSE
Version 2, June 1991
Copyright (C) 1989, 1991 Free Software Foundation, Inc.
59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The licenses for most software are designed to take away your
freedom to share and change it. By contrast, the GNU General Public
License is intended to guarantee your freedom to share and change free
software--to make sure the software is free for all its users. This
General Public License applies to most of the Free Software
Foundation's software and to any other program whose authors commit to
using it. (Some other Free Software Foundation software is covered by
the GNU Library General Public License instead.) You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
this service if you wish), that you receive source code or can get it
if you want it, that you can change the software or use pieces of it
in new free programs; and that you know you can do these things.
To protect your rights, we need to make restrictions that forbid
anyone to deny you these rights or to ask you to surrender the rights.
These restrictions translate to certain responsibilities for you if you
distribute copies of the software, or if you modify it.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must give the recipients all the rights that
you have. You must make sure that they, too, receive or can get the
source code. And you must show them these terms so they know their
rights.
We protect your rights with two steps: (1) copyright the software, and
(2) offer you this license which gives you legal permission to copy,
distribute and/or modify the software.
Also, for each author's protection and ours, we want to make certain
that everyone understands that there is no warranty for this free
software. If the software is modified by someone else and passed on, we
want its recipients to know that what they have is not the original, so
that any problems introduced by others will not reflect on the original
authors' reputations.
Finally, any free program is threatened constantly by software
patents. We wish to avoid the danger that redistributors of a free
program will individually obtain patent licenses, in effect making the
program proprietary. To prevent this, we have made it clear that any
patent must be licensed for everyone's free use or not licensed at all.
The precise terms and conditions for copying, distribution and
modification follow.
GNU GENERAL PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
0. This License applies to any program or other work which contains
a notice placed by the copyright holder saying it may be distributed
under the terms of this General Public License. The "Program", below,
refers to any such program or work, and a "work based on the Program"
means either the Program or any derivative work under copyright law:
that is to say, a work containing the Program or a portion of it,
either verbatim or with modifications and/or translated into another
language. (Hereinafter, translation is included without limitation in
the term "modification".) Each licensee is addressed as "you".
Activities other than copying, distribution and modification are not
covered by this License; they are outside its scope. The act of
running the Program is not restricted, and the output from the Program
is covered only if its contents constitute a work based on the
Program (independent of having been made by running the Program).
Whether that is true depends on what the Program does.
1. You may copy and distribute verbatim copies of the Program's
source code as you receive it, in any medium, provided that you
conspicuously and appropriately publish on each copy an appropriate
copyright notice and disclaimer of warranty; keep intact all the
notices that refer to this License and to the absence of any warranty;
and give any other recipients of the Program a copy of this License
along with the Program.
You may charge a fee for the physical act of transferring a copy, and
you may at your option offer warranty protection in exchange for a fee.
2. You may modify your copy or copies of the Program or any portion
of it, thus forming a work based on the Program, and copy and
distribute such modifications or work under the terms of Section 1
above, provided that you also meet all of these conditions:
a) You must cause the modified files to carry prominent notices
stating that you changed the files and the date of any change.
b) You must cause any work that you distribute or publish, that in
whole or in part contains or is derived from the Program or any
part thereof, to be licensed as a whole at no charge to all third
parties under the terms of this License.
c) If the modified program normally reads commands interactively
when run, you must cause it, when started running for such
interactive use in the most ordinary way, to print or display an
announcement including an appropriate copyright notice and a
notice that there is no warranty (or else, saying that you provide
a warranty) and that users may redistribute the program under
these conditions, and telling the user how to view a copy of this
License. (Exception: if the Program itself is interactive but
does not normally print such an announcement, your work based on
the Program is not required to print an announcement.)
These requirements apply to the modified work as a whole. If
identifiable sections of that work are not derived from the Program,
and can be reasonably considered independent and separate works in
themselves, then this License, and its terms, do not apply to those
sections when you distribute them as separate works. But when you
distribute the same sections as part of a whole which is a work based
on the Program, the distribution of the whole must be on the terms of
this License, whose permissions for other licensees extend to the
entire whole, and thus to each and every part regardless of who wrote it.
Thus, it is not the intent of this section to claim rights or contest
your rights to work written entirely by you; rather, the intent is to
exercise the right to control the distribution of derivative or
collective works based on the Program.
In addition, mere aggregation of another work not based on the Program
with the Program (or with a work based on the Program) on a volume of
a storage or distribution medium does not bring the other work under
the scope of this License.
3. You may copy and distribute the Program (or a work based on it,
under Section 2) in object code or executable form under the terms of
Sections 1 and 2 above provided that you also do one of the following:
a) Accompany it with the complete corresponding machine-readable
source code, which must be distributed under the terms of Sections
1 and 2 above on a medium customarily used for software interchange; or,
b) Accompany it with a written offer, valid for at least three
years, to give any third party, for a charge no more than your
cost of physically performing source distribution, a complete
machine-readable copy of the corresponding source code, to be
distributed under the terms of Sections 1 and 2 above on a medium
customarily used for software interchange; or,
c) Accompany it with the information you received as to the offer
to distribute corresponding source code. (This alternative is
allowed only for noncommercial distribution and only if you
received the program in object code or executable form with such
an offer, in accord with Subsection b above.)
The source code for a work means the preferred form of the work for
making modifications to it. For an executable work, complete source
code means all the source code for all modules it contains, plus any
associated interface definition files, plus the scripts used to
control compilation and installation of the executable. However, as a
special exception, the source code distributed need not include
anything that is normally distributed (in either source or binary
form) with the major components (compiler, kernel, and so on) of the
operating system on which the executable runs, unless that component
itself accompanies the executable.
If distribution of executable or object code is made by offering
access to copy from a designated place, then offering equivalent
access to copy the source code from the same place counts as
distribution of the source code, even though third parties are not
compelled to copy the source along with the object code.
4. You may not copy, modify, sublicense, or distribute the Program
except as expressly provided under this License. Any attempt
otherwise to copy, modify, sublicense or distribute the Program is
void, and will automatically terminate your rights under this License.
However, parties who have received copies, or rights, from you under
this License will not have their licenses terminated so long as such
parties remain in full compliance.
5. You are not required to accept this License, since you have not
signed it. However, nothing else grants you permission to modify or
distribute the Program or its derivative works. These actions are
prohibited by law if you do not accept this License. Therefore, by
modifying or distributing the Program (or any work based on the
Program), you indicate your acceptance of this License to do so, and
all its terms and conditions for copying, distributing or modifying
the Program or works based on it.
6. Each time you redistribute the Program (or any work based on the
Program), the recipient automatically receives a license from the
original licensor to copy, distribute or modify the Program subject to
these terms and conditions. You may not impose any further
restrictions on the recipients' exercise of the rights granted herein.
You are not responsible for enforcing compliance by third parties to
this License.
7. If, as a consequence of a court judgment or allegation of patent
infringement or for any other reason (not limited to patent issues),
conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot
distribute so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you
may not distribute the Program at all. For example, if a patent
license would not permit royalty-free redistribution of the Program by
all those who receive copies directly or indirectly through you, then
the only way you could satisfy both it and this License would be to
refrain entirely from distribution of the Program.
If any portion of this section is held invalid or unenforceable under
any particular circumstance, the balance of the section is intended to
apply and the section as a whole is intended to apply in other
circumstances.
It is not the purpose of this section to induce you to infringe any
patents or other property right claims or to contest validity of any
such claims; this section has the sole purpose of protecting the
integrity of the free software distribution system, which is
implemented by public license practices. Many people have made
generous contributions to the wide range of software distributed
through that system in reliance on consistent application of that
system; it is up to the author/donor to decide if he or she is willing
to distribute software through any other system and a licensee cannot
impose that choice.
This section is intended to make thoroughly clear what is believed to
be a consequence of the rest of this License.
8. If the distribution and/or use of the Program is restricted in
certain countries either by patents or by copyrighted interfaces, the
original copyright holder who places the Program under this License
may add an explicit geographical distribution limitation excluding
those countries, so that distribution is permitted only in or among
countries not thus excluded. In such case, this License incorporates
the limitation as if written in the body of this License.
9. The Free Software Foundation may publish revised and/or new versions
of the General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the Program
specifies a version number of this License which applies to it and "any
later version", you have the option of following the terms and conditions
either of that version or of any later version published by the Free
Software Foundation. If the Program does not specify a version number of
this License, you may choose any version ever published by the Free Software
Foundation.
10. If you wish to incorporate parts of the Program into other free
programs whose distribution conditions are different, write to the author
to ask for permission. For software which is copyrighted by the Free
Software Foundation, write to the Free Software Foundation; we sometimes
make exceptions for this. Our decision will be guided by the two goals
of preserving the free status of all derivatives of our free software and
of promoting the sharing and reuse of software generally.
NO WARRANTY
11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS
TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE
PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
REPAIR OR CORRECTION.
12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
convey the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) 19yy <name of author>
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
Also add information on how to contact you by electronic and paper mail.
If the program is interactive, make it output a short notice like this
when it starts in an interactive mode:
Gnomovision version 69, Copyright (C) 19yy name of author
Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, the commands you use may
be called something other than `show w' and `show c'; they could even be
mouse-clicks or menu items--whatever suits your program.
You should also get your employer (if you work as a programmer) or your
school, if any, to sign a "copyright disclaimer" for the program, if
necessary. Here is a sample; alter the names:
Yoyodyne, Inc., hereby disclaims all copyright interest in the program
`Gnomovision' (which makes passes at compilers) written by James Hacker.
<signature of Ty Coon>, 1 April 1989
Ty Coon, President of Vice
This General Public License does not permit incorporating your program into
proprietary programs. If your program is a subroutine library, you may
consider it more useful to permit linking proprietary applications with the
library. If this is what you want to do, use the GNU Library General
Public License instead of this License.

View File

@@ -0,0 +1,45 @@
Pyzor requires at least Python 2.6
To install this distribution, simply run the following:
python setup.py build
python setup.py install
Note that your system might install the modules and scripts
with non-world-readable permissions.
Correct this with a command such as:
chmod -R a+rX /usr/share/doc/pyzor \
/usr/lib/python2.2/site-packages/pyzor \
/usr/bin/pyzor /usr/bin/pyzord
To use the server, the Python gdbm module or MySQLdb module is required.
You can generally check if you have the gdbm module by executing:
python -c 'import gdbm' && echo 'gdbm found'
You can generally check if you have the MySQLdb module by executing:
python -c 'import MySQLdb' && echo 'MySQLdb found'
The gdbm module is available at:
Debian GNU/Linux:
http://packages.debian.org/stable/interpreters/python-gdbm.html
Gentoo Linux:
Will be built with Python if the gdbm library is found.
If it isn't there with your Python, try stealing
the FreeBSD setup.py patchfile in their ports to
install just the gdbm module, or simply re-install
Python.
FreeBSD:
ports/databases/py-gdbm
tar.gz:
included in the Python distribution
(not sure of the precise procedure for simply installing
the gdbm module; try stealing the FreeBSD setup.py patchfile
in their ports).
Pyzor also works with Python3.3. The code will be automatically refactored with
2to3 during the setup:
python3.3 setup.py install
Note that the MySQLdb library does not currently support Python3.
See docs/usage.html for usage documentation, and if you are upgrading
from another version of Pyzor, please read the UPGRADING file.

View File

@@ -0,0 +1,13 @@
include COPYING
include INSTALL
include NEWS
include README.txt
include THANKS
include UPGRADING
include requirements.txt
include config/*
include pyzor/*
include pyzor/engines/*
include pyzor/hacks/*
include scripts/*

View File

@@ -0,0 +1,250 @@
Noteworthy changes in 0.7.0
-----------------------------------------------------------------
Bug fixes:
* Fix decoding bug when messages are badly formed
* Pyzor now correctly creates the specified homedir, not the user's one
New features:
* Logging is now disabled by default
* Automatically run 2to3 during installation (if required)
New pyzord features:
* Added ability to disable expiry
* New redis engine support has been added
* New option to enable gevent
* Added the ability to reload accounts and access files using USR1 signal
* Added the ability to safely stop the daemon with TERM signal
* Split the usage-log and normal log in two separate files
* Pyzord daemon can now daemonize and detach itself
Noteworthy changes in 0.6.0
-----------------------------------------------------------------
* pyzor and pyzord will now work with Python3.3 (if
the the 2to3-3.3 is previously ran)
* pyzord and pyzor now supports IPv6
* Improved handling of multi-threading (signals where
again removed) for the mysql engine
* Introduced multi-processing capabilities
* Improved HTML parsing
* Introduced self document sample configurations
* Introduced ability to set custom report/whitelist thresholds
for the pyzor client
* Greatly improved tests coverage
Noteworthy changes in 0.5.0
-----------------------------------------------------------------
Note that the majority of changes in this release were contributed back
from the Debian pyzor package.
* Man pages for pyzor and pyzord.
* Changing back to signals for database locking,
rather than threads. It is likely that signals
will be removed again in the future, but the
existing threading changes caused problems.
* Basic checks on the results of "discover".
* Extended mbox support throughout the library.
* Better handling on unknown encodings.
* Added a --log option to log to a file.
* Better handling of command-line options.
* Improved error handling.
Noteworthy changes in 0.4.x
-----------------------------------------------------------------
* pyzor client now more gracefully handles base64 and
multipart decoding errors, so that it can be used
over an mbox.
* pyzor client has new config file option in the [client]
section, Timeout, which specifies a timeout in seconds
for queries to come back to the client.
* pyzord no longer daemonizes itself, and now writes
it logging to standard output.
* The following server config options no longer have effect:
PidFile, LogFile.
* Upped the allowed signed timestamp difference to be up to
5 minutes (up from 3 minutes).
* Removed the 'shutdown' command; implementation of
'meta' commands need to be re-thought.
* Rewrite of threads locking to access the database.
* pyzord no longer handles USR1 signals; instead, it now
automatically reorganizes and cleans-up the database daily.
* Client code now uses threading to catch timeouts,
rather than an alarm signal.
Noteworthy changes in 0.4.0
-----------------------------------------------------------------
* Messages are now decoded if they are encoded,
and subparts that are not encoded text/* are ignored.
Currently base64, quoted-printable, and uuencode
is supported.
* Message normalization now removes HTML tags
(irregardless of Content-Type).
* Message lines with less than 8 chars after normalization
are now not included in digests.
* Messages having less than or equal to 4 lines are entirely
digested.
* Implemented 'digest' command, which simply prints
out the digest(s) of the messages encountered.
* Implemented 'predigest' command which prints out
the data that is actually digested in a message.
* If HOME is unspecified and no --homedir is given,
The the config directory is /etc/pyzor
* If the pyzord process receives a HUP signal, it re-opens
the logfile.
Noteworthy changes in 0.3.1
-----------------------------------------------------------------
* Fixed bug where if pyzor would send reports or
whitelists to each server N times, where N
is the number of servers.
* Server now keeps database file open, instead of re-opening
it on each request.
* pyzord.log now includes response code.
Noteworthy changes in 0.3.0
-----------------------------------------------------------------
* Pyzor now requires Python 2.2.1.
* The protocol is not backwards compatible, so please
remove old ~/.pyzor/servers files, and they will
be refreshened to point to new servers.
* The pyzor system now has accounts, access controls on
users. anonymous users by default can do
['check', 'report', 'ping', 'info'].
For more information on this, please refer to the
documentation.
* Documentation has moved from the source files
(e.g., 'pydoc pyzor') into a separate XHTML document,
located in docs/usage.html, and normally installed
into a location such as /usr/share/doc/pyzor
* Messages are authenticated using digest-signing, similar
to HTTP-digest authentication. This is a
is a shared-secret scheme, but the secret is very
hard to recover from what is passed in the signature.
* Whitelisting messages is now possible.
* An 'info' command is no implemented
This returns extra info about any digest, such as
when it was first entered and last updated.
* a 'genkey' command has been implemented for the client;
this is used to create a (salt, key) string
used for authentication.
* a 'shutdown' commmand has been implemented, which can
be used to shutdown a server.
* Expiring of digests using a USR1 signal has been removed
for now. In the future a client/server message
will be likely be implemented for this functionality.
* pyzrod logfile now contains a human-readable timestamp
field in addition to the epoch-seconds field.
Noteworthy changes in 0.2.1
-----------------------------------------------------------------
* Fixed major bug where the incorrect exit
code is given.
Noteworthy changes in 0.2.0
-----------------------------------------------------------------
* Protocol break. Old clients will not work
with new servers, and vice versa.
* ~/.pyzor is now a directory, with ~/.pyzor/config
containing configuration directives.
~/.pyzor/servers contains a list of servers.
* pyzord's command-line interface has changed,
now being primarily configured in ~/.pyzor/config.
* pyzord now does logging (~/.pyzor/pyzord.log) and has
a pidfile (~/.pyzor/pyzord.pid).
* Debugging for client and server improved.
* Client now contacts each server listed
when doing a check/report/ping.
* Can now be used with ReadyExec,
http://readyexec.sourceforge.net/
Documentation on how to use ReadyExec is
in the pyzor documentation.
Noteworthy changes in 0.1.1
-----------------------------------------------------------------
* Fixed problem when trying to report messages
in non-unix mailbox format.
* Added --mbox option for 'pyzor report' for when
reporting entire mailboxes.
* No changes were made in the server portion.
Noteworthy changes in 0.1.0
-----------------------------------------------------------------
* Initial release.

View File

@@ -0,0 +1,7 @@
Pyzor is a Python implementation of a spam-blocking
networked system that use spam signatures to identify them.
http://pyzor.org/
See INSTALL for install documentation.
See https://sourceforge.net/apps/trac/pyzor/wiki/0.7/usage for usage documentation.

View File

@@ -0,0 +1,14 @@
Pyzor was originally written by Frank Tobin. Other people
contributed by reporting problems, suggesting various improvements or
submitting actual code. Here is a list of those people. Help me keep
it complete and free of errors.
Frank Tobin ftobin@neverending.org
Rick Macdougall rickm@nougen.com
Colin Smith colin@archeus.plus.com
Bobby Rose brose@med.wayne.edu
Roman Suzi rnd@onego.ru
Robert Schiele schiele@users.sourceforge.net
Tobias Klauser tux_edo@users.sourceforge.net
Tony Meyer tony.meyer@gmail.com
Alexandru Chirila chirila.s.alexandru@gmail.com

View File

@@ -0,0 +1,5 @@
If you are upgrading to Pyzor 0.3.x or newer from an older version,
please remove your old ~/.pyzor/servers file, as the protocol
is not backwards compatible. The Pyzor client will refreshen
the servers file to point to the the public server handling
the new protocol.

View File

@@ -0,0 +1,4 @@
## This file should contain a list of `host : port : username : salt,key`
## each on a new line. The salt and key can be generated with genkey command
## in the pyzor client. Example:
# 127.0.0.1 : 24441 : alice : d28f86151e80a9accba4a4eba81c460532384cd6,fc7f1cad729b5f3862b2ef192e2d9e0d0d4bd515

View File

@@ -0,0 +1,96 @@
## Note that the options that require a file name, must not contain absolute
## paths. They are relative to the specified --homedir, which defaults to
## ~/.pyzor
## All of these options are overridable from the respective command-line
## arguments.
## The client section only affects the pyzor client.
[client]
## The `ServersFile` must contain a newline-separated list of server
## addresses to report/whitelist/check with.
# ServersFile = servers
## The `AccountsFile` file containing information about accounts on servers.
# AccountsFile = accounts
## This option specifies the name of the log file.
# LogFile =
## This options specifies the number of seconds that the pyzor client should
## wait for a response from the server before timing out.
# Timeout = 5
## This options specifies the input style of the pyzor client. Current options
## are:
## - msg (individual RFC5321 message)
## - mbox (mbox file of messages)
## - digests (Pyzor digests, one per line)
# Style = msg
## Thes options specify the threshold for number of reports/whitelists.
## According to these thresholds the pyzor client exit code will differ.
# ReportThreshold = 0
# WhitelistThreshold = 0
## The server section only affects the pyzord server.
[server]
## Specifes the port and interface to listen on.
# Port = 24441
# ListenAddress = 0.0.0.0
## This option specifies the name of the log file.
# LogFile =
## This option specifies the name of the usage log file.
# UsageLogFile =
## This file will contain the PID of the pyzord daemon, when the it's
## started with the --detach options. The file is removed when the daemon is
## closed
# PidFile = pyzord.pid
## This file must contain the username and their keys
# PasswdFile = pyzord.passwd
## This file defines the ACL for the users
# AccessFile = pyzord.access
## These settings define the storage engine that the pyzord server should use.
## Example for gdbm (default):
# Engine = gdbm
# DigestDB = pyzord.db
## Example for mysql:
# Engine = mysql
# DigestDB = localhost,user,passwd,pyzor_db,pyzor_table
## Example for redis:
# Engine = redis
# DigestDB = localhost,6379,,0
## Or if a password is required
# DigestDB = localhost,6379,passwd,0
## The maximum age of an record, after which it will be removed.
## To disable this set this to 0.
# CleanupAge = 10368000 # aprox 4 months
## These setting define how and if the pyzord server should use concurrency
## For multi-threading:
# Threads = False
# MaxThreads = 0 # unlimited
# DBConnections = 0 # new connection for each request
## For multi-processing:
# Processes = False
# MaxProcesses = 40

View File

@@ -0,0 +1,5 @@
## This defines the ACL for each user, by default if a user is not specified
## here he is denied all access ( this includes anonymous users). Examples:
# check report ping pong info whitelist : alice : allow
# ALL : anonymous : allow
# whitelist : anonymous : deny

View File

@@ -0,0 +1,4 @@
## This file must contain the username and their keys, so that the recieving
## server can verify the user's signature. Example
# alice : fc7f1cad729b5f3862b2ef192e2d9e0d0d4bd515
# bob : cf88277c5d4abdc0a3f56f416011966d04a3f462

View File

@@ -0,0 +1,3 @@
## This file should contain a list of pyzor servers to which to direct the
## requests. Each address:port on a new line.
public.pyzor.org:24441

View File

@@ -0,0 +1 @@
*.pyc

View File

@@ -0,0 +1,60 @@
"""Networked spam-signature detection."""
__author__ = "Frank J. Tobin, ftobin@neverending.org"
__credits__ = "Tony Meyer, Dreas von Donselaar, all the Pyzor contributors."
__version__ = "0.7.0"
import hashlib
proto_name = 'pyzor'
proto_version = 2.1
anonymous_user = 'anonymous'
# We would like to use sha512, but that would mean that all the digests
# changed, so for now, we stick with sha1 (which is the same as the old
# sha module).
sha = hashlib.sha1
# This is the maximum time between a client signing a Pyzor request and the
# server checking the signature.
MAX_TIMESTAMP_DIFFERENCE = 300 # seconds
class CommError(Exception):
"""Something in general went wrong with the transaction."""
pass
class ProtocolError(CommError):
"""Something is wrong with talking the protocol."""
pass
class TimeoutError(CommError):
"""The connection timed out."""
pass
class IncompleteMessageError(ProtocolError):
"""A complete requested was not received."""
pass
class UnsupportedVersionError(ProtocolError):
"""Client is using an unsupported protocol version."""
pass
class SignatureError(CommError):
"""Unknown user, signature on msg invalid, or not within allowed time
range."""
pass
class AuthorizationError(CommError):
"""The signature was valid, but the user is not permitted to do the
requested action."""
pass

View File

@@ -0,0 +1,75 @@
"""A collection of utilities that facilitate working with Pyzor accounts.
Note that accounts are not necessary (on the client or server), as an
"anonymous" account always exists."""
import time
import hashlib
import pyzor
def sign_msg(hashed_key, timestamp, msg, hash_=hashlib.sha1):
"""Converts the key, timestamp (epoch seconds), and msg into a digest.
lower(H(H(M) + ':' T + ':' + K))
M is message
T is integer epoch timestamp
K is hashed_key
H is the hash function (currently SHA1)
"""
M = msg.as_string().strip().encode("utf8")
digest = hash_()
digest.update(hash_(M).digest())
digest.update((":%d:%s" % (timestamp, hashed_key)).encode("utf8"))
return digest.hexdigest().lower()
def hash_key(key, user, hash_=hashlib.sha1):
"""Returns the hash key for this username and password.
lower(H(U + ':' + lower(K)))
K is key (hex string)
U is username
H is the hash function (currently SHA1)
"""
S = ("%s:%s" % (user, key.lower())).encode("utf8")
return hash_(S).hexdigest().lower()
def verify_signature(msg, user_key):
"""Verify that the provided message is correctly signed.
The message must have "User", "Time", and "Sig" headers.
If the signature is valid, then the function returns normally.
If the signature is not valid, then a pyzor.SignatureError() exception
is raised."""
timestamp = int(msg["Time"])
user = msg["User"]
provided_signature = msg["Sig"]
# Check that this signature is not too old.
if abs(time.time() - timestamp) > pyzor.MAX_TIMESTAMP_DIFFERENCE:
raise pyzor.SignatureError("Timestamp not within allowed range.")
# Calculate what the correct signature is.
hashed_user_key = hash_key(user_key, user)
# The signature is not part of the message that is signed.
del msg["Sig"]
correct_signature = sign_msg(hashed_user_key, timestamp, msg)
if correct_signature != provided_signature:
raise pyzor.SignatureError("Invalid signature.")
class Account(object):
def __init__(self, username, salt, key):
self.username = username
self.salt = salt
self.key = key
def key_from_hexstr(s):
try:
salt, key = s.split(",")
except ValueError:
raise ValueError("Invalid number of parts for key; perhaps you "
"forgot the comma at the beginning for the "
"salt divider?")
return salt, key
AnonymousAccount = Account(pyzor.anonymous_user, None, "")

View File

@@ -0,0 +1,255 @@
"""Networked spam-signature detection client.
>>> import pyzor
>>> import pyzor.client
>>> import pyzor.digest
>>> import pyzor.config
To load the accounts file:
>>> accounts = pyzor.config.load_accounts(filename)
To create a client (to then issue commands):
>>> client = pyzor.client.Client(accounts)
To create a client, using the anonymous user:
>>> client = pyzor.client.Client()
To get a digest (of an email.message.Message object, or similar):
>>> digest = pyzor.digest.get_digest(msg)
To query a server (where address is a (host, port) pair):
>>> client.ping(address)
>>> client.info(digest, address)
>>> client.report(digest, address)
>>> client.whitelist(digest, address)
>>> client.check(digest, address)
To query the default server (public.pyzor.org):
>>> client.ping()
>>> client.info(digest)
>>> client.report(digest)
>>> client.whitelist(digest)
>>> client.check(digest)
Response will contain, depending on the type of request, some
of the following keys (e.g. client.ping()['Code']):
All responses will have:
- 'Diag' 'OK' or error message
- 'Code' '200' if OK
- 'PV' Protocol Version
- 'Thread'
`info` and `check` responses will also contain:
- '[WL-]Count' Whitelist/Blacklist count
`info` responses will also have:
- '[WL-]Entered' timestamp when message was first whitelisted/blacklisted
- '[WL-]Updated' timestamp when message was last whitelisted/blacklisted
"""
import sys
import time
import email
import socket
import logging
import pyzor.digest
import pyzor.account
import pyzor.message
import pyzor.hacks.py26
pyzor.hacks.py26.hack_email()
class Client(object):
timeout = 5
max_packet_size = 8192
def __init__(self, accounts=None, timeout=None):
if accounts:
self.accounts = accounts
else:
self.accounts = {}
if timeout is not None:
self.timeout = timeout
self.log = logging.getLogger("pyzor")
def ping(self, address=("public.pyzor.org", 24441)):
msg = pyzor.message.PingRequest()
sock = self.send(msg, address)
return self.read_response(sock, msg.get_thread())
def pong(self, digest, address=("public.pyzor.org", 24441)):
msg = pyzor.message.PongRequest(digest)
sock = self.send(msg, address)
return self.read_response(sock, msg.get_thread())
def info(self, digest, address=("public.pyzor.org", 24441)):
msg = pyzor.message.InfoRequest(digest)
sock = self.send(msg, address)
return self.read_response(sock, msg.get_thread())
def report(self, digest, address=("public.pyzor.org", 24441),
spec=pyzor.digest.digest_spec):
msg = pyzor.message.ReportRequest(digest, spec)
sock = self.send(msg, address)
return self.read_response(sock, msg.get_thread())
def whitelist(self, digest, address=("public.pyzor.org", 24441),
spec=pyzor.digest.digest_spec):
msg = pyzor.message.WhitelistRequest(digest, spec)
sock = self.send(msg, address)
return self.read_response(sock, msg.get_thread())
def check(self, digest, address=("public.pyzor.org", 24441)):
msg = pyzor.message.CheckRequest(digest)
sock = self.send(msg, address)
return self.read_response(sock, msg.get_thread())
def send(self, msg, address=("public.pyzor.org", 24441)):
msg.init_for_sending()
try:
account = self.accounts[address]
except KeyError:
account = pyzor.account.AnonymousAccount
timestamp = int(time.time())
msg["User"] = account.username
msg["Time"] = str(timestamp)
msg["Sig"] = pyzor.account.sign_msg(pyzor.account.hash_key(
account.key, account.username), timestamp, msg)
self.log.debug("sending: %r", msg.as_string())
return self._send(msg, address)
def _send(self, msg, addr):
sock = None
for res in socket.getaddrinfo(addr[0], addr[1], 0, socket.SOCK_DGRAM,
socket.IPPROTO_UDP):
af, socktype, proto, _, sa = res
try:
sock = socket.socket(af, socktype, proto)
except socket.error:
sock = None
continue
try:
sock.sendto(msg.as_string().encode("utf8"), 0, sa)
except socket.timeout:
sock.close()
raise pyzor.TimeoutError("Sending to %s time-outed" % sa)
except socket.error:
sock.close()
sock = None
continue
break
if sock is None:
raise pyzor.CommError("Unable to send to %s" % addr)
return sock
def read_response(self, sock, expected_id):
sock.settimeout(self.timeout)
try:
packet, address = sock.recvfrom(self.max_packet_size)
except socket.timeout as e:
sock.close()
raise pyzor.TimeoutError("Reading response timed-out.")
except socket.error as e:
sock.close()
raise pyzor.CommError("Socket error while reading response: %s"
% e)
self.log.debug("received: %r/%r", packet, address)
msg = email.message_from_bytes(packet, _class=pyzor.message.Response)
msg.ensure_complete()
try:
thread_id = msg.get_thread()
if thread_id != expected_id:
if thread_id.in_ok_range():
raise pyzor.ProtocolError(
"received unexpected thread id %d (expected %d)" %
(thread_id, expected_id))
self.log.warn("received error thread id %d (expected %d)",
thread_id, expected_id)
except KeyError:
self.log.warn("no thread id received")
return msg
class ClientRunner(object):
__slots__ = ['routine', 'all_ok', 'log']
def __init__(self, routine):
self.log = logging.getLogger("pyzor")
self.routine = routine
self.all_ok = True
def run(self, server, args, kwargs=None):
if kwargs is None:
kwargs = {}
message = "%s:%s\t" % server
response = None
try:
response = self.routine(*args, **kwargs)
self.handle_response(response, message)
except (pyzor.CommError, KeyError, ValueError), e:
self.log.error("%s\t%s: %s", server, e.__class__.__name__, e)
self.all_ok = False
def handle_response(self, response, message):
"""mesaage is a string we've built up so far"""
if not response.is_ok():
self.all_ok = False
sys.stdout.write("%s%s\n" % (message, response.head_tuple()))
class CheckClientRunner(ClientRunner):
def __init__(self, routine, r_count=0, wl_count=0):
ClientRunner.__init__(self, routine)
self.found_hit = False
self.whitelisted = False
self.hit_count = 0
self.whitelist_count = 0
self.r_count_found = r_count
self.wl_count_clears = wl_count
def handle_response(self, response, message):
message += "%s\t" % str(response.head_tuple())
if response.is_ok():
self.hit_count = int(response['Count'])
self.whitelist_count = int(response['WL-Count'])
if self.whitelist_count > self.wl_count_clears:
self.whitelisted = True
elif self.hit_count > self.r_count_found:
self.found_hit = True
message += "%d\t%d" % (self.hit_count, self.whitelist_count)
sys.stdout.write(message + '\n')
else:
self.all_ok = False
sys.stdout.write(message + '\n')
class InfoClientRunner(ClientRunner):
def handle_response(self, response, message):
message += "%s\n" % str(response.head_tuple())
if response.is_ok():
for f in ('Count', 'Entered', 'Updated',
'WL-Count', 'WL-Entered', 'WL-Updated'):
if response.has_key(f):
val = int(response[f])
if 'Count' in f:
stringed = str(val)
elif val == -1:
stringed = 'Never'
else:
stringed = time.ctime(val)
message += ("\t%s: %s\n" % (f, stringed))
sys.stdout.write(message + "\n")
else:
self.all_ok = False
sys.stdout.write(message + "\n")

View File

@@ -0,0 +1,220 @@
"""Functions that handle parsing pyzor configuration files."""
import os
import re
import logging
import collections
import pyzor.account
# Configuration files for the Pyzor Server
def load_access_file(access_fn, accounts):
"""Load the ACL from the specified file, if it exists, and return an
ACL dictionary, where each key is a username and each value is a set
of allowed permissions (if the permission is not in the set, then it
is not allowed).
'accounts' is a dictionary of accounts that exist on the server - only
the keys are used, which must be the usernames (these are the users
that are granted permission when the 'all' keyword is used, as
described below).
Each line of the file should be in the following format:
operation : user : allow|deny
where 'operation' is a space-separated list of pyzor commands or the
keyword 'all' (meaning all commands), 'username' is a space-separated
list of usernames or the keyword 'all' (meaning all users) - the
anonymous user is called "anonymous", and "allow|deny" indicates whether
or not the specified user(s) may execute the specified operations.
The file is processed from top to bottom, with the final match for
user/operation being the value taken. Every file has the following
implicit final rule:
all : all : deny
If the file does not exist, then the following default is used:
check report ping info : anonymous : allow
"""
log = logging.getLogger("pyzord")
# A defaultdict is safe, because if we get a non-existant user, we get
# the empty set, which is the same as a deny, which is the final
# implicit rule.
acl = collections.defaultdict(set)
if not os.path.exists(access_fn):
log.info("Using default ACL: the anonymous user may use the check, "
"report, ping and info commands.")
acl[pyzor.anonymous_user] = set(("check", "report", "ping", "pong",
"info"))
return acl
for line in open(access_fn):
if not line.strip() or line[0] == "#":
continue
try:
operations, users, allowed = [part.lower().strip()
for part in line.split(":")]
except ValueError:
log.warn("Invalid ACL line: %r", line)
continue
try:
allowed = {"allow": True, "deny" : False}[allowed]
except KeyError:
log.warn("Invalid ACL line: %r", line)
continue
if operations == "all":
operations = ("check", "report", "ping", "pong", "info",
"whitelist")
else:
operations = [operation.strip()
for operation in operations.split()]
if users == "all":
users = accounts
else:
users = [user.strip() for user in users.split()]
for user in users:
if allowed:
log.debug("Granting %s to %s.", ",".join(operations), user)
# If these operations are already allowed, this will have
# no effect.
acl[user].update(operations)
else:
log.debug("Revoking %s from %s.", ",".join(operations), user)
# If these operations are not allowed yet, this will have
# no effect.
acl[user].difference_update(operations)
log.info("ACL: %r", acl)
return acl
def load_passwd_file(passwd_fn):
"""Load the accounts from the specified file.
Each line of the file should be in the format:
username : key
If the file does not exist, then an empty dictionary is returned;
otherwise, a dictionary of (username, key) items is returned.
"""
log = logging.getLogger("pyzord")
accounts = {}
if not os.path.exists(passwd_fn):
log.info("Accounts file does not exist - only the anonymous user "
"will be available.")
return accounts
for line in open(passwd_fn):
if not line.strip() or line[0] == "#":
continue
try:
user, key = line.split(":")
except ValueError:
log.warn("Invalid accounts line: %r", line)
continue
user = user.strip()
key = key.strip()
log.debug("Creating an account for %s with key %s.", user, key)
accounts[user] = key
# Don't log the keys at 'info' level, just ther usernames.
log.info("Accounts: %s", ",".join(accounts))
return accounts
# Configuration files for the Pyzor Client
def load_accounts(filepath):
"""Layout of file is: host : port : username : salt,key"""
accounts = {}
log = logging.getLogger("pyzor")
if os.path.exists(filepath):
for lineno, orig_line in enumerate(open(filepath)):
line = orig_line.strip()
if not line or line.startswith('#'):
continue
try:
host, port, username, key = [x.strip()
for x in line.split(":")]
except ValueError:
log.warn("account file: invalid line %d: wrong number of "
"parts", lineno)
continue
try:
port = int(port)
except ValueError, e:
log.warn("account file: invalid line %d: %s", lineno, e)
address = (host, port)
salt, key = pyzor.account.key_from_hexstr(key)
if not salt and not key:
log.warn("account file: invalid line %d: keystuff can't be "
"all None's", lineno)
continue
try:
accounts[address] = pyzor.account.Account(username, salt, key)
except ValueError, e:
log.warn("account file: invalid line %d: %s", lineno, e)
else:
log.warn("No accounts are setup. All commands will be executed by "
"the anonymous user.")
return accounts
def load_servers(filepath):
"""Load the servers file."""
logger = logging.getLogger("pyzor")
if not os.path.exists(filepath):
servers = []
else:
servers = []
with open(filepath) as f:
for line in f:
line = line.strip()
if re.match("[^#][a-zA-Z0-9.-]+:[0-9]+", line):
address, port = line.rsplit(":", 1)
servers.append((address, int(port)))
if not servers:
logger.info("No servers specified, defaulting to public.pyzor.org.")
servers = [("public.pyzor.org", 24441)]
return servers
# Common configurations
def setup_logging(log_name, filepath, debug):
"""Setup logging according to the specified options. Return the Logger
object.
"""
fmt = logging.Formatter('%(asctime)s %(levelname)s %(message)s')
stream_handler = logging.StreamHandler()
file_handler = None
if debug:
stream_log_level = logging.DEBUG
file_log_level = logging.DEBUG
else:
stream_log_level = logging.CRITICAL
file_log_level = logging.INFO
logger = logging.getLogger(log_name)
logger.setLevel(file_log_level)
stream_handler.setLevel(stream_log_level)
stream_handler.setFormatter(fmt)
logger.addHandler(stream_handler)
if filepath:
file_handler = logging.FileHandler(filepath)
file_handler.setLevel(file_log_level)
file_handler.setFormatter(fmt)
logger.addHandler(file_handler)
return logger
def expand_homefiles(homefiles, category, homedir, config):
"""Set the full file path for these configuration files."""
for filename in homefiles:
filepath = config.get(category, filename)
if not filepath:
continue
filepath = os.path.expanduser(filepath)
if not os.path.isabs(filepath):
filepath = os.path.join(homedir, filepath)
config.set(category, filename, filepath)

View File

@@ -0,0 +1,149 @@
import re
import hashlib
import HTMLParser
# Hard-coded for the moment.
digest_spec = ([(20, 3), (60, 3)])
class HTMLStripper(HTMLParser.HTMLParser):
"""Strip all tags from the HTML."""
def __init__(self, collector):
HTMLParser.HTMLParser.__init__(self)
self.reset()
self.collector = collector
def handle_data(self, data):
"""Keep track of the data."""
data = data.strip()
if data:
self.collector.append(data)
class DataDigester(object):
"""The major workhouse class."""
__slots__ = ['value', 'digest']
# Minimum line length for it to be included as part of the digest.
min_line_length = 8
# If a message is this many lines or less, then we digest the whole
# message.
atomic_num_lines = 4
# We're not going to try to match email addresses as per the spec
# because it's too difficult. Plus, regular expressions don't work well
# for them. (BNF is better at balanced parens and such).
email_ptrn = re.compile(r'\S+@\S+')
# Same goes for URLs.
url_ptrn = re.compile(r'[a-z]+:\S+', re.IGNORECASE)
# We also want to remove anything that is so long it looks like possibly
# a unique identifier.
longstr_ptrn = re.compile(r'\S{10,}')
ws_ptrn = re.compile(r'\s')
# String that the above patterns will be replaced with.
# Note that an empty string will always be used to remove whitespace.
unwanted_txt_repl = ''
def __init__(self, msg, spec=digest_spec):
self.value = None
self.digest = hashlib.sha1()
# Need to know the total number of lines in the content.
lines = []
for payload in self.digest_payloads(msg):
for line in payload.splitlines():
norm = self.normalize(line)
if self.should_handle_line(norm):
lines.append(norm.encode("utf8"))
if len(lines) <= self.atomic_num_lines:
self.handle_atomic(lines)
else:
self.handle_pieced(lines, spec)
self.value = self.digest.hexdigest()
assert len(self.value) == len(hashlib.sha1(b"").hexdigest())
assert self.value is not None
def handle_atomic(self, lines):
"""We digest everything."""
for line in lines:
self.handle_line(line)
def handle_pieced(self, lines, spec):
"""Digest stuff according to the spec."""
for offset, length in spec:
for i in xrange(length):
try:
line = lines[int(offset * len(lines) // 100) + i]
except IndexError:
pass
else:
self.handle_line(line)
def handle_line(self, line):
self.digest.update(line.rstrip())
@classmethod
def normalize(cls, s):
repl = cls.unwanted_txt_repl
s = cls.longstr_ptrn.sub(repl, s)
s = cls.email_ptrn.sub(repl, s)
s = cls.url_ptrn.sub(repl, s)
# Make sure we do the whitespace last because some of the previous
# patterns rely on whitespace.
return cls.ws_ptrn.sub('', s).strip()
@staticmethod
def normalize_html_part(s):
data = []
stripper = HTMLStripper(data)
try:
stripper.feed(s)
except (UnicodeDecodeError, HTMLParser.HTMLParseError):
# We can't parse the HTML, so just strip it. This is still
# better than including generic HTML/CSS text.
pass
return " ".join(data)
@classmethod
def should_handle_line(cls, s):
return len(s) and cls.min_line_length <= len(s)
@classmethod
def digest_payloads(cls, msg):
for part in msg.walk():
if part.get_content_maintype() == "text":
payload = part.get_payload(decode=True)
charset = part.get_content_charset()
if not charset:
charset = "ascii"
try:
payload = payload.decode(charset, "ignore")
except LookupError:
payload = payload.decode("ascii", "ignore")
if part.get_content_subtype() == "html":
yield cls.normalize_html_part(payload)
else:
yield payload
elif part.is_multipart():
# Skip, because walk() will give us the payload next.
pass
else:
# Non-text parts are passed through as-is.
yield part.get_payload()
class PrintingDataDigester(DataDigester):
"""Extends DataDigester: prints out what we're digesting."""
def handle_line(self, line):
print line.decode("utf8")
super(PrintingDataDigester, self).handle_line(line)
# Convenience function.
def get_digest(msg):
return DataDigester(msg).value

View File

@@ -0,0 +1,27 @@
"""Database backends for pyzord.
The database class must expose a dictionary-like interface, allowing access
via __getitem__, __setitem__, and __delitem__. The key will be a forty
character string, and the value should be an instance of the Record class.
If the database backend cannot store the Record objects natively, then it
must transparently take care of translating to/from Record objects in
__setitem__ and __getitem__.
The database class should take care of expiring old values at the
appropriate interval.
"""
from pyzor.engines import gdbm_
from pyzor.engines import mysql
from pyzor.engines import redis_
__all__ = ["database_classes"]
database_classes = {"gdbm": gdbm_.handle,
"mysql": mysql.handle,
"redis": redis_.handle,
}

View File

@@ -0,0 +1,51 @@
"""Common library shared by different engines."""
import sys
import datetime
from collections import namedtuple
__all__ = ["DBHandle", "DatabaseError", "Record"]
DBHandle = namedtuple("DBHandle", ["single_threaded", "multi_threaded",
"multi_processing"])
class DatabaseError(Exception):
pass
class Record(object):
"""Prefix conventions used in this class:
r = report (spam)
wl = whitelist
"""
def __init__(self, r_count=0, wl_count=0, r_entered=None,
r_updated=None, wl_entered=None, wl_updated=None):
self.r_count = r_count
self.wl_count = wl_count
self.r_entered = r_entered
self.r_updated = r_updated
self.wl_entered = wl_entered
self.wl_updated = wl_updated
def wl_increment(self):
# overflow prevention
if self.wl_count < sys.maxint:
self.wl_count += 1
if self.wl_entered is None:
self.wl_entered = datetime.datetime.now()
self.wl_update()
def r_increment(self):
# overflow prevention
if self.r_count < sys.maxint:
self.r_count += 1
if self.r_entered is None:
self.r_entered = datetime.datetime.now()
self.r_update()
def r_update(self):
self.r_updated = datetime.datetime.now()
def wl_update(self):
self.wl_updated = datetime.datetime.now()

View File

@@ -0,0 +1,174 @@
"""Gdbm database engine."""
try:
import gdbm
except ImportError:
gdbm = None
import sys
import time
import logging
import datetime
import threading
from pyzor.engines.common import *
class GdbmDBHandle(object):
absolute_source = True
sync_period = 60
reorganize_period = 3600 * 24 # 1 day
_dt_decode = lambda x: None if x == 'None' else datetime.datetime.strptime(x, "%Y-%m-%d %H:%M:%S.%f")
fields = (
'r_count', 'r_entered', 'r_updated',
'wl_count', 'wl_entered', 'wl_updated',
)
_fields = [('r_count', int),
('r_entered', _dt_decode),
('r_updated', _dt_decode),
('wl_count', int),
('wl_entered', _dt_decode),
('wl_updated', _dt_decode)]
this_version = '1'
log = logging.getLogger("pyzord")
def __init__(self, fn, mode, max_age=None):
self.max_age = max_age
self.db = gdbm.open(fn, mode)
self.start_reorganizing()
self.start_syncing()
def apply_method(self, method, varargs=(), kwargs=None):
if kwargs is None:
kwargs = {}
return apply(method, varargs, kwargs)
def __getitem__(self, key):
return self.apply_method(self._really_getitem, (key,))
def _really_getitem(self, key):
return GdbmDBHandle.decode_record(self.db[key])
def __setitem__(self, key, value):
self.apply_method(self._really_setitem, (key, value))
def _really_setitem(self, key, value):
self.db[key] = GdbmDBHandle.encode_record(value)
def __delitem__(self, key):
self.apply_method(self._really_delitem, (key,))
def _really_delitem(self, key):
del self.db[key]
def start_syncing(self):
if self.db:
self.apply_method(self._really_sync)
self.sync_timer = threading.Timer(self.sync_period,
self.start_syncing)
self.sync_timer.setDaemon(True)
self.sync_timer.start()
def _really_sync(self):
self.db.sync()
def start_reorganizing(self):
if not self.max_age:
return
if self.db:
self.apply_method(self._really_reorganize)
self.reorganize_timer = threading.Timer(self.reorganize_period,
self.start_reorganizing)
self.reorganize_timer.setDaemon(True)
self.reorganize_timer.start()
def _really_reorganize(self):
self.log.debug("reorganizing the database")
key = self.db.firstkey()
breakpoint = time.time() - self.max_age
while key is not None:
rec = self._really_getitem(key)
delkey = None
if int(time.mktime(rec.r_updated.timetuple())) < breakpoint:
self.log.debug("deleting key %s", key)
delkey = key
key = self.db.nextkey(key)
if delkey:
self._really_delitem(delkey)
self.db.reorganize()
@classmethod
def encode_record(cls, value):
values = [cls.this_version]
values.extend(["%s" % getattr(value, x) for x in cls.fields])
return ",".join(values)
@classmethod
def decode_record(cls, s):
try:
s = s.decode("utf8")
except UnicodeError:
raise StandardError("don't know how to handle db value %s" %
repr(s))
parts = s.split(',')
dispatch = None
version = parts[0]
if len(parts) == 3:
dispatch = cls.decode_record_0
elif version == '1':
dispatch = cls.decode_record_1
else:
raise StandardError("don't know how to handle db value %s" %
repr(s))
return dispatch(s)
@staticmethod
def decode_record_0(s):
r = Record()
parts = s.split(',')
fields = ('r_count', 'r_entered', 'r_updated')
assert len(parts) == len(fields)
for i in range(len(parts)):
setattr(r, fields[i], int(parts[i]))
return r
@classmethod
def decode_record_1(cls, s):
r = Record()
parts = s.split(',')[1:]
assert len(parts) == len(cls.fields)
for part, field in zip(parts, cls._fields):
f, decode = field
setattr(r, f, decode(part))
return r
class ThreadedGdbmDBHandle(GdbmDBHandle):
"""Like GdbmDBHandle, but handles multi-threaded access."""
def __init__(self, fn, mode, max_age=None, bound=None):
self.db_lock = threading.Lock()
GdbmDBHandle.__init__(self, fn, mode, max_age=max_age)
def apply_method(self, method, varargs=(), kwargs=None):
if kwargs is None:
kwargs = {}
with self.db_lock:
return GdbmDBHandle.apply_method(self, method, varargs=varargs,
kwargs=kwargs)
# This won't work because the gdbm object needs to be in shared memory of the
# spawned processes.
# class ProcessGdbmDBHandle(ThreadedGdbmDBHandle):
# def __init__(self, fn, mode, max_age=None, bound=None):
# ThreadedGdbmDBHandle.__init__(self, fn, mode, max_age=max_age,
# bound=bound)
# self.db_lock = multiprocessing.Lock()
if sys.version_info[0] != 3 and gdbm is None:
handle = DBHandle(single_threaded=None,
multi_threaded=None,
multi_processing=None)
else:
handle = DBHandle(single_threaded=GdbmDBHandle,
multi_threaded=ThreadedGdbmDBHandle,
multi_processing=None)

View File

@@ -0,0 +1,268 @@
"""MySQLdb database engine."""
import time
import Queue
import logging
import datetime
import threading
try:
import MySQLdb
except ImportError:
# The SQL database backend will not work.
MySQLdb = None
from pyzor.engines.common import *
class MySQLDBHandle(object):
absolute_source = False
# The table must already exist, and have this schema:
# CREATE TABLE `public` (
# `digest` char(40) default NULL,
# `r_count` int(11) default NULL,
# `wl_count` int(11) default NULL,
# `r_entered` datetime default NULL,
# `wl_entered` datetime default NULL,
# `r_updated` datetime default NULL,
# `wl_updated` datetime default NULL,
# PRIMARY KEY (`digest`)
# )
# XXX Re-organising might be faster with a r_updated index. However,
# XXX the re-organisation time isn't that important, and that would
# XXX (slightly) slow down all inserts, so we leave it for now.
reorganize_period = 3600 * 24 # 1 day
reconnect_period = 60 # seconds
log = logging.getLogger("pyzord")
def __init__(self, fn, mode, max_age=None):
self.max_age = max_age
self.db = None
# The 'fn' is host,user,password,db,table. We ignore mode.
# We store the authentication details so that we can reconnect if
# necessary.
self.host, self.user, self.passwd, self.db_name, \
self.table_name = fn.split(",")
self.last_connect_attempt = 0 # We have never connected.
self.reconnect()
self.start_reorganizing()
def _get_new_connection(self):
"""Returns a new db connection."""
db = MySQLdb.connect(host=self.host, user=self.user,
db=self.db_name, passwd=self.passwd)
db.autocommit(True)
return db
def _check_reconnect_time(self):
if time.time() - self.last_connect_attempt < self.reconnect_period:
# Too soon to reconnect.
self.log.debug("Can't reconnect until %s",
(time.ctime(self.last_connect_attempt +
self.reconnect_period)))
return False
return True
def reconnect(self):
if not self._check_reconnect_time():
return
if self.db:
try:
self.db.close()
except MySQLdb.Error:
pass
try:
self.db = self._get_new_connection()
except MySQLdb.Error, e:
self.log.error("Unable to connect to database: %s", e)
self.db = None
# Keep track of when we connected, so that we don't retry too often.
self.last_connect_attempt = time.time()
def __del__(self):
"""Close the database when the object is no longer needed."""
try:
if self.db:
self.db.close()
except MySQLdb.Error:
pass
def _safe_call(self, name, method, args):
try:
return method(*args, db=self.db)
except (MySQLdb.Error, AttributeError), e:
self.log.error("%s failed: %s", name, e)
self.reconnect()
# Retrying just complicates the logic - we don't really care if
# a single query fails (and it's possible that it would fail)
# on the second attempt anyway. Any exceptions are caught by
# the server, and a 'nice' message provided to the caller.
raise DatabaseError("Database temporarily unavailable.")
def __getitem__(self, key):
return self._safe_call("getitem", self._really__getitem__, (key,))
def __setitem__(self, key, value):
return self._safe_call("setitem", self._really__setitem__,
(key, value))
def __delitem__(self, key):
return self._safe_call("delitem", self._really__delitem__, (key,))
def _really__getitem__(self, key, db=None):
"""__getitem__ without the exception handling."""
c = db.cursor()
# The order here must match the order of the arguments to the
# Record constructor.
c.execute("SELECT r_count, wl_count, r_entered, r_updated, "
"wl_entered, wl_updated FROM %s WHERE digest=%%s" %
self.table_name, (key,))
try:
try:
return Record(*c.fetchone())
except TypeError:
# fetchone() returned None, i.e. there is no such record
raise KeyError()
finally:
c.close()
def _really__setitem__(self, key, value, db=None):
"""__setitem__ without the exception handling."""
c = db.cursor()
try:
c.execute("INSERT INTO %s (digest, r_count, wl_count, "
"r_entered, r_updated, wl_entered, wl_updated) "
"VALUES (%%s, %%s, %%s, %%s, %%s, %%s, %%s) ON "
"DUPLICATE KEY UPDATE r_count=%%s, wl_count=%%s, "
"r_entered=%%s, r_updated=%%s, wl_entered=%%s, "
"wl_updated=%%s" % self.table_name,
(key, value.r_count, value.wl_count, value.r_entered,
value.r_updated, value.wl_entered, value.wl_updated,
value.r_count, value.wl_count, value.r_entered,
value.r_updated, value.wl_entered, value.wl_updated))
finally:
c.close()
def _really__delitem__(self, key, db=None):
"""__delitem__ without the exception handling."""
c = db.cursor()
try:
c.execute("DELETE FROM %s WHERE digest=%%s" % self.table_name,
(key,))
finally:
c.close()
def start_reorganizing(self):
if not self.max_age:
return
self.log.debug("reorganizing the database")
breakpoint = (datetime.datetime.now() -
datetime.timedelta(seconds=self.max_age))
db = self._get_new_connection()
c = db.cursor()
try:
c.execute("DELETE FROM %s WHERE r_updated<%%s" %
self.table_name, (breakpoint,))
except (MySQLdb.Error, AttributeError), e:
self.log.warn("Unable to reorganise: %s", e)
finally:
c.close()
db.close()
self.reorganize_timer = threading.Timer(self.reorganize_period,
self.start_reorganizing)
self.reorganize_timer.setDaemon(True)
self.reorganize_timer.start()
class ThreadedMySQLDBHandle(MySQLDBHandle):
def __init__(self, fn, mode, max_age=None, bound=None):
self.bound = bound
if self.bound:
self.db_queue = Queue.Queue()
MySQLDBHandle.__init__(self, fn, mode, max_age=max_age)
def _get_connection(self):
if self.bound:
return self.db_queue.get()
else:
return self._get_new_connection()
def _release_connection(self, db):
if self.bound:
self.db_queue.put(db)
else:
db.close()
def _safe_call(self, name, method, args):
db = self._get_connection()
try:
return method(*args, db=db)
except (MySQLdb.Error, AttributeError) as e:
self.log.error("%s failed: %s", name, e)
if not self.bound:
raise DatabaseError("Database temporarily unavailable.")
try:
# Connection might be timeout, ping and retry
db.ping(True)
return method(*args, db=db)
except (MySQLdb.Error, AttributeError) as e:
# attempt a new connection, if we can retry
db = self._reconnect(db)
raise DatabaseError("Database temporarily unavailable.")
finally:
self._release_connection(db)
def reconnect(self):
if not self.bound:
return
for _ in xrange(self.bound):
self.db_queue.put(self._get_new_connection())
def _reconnect(self, db):
if not self._check_reconnect_time():
return db
else:
self.last_connect_attempt = time.time()
return self._get_new_connection()
def __del__(self):
if not self.bound:
return
for db in iter(self.db_queue.get_nowait):
try:
db.close()
except MySQLdb.Error:
continue
except Queue.Empty:
break
class ProcessMySQLDBHandle(MySQLDBHandle):
def __init__(self, fn, mode, max_age=None):
MySQLDBHandle.__init__(self, fn, mode, max_age=max_age)
def reconnect(self):
pass
def __del__(self):
pass
def _safe_call(self, name, method, args):
db = None
try:
db = self._get_new_connection()
return method(*args, db=db)
except (MySQLdb.Error, AttributeError) as e:
self.log.error("%s failed: %s", name, e)
raise DatabaseError("Database temporarily unavailable.")
finally:
if db is not None:
db.close()
if MySQLdb is None:
handle = DBHandle(single_threaded=None,
multi_threaded=None,
multi_processing=None)
else:
handle = DBHandle(single_threaded=MySQLDBHandle,
multi_threaded=ThreadedMySQLDBHandle,
multi_processing=ProcessMySQLDBHandle)

View File

@@ -0,0 +1,110 @@
"""Redis database engine."""
import logging
import datetime
try:
import redis
except ImportError:
redis = None
from pyzor.engines.common import *
NAMESPACE = "pyzord.digest"
encode_date = lambda d: "" if d is None else d.strftime("%Y-%m-%d %H:%M:%S")
decode_date = lambda x: None if x == "" else datetime.datetime.strptime(x, "%Y-%m-%d %H:%M:%S")
def safe_call(f):
"""Decorator that wraps a method for handling database operations."""
def wrapped_f(self, *args, **kwargs):
# This only logs the error and raise the usual Error for consistency,
# the redis library takes care of reconnecting and everything else.
try:
return f(self, *args, **kwargs)
except redis.exceptions.RedisError as e:
self.log.error("Redis error while calling %s: %s",
f.__name__, e)
raise DatabaseError("Database temporarily unavailable.")
return wrapped_f
class RedisDBHandle(object):
absolute_source = False
log = logging.getLogger("pyzord")
def __init__(self, fn, mode, max_age=None):
self.max_age = max_age
# The 'fn' is host,port,password,db. We ignore mode.
# We store the authentication details so that we can reconnect if
# necessary.
fn = fn.split(",")
self.host = fn[0] or "localhost"
self.port = fn[1] or "6379"
self.passwd = fn[2] or None
self.db_name = fn[3] or "0"
self.db = self._get_new_connection()
@staticmethod
def _encode_record(r):
return ("%s,%s,%s,%s,%s,%s" %
(r.r_count,
encode_date(r.r_entered),
encode_date(r.r_updated),
r.wl_count,
encode_date(r.wl_entered),
encode_date(r.wl_updated))).encode()
@staticmethod
def _decode_record(r):
if r is None:
return Record()
fields = r.decode().split(",")
return Record(r_count=int(fields[0]),
r_entered=decode_date(fields[1]),
r_updated=decode_date(fields[2]),
wl_count=int(fields[3]),
wl_entered=decode_date(fields[4]),
wl_updated=decode_date(fields[5]))
@staticmethod
def _real_key(key):
return "%s.%s" % (NAMESPACE, key)
@safe_call
def _get_new_connection(self):
return redis.StrictRedis(host=self.host, port=int(self.port),
db=int(self.db_name), password=self.passwd)
@safe_call
def __getitem__(self, key):
return self._decode_record(self.db.get(self._real_key(key)))
@safe_call
def __setitem__(self, key, value):
if self.max_age is None:
self.db.set(self._real_key(key), self._encode_record(value))
else:
self.db.setex(self._real_key(key), self.max_age,
self._encode_record(value))
@safe_call
def __delitem__(self, key):
self.db.delete(self._real_key(key))
class ThreadedRedisDBHandle(RedisDBHandle):
def __init__(self, fn, mode, max_age=None, bound=None):
RedisDBHandle.__init__(self, fn, mode, max_age=max_age)
if redis is None:
handle = DBHandle(single_threaded=None,
multi_threaded=None,
multi_processing=None)
else:
handle = DBHandle(single_threaded=RedisDBHandle,
multi_threaded=ThreadedRedisDBHandle,
multi_processing=None)

View File

@@ -0,0 +1 @@
"""Various hack to make pyzor compatible with different Python versions."""

View File

@@ -0,0 +1,42 @@
"""Hacks for Python 2.6"""
__all__ = ["hack_all", "hack_email", "hack_select"]
def hack_all(email=True, select=True):
if email:
hack_email()
if select:
hack_select()
def hack_email():
"""The python2.6 version of email.message_from_string, doesn't work with
unicode strings. And in python3 it will only work with a decoded.
So switch to using only message_from_bytes.
"""
import email
if not hasattr(email, "message_from_bytes"):
email.message_from_bytes = email.message_from_string
def hack_select():
"""The python2.6 version of SocketServer does not handle interrupt calls
from signals. Patch the select call if necessary.
"""
import sys
if sys.version_info[0] == 2 and sys.version_info[1] == 6:
import select
import errno
real_select = select.select
def _eintr_retry(*args):
"""restart a system call interrupted by EINTR"""
while True:
try:
return real_select(*args)
except (OSError, select.error) as e:
if e.args[0] != errno.EINTR:
raise
select.select = _eintr_retry

View File

@@ -0,0 +1,152 @@
"""This modules contains the various messages used in the pyzor client server
communication.
"""
import random
import email.message
import pyzor
class Message(email.message.Message):
def __init__(self):
email.message.Message.__init__(self)
self.setup()
def setup(self):
pass
def init_for_sending(self):
self.ensure_complete()
def __str__(self):
# The parent class adds the unix From header.
return self.as_string()
def ensure_complete(self):
pass
class ThreadedMessage(Message):
def init_for_sending(self):
if not self.has_key('Thread'):
self.set_thread(ThreadId.generate())
assert self.has_key('Thread')
self["PV"] = str(pyzor.proto_version)
Message.init_for_sending(self)
def ensure_complete(self):
if not (self.has_key('PV') and self.has_key('Thread')):
raise pyzor.IncompleteMessageError("Doesn't have fields for a "
"ThreadedMessage.")
Message.ensure_complete(self)
def get_protocol_version(self):
return float(self['PV'])
def get_thread(self):
return ThreadId(self['Thread'])
def set_thread(self, i):
self['Thread'] = str(i)
class Response(ThreadedMessage):
ok_code = 200
def ensure_complete(self):
if not (self.has_key('Code') and self.has_key('Diag')):
raise pyzor.IncompleteMessageError("doesn't have fields for a "
"Response")
ThreadedMessage.ensure_complete(self)
def is_ok(self):
return self.get_code() == self.ok_code
def get_code(self):
return int(self['Code'])
def get_diag(self):
return self['Diag']
def head_tuple(self):
return self.get_code(), self.get_diag()
class Request(ThreadedMessage):
"""This is the class that should be used to read in Requests of any type.
Subclasses are responsible for setting 'Op' if they are generating a
message,"""
def get_op(self):
return self['Op']
def ensure_complete(self):
if not self.has_key('Op'):
raise pyzor.IncompleteMessageError("doesn't have fields for a "
"Request")
ThreadedMessage.ensure_complete(self)
class ClientSideRequest(Request):
op = None
def setup(self):
Request.setup(self)
self["Op"] = self.op
class SimpleDigestBasedRequest(ClientSideRequest):
def __init__(self, digest):
ClientSideRequest.__init__(self)
self["Op-Digest"] = digest
class SimpleDigestSpecBasedRequest(SimpleDigestBasedRequest):
def __init__(self, digest, spec):
SimpleDigestBasedRequest.__init__(self, digest)
flat_spec = [item for sublist in spec for item in sublist]
self["Op-Spec"] = ",".join(str(part) for part in flat_spec)
class PingRequest(ClientSideRequest):
op = "ping"
class PongRequest(SimpleDigestBasedRequest):
op = "pong"
class CheckRequest(SimpleDigestBasedRequest):
op = "check"
class InfoRequest(SimpleDigestBasedRequest):
op = "info"
class ReportRequest(SimpleDigestSpecBasedRequest):
op = "report"
class WhitelistRequest(SimpleDigestSpecBasedRequest):
op = "whitelist"
class ThreadId(int):
# (0, 1024) is reserved
full_range = (0, 2 ** 16)
ok_range = (1024, full_range[1])
error_value = 0
def __new__(cls, i):
self = int.__new__(cls, i)
if not (cls.full_range[0] <= self < cls.full_range[1]):
raise ValueError("value outside of range")
return self
@classmethod
def generate(cls):
return cls(random.randrange(*cls.ok_range))
def in_ok_range(self):
return self.ok_range[0] <= self < self.ok_range[1]

View File

@@ -0,0 +1,299 @@
"""Networked spam-signature detection server.
The server receives the request in the form of a RFC5321 message, and
responds with another RFC5321 message. Neither of these messages has a
body - all of the data is encapsulated in the headers.
The response headers will always include a "Code" header, which is a
HTTP-style response code, and a "Diag" header, which is a human-readable
message explaining the response code (typically this will be "OK").
Both the request and response headers always include a "PV" header, which
indicates the protocol version that is being used (in a major.minor format).
Both the requestion and response headers also always include a "Thread",
which uniquely identifies the request (this is a requirement of using UDP).
Responses to requests may arrive in any order, but the "Thread" header of
a response will always match the "Thread" header of the appropriate request.
Authenticated requests must also have "User", "Time" (timestamp), and "Sig"
(signature) headers.
"""
import sys
import time
import socket
import signal
import logging
import StringIO
import threading
import traceback
import SocketServer
import email.message
import pyzor.config
import pyzor.account
import pyzor.engines.common
import pyzor.hacks.py26
pyzor.hacks.py26.hack_all()
class Server(SocketServer.UDPServer):
"""The pyzord server. Handles incoming UDP connections in a single
thread and single process."""
max_packet_size = 8192
time_diff_allowance = 180
def __init__(self, address, database, passwd_fn, access_fn):
if ":" in address[0]:
Server.address_family = socket.AF_INET6
else:
Server.address_family = socket.AF_INET
self.log = logging.getLogger("pyzord")
self.usage_log = logging.getLogger("pyzord-usage")
self.database = database
# Handle configuration files
self.passwd_fn = passwd_fn
self.access_fn = access_fn
self.load_config()
self.log.debug("Listening on %s", address)
SocketServer.UDPServer.__init__(self, address, RequestHandler,
bind_and_activate=False)
try:
self.socket.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 0)
except (AttributeError, socket.error) as e:
self.log.debug("Unable to set IPV6_V6ONLY to false %s", e)
self.server_bind()
self.server_activate()
# Finally, set signals
signal.signal(signal.SIGUSR1, self.reload_handler)
signal.signal(signal.SIGTERM, self.shutdown_handler)
def load_config(self):
"""Reads the configuration files and loads the accounts and ACLs."""
self.accounts = pyzor.config.load_passwd_file(self.passwd_fn)
self.acl = pyzor.config.load_access_file(self.access_fn, self.accounts)
def shutdown_handler(self, *args, **kwargs):
"""Handler for the SIGTERM signal. This should be used to kill the
daemon and ensure proper clean-up.
"""
self.log.info("SIGTERM received. Shutting down.")
t = threading.Thread(target=self.shutdown)
t.start()
def reload_handler(self, *args, **kwargs):
"""Handler for the SIGUSR1 signal. This should be used to reload
the configuration files.
"""
self.log.info("SIGUSR1 received. Reloading configuration.")
t = threading.Thread(target=self.load_config)
t.start()
class ThreadingServer(SocketServer.ThreadingMixIn, Server):
"""A threaded version of the pyzord server. Each connection is served
in a new thread. This may not be suitable for all database types."""
pass
class BoundedThreadingServer(ThreadingServer):
"""Same as ThreadingServer but this also accepts a limited number of
concurrent threads.
"""
def __init__(self, address, database, passwd_fn, access_fn, max_threads):
ThreadingServer.__init__(self, address, database, passwd_fn, access_fn)
self.semaphore = threading.Semaphore(max_threads)
def process_request(self, request, client_address):
self.semaphore.acquire()
ThreadingServer.process_request(self, request, client_address)
def process_request_thread(self, request, client_address):
ThreadingServer.process_request_thread(self, request, client_address)
self.semaphore.release()
class ProcessServer(SocketServer.ForkingMixIn, Server):
"""A multi-processing version of the pyzord server. Each connection is
served in a new process. This may not be suitable for all database types.
"""
def __init__(self, address, database, passwd_fn, access_fn,
max_children=40):
ProcessServer.max_children = max_children
Server.__init__(self, address, database, passwd_fn, access_fn)
class RequestHandler(SocketServer.DatagramRequestHandler):
"""Handle a single pyzord request."""
def __init__(self, *args, **kwargs):
self.response = email.message.Message()
SocketServer.DatagramRequestHandler.__init__(self, *args, **kwargs)
def handle(self):
"""Handle a pyzord operation, cleanly handling any errors."""
self.response["Code"] = "200"
self.response["Diag"] = "OK"
self.response["PV"] = "%s" % pyzor.proto_version
try:
self._really_handle()
except NotImplementedError, e:
self.handle_error(501, "Not implemented: %s" % e)
except pyzor.UnsupportedVersionError, e:
self.handle_error(505, "Version Not Supported: %s" % e)
except pyzor.ProtocolError, e:
self.handle_error(400, "Bad request: %s" % e)
except pyzor.SignatureError, e:
self.handle_error(401, "Unauthorized: Signature Error: %s" % e)
except pyzor.AuthorizationError, e:
self.handle_error(403, "Forbidden: %s" % e)
except Exception, e:
self.handle_error(500, "Internal Server Error: %s" % e)
trace = StringIO.StringIO()
traceback.print_exc(file=trace)
trace.seek(0)
self.server.log.error(trace.read())
self.server.log.debug("Sending: %r", self.response.as_string())
self.wfile.write(self.response.as_string().encode("utf8"))
def _really_handle(self):
"""handle() without the exception handling."""
self.server.log.debug("Received: %r", self.packet)
# Read the request.
# Old versions of the client sent a double \n after the signature,
# which screws up the RFC5321 format. Specifically handle that
# here - this could be removed in time.
request = email.message_from_bytes(
self.rfile.read().replace(b"\n\n", b"\n") + b"\n")
# Ensure that the response can be paired with the request.
self.response["Thread"] = request["Thread"]
# If this is an authenticated request, then check the authentication
# details.
user = request["User"] or pyzor.anonymous_user
if user != pyzor.anonymous_user:
try:
pyzor.account.verify_signature(request,
self.server.accounts[user])
except KeyError:
raise pyzor.SignatureError("Unknown user.")
if "PV" not in request:
raise pyzor.ProtocolError("Protocol Version not specified in request")
# The protocol version is compatible if the major number is
# identical (changes in the minor number are unimportant).
if int(float(request["PV"])) != int(pyzor.proto_version):
raise pyzor.UnsupportedVersionError()
# Check that the user has permission to execute the requested
# operation.
opcode = request["Op"]
if opcode not in self.server.acl[user]:
raise pyzor.AuthorizationError(
"User is not authorized to request the operation.")
self.server.log.debug("Got a %s command from %s", opcode,
self.client_address[0])
# Get a handle to the appropriate method to execute this operation.
try:
dispatch = self.dispatches[opcode]
except KeyError:
raise NotImplementedError("Requested operation is not "
"implemented.")
# Get the existing record from the database (or a blank one if
# there is no matching record).
digest = request["Op-Digest"]
# Do the requested operation, log what we have done, and return.
if dispatch:
try:
record = self.server.database[digest]
except KeyError:
record = pyzor.engines.common.Record()
dispatch(self, digest, record)
self.server.usage_log.info("%s,%s,%s,%r,%s", user,
self.client_address[0], opcode, digest,
self.response["Code"])
def handle_error(self, code, message):
"""Create an appropriate response for an error."""
self.server.usage_log.error("%s: %s", code, message)
self.response.replace_header("Code", "%d" % code)
self.response.replace_header("Diag", message)
def handle_pong(self, digest, _):
"""Handle the 'pong' command.
This command returns maxint for report counts and 0 whitelist.
"""
self.server.log.debug("Request pong for %s", digest)
self.response["Count"] = "%d" % sys.maxint
self.response["WL-Count"] = "%d" % 0
def handle_check(self, digest, record):
"""Handle the 'check' command.
This command returns the spam/ham counts for the specified digest.
"""
self.server.log.debug("Request to check digest %s", digest)
self.response["Count"] = "%d" % record.r_count
self.response["WL-Count"] = "%d" % record.wl_count
def handle_report(self, digest, record):
"""Handle the 'report' command.
This command increases the spam count for the specified digest."""
self.server.log.debug("Request to report digest %s", digest)
# Increase the count, and store the altered record back in the
# database.
record.r_increment()
self.server.database[digest] = record
def handle_whitelist(self, digest, record):
"""Handle the 'whitelist' command.
This command increases the ham count for the specified digest."""
self.server.log.debug("Request to whitelist digest %s", digest)
# Increase the count, and store the altered record back in the
# database.
record.wl_increment()
self.server.database[digest] = record
def handle_info(self, digest, record):
"""Handle the 'info' command.
This command returns diagnostic data about a digest (timestamps for
when the digest was first/last seen as spam/ham, and spam/ham
counts).
"""
self.server.log.debug("Request for information about digest %s",
digest)
def time_output(time_obj):
"""Convert a datetime object to a POSIX timestamp.
If the object is None, then return 0.
"""
if not time_obj:
return 0
return time.mktime(time_obj.timetuple())
self.response["Entered"] = "%d" % time_output(record.r_entered)
self.response["Updated"] = "%d" % time_output(record.r_updated)
self.response["WL-Entered"] = "%d" % time_output(record.wl_entered)
self.response["WL-Updated"] = "%d" % time_output(record.wl_updated)
self.response["Count"] = "%d" % record.r_count
self.response["WL-Count"] = "%d" % record.wl_count
dispatches = {
'ping' : None,
'pong' : handle_pong,
'info' : handle_info,
'check' : handle_check,
'report' : handle_report,
'whitelist' : handle_whitelist,
}

View File

@@ -0,0 +1,10 @@
# Depending on what engine type you want to use
# you will need one of the following
MySQL-python==1.2.4
redis==2.9.1
# python-gdbm # not available via pip
# If you want to use gevent you will also require
# this.
gevent==1.0.1

View File

@@ -0,0 +1,3 @@
pyzord.db
pyzord.log
pyzord.pid

View File

@@ -0,0 +1,309 @@
#! /usr/bin/env python
"""Pyzor client."""
import os
import sys
import email
import random
import mailbox
import hashlib
import getpass
import logging
import optparse
import tempfile
import ConfigParser
import pyzor.digest
import pyzor.client
import pyzor.config
def load_configuration():
"""Load the configuration for the server.
The configuration comes from three sources: the default values, the
configuration file, and command-line options."""
# Work out the default directory for configuration files.
# If $HOME is defined, then use $HOME/.pyzor, otherwise use /etc/pyzor.
userhome = os.getenv("HOME")
if userhome:
homedir = os.path.join(userhome, '.pyzor')
else:
homedir = os.path.join("/etc", "pyzor")
# Configuration defaults. The configuration file overrides these, and
# then the command-line options override those.
defaults = {
"ServersFile" : "servers",
"AccountsFile" : "accounts",
"LogFile" : "",
"Timeout" : "5", # seconds
"Style" : "msg",
"ReportThreshold" : "0",
"WhitelistThreshold" : "0"
}
# Process any command line options.
description = ("Read data from stdin and execute the requested command "
"(one of 'check', 'report', 'ping', 'pong', 'digest', "
"'predigest', 'genkey').")
opt = optparse.OptionParser(description=description)
opt.add_option("-n", "--nice", dest="nice", type="int",
help="'nice' level", default=0)
opt.add_option("-d", "--debug", action="store_true", default=False,
dest="debug", help="enable debugging output")
opt.add_option("--homedir", action="store", default=homedir,
dest="homedir", help="configuration directory")
opt.add_option("-s", "--style", action="store",
dest="Style", default=None,
help="input style: 'msg' (individual RFC5321 message), "
"'mbox' (mbox file of messages), 'digests' (Pyzor "
"digests, one per line).")
opt.add_option("--log-file", action="store", default=None,
dest="LogFile", help="name of log file")
opt.add_option("--servers-file", action="store", default=None,
dest="ServersFile", help="name of servers file")
opt.add_option("--accounts-file", action="store", default=None,
dest="AccountsFile", help="name of accounts file")
opt.add_option("-t", "--timeout", dest="Timeout", type="int",
help="timeout (in seconds)", default=None)
opt.add_option("-r", "--report-threshold", dest="ReportThreshold",
type="int", default=None,
help="threshold for number of reports")
opt.add_option("-w", "--whitelist-threshold", dest="WhitelistThreshold",
type="int", default=None,
help="threshold for number of whitelist")
opt.add_option("-V", "--version", action="store_true", default=False,
dest="version", help="print version and exit")
options, args = opt.parse_args()
if options.version:
print "%s %s" % (sys.argv[0], pyzor.__version__)
sys.exit(0)
if not len(args):
opt.print_help()
sys.exit()
os.nice(options.nice)
# Create the configuration directory if it doesn't already exist.
if not os.path.exists(options.homedir):
os.mkdir(options.homedir)
# Load the configuration.
config = ConfigParser.ConfigParser()
# Set the defaults.
config.add_section("client")
for key, value in defaults.iteritems():
config.set("client", key, value)
# Override with the configuration.
config.read(os.path.join(options.homedir, "config"))
# Override with the command-line options.
for key in defaults:
value = getattr(options, key)
if value is not None:
config.set("client", key, str(value))
return config, options, args
def main():
"""Execute any requested actions."""
# Set umask - this restricts this process from granting any world access
# to files/directories created by this process.
os.umask(0077)
config, options, args = load_configuration()
homefiles = ["LogFile", "ServersFile", "AccountsFile"]
pyzor.config.expand_homefiles(homefiles, "client", options.homedir, config)
logger = pyzor.config.setup_logging("pyzor",
config.get("client", "LogFile"),
options.debug)
servers = pyzor.config.load_servers(config.get("client", "ServersFile"))
accounts = pyzor.config.load_accounts(config.get("client", "AccountsFile"))
# Run the specified commands.
client = pyzor.client.Client(accounts,
int(config.get("client", "Timeout")))
for command in args:
try:
dispatch = DISPATCHES[command]
except KeyError:
logger.error("Unknown command: %s", command)
else:
try:
if not dispatch(client, servers, config):
sys.exit(1)
except pyzor.TimeoutError:
# Note that most of the methods will trap their own timeout
# error.
logger.error("Timeout from server in %s", command)
def get_input_handler(style="msg", digester=pyzor.digest.DataDigester):
"""Return an object that can be iterated over to get all the digests."""
if style not in ("msg", "mbox", "digests"):
raise ValueError("Unknown input style.")
if style == "digests":
for line in sys.stdin:
yield line.strip()
return
if style == "msg":
tfile = None
msg = email.message_from_file(sys.stdin)
mbox = [msg]
elif style == 'mbox':
# We have to write the mbox to disk in order to use mailbox to work
# with it.
tfile = tempfile.NamedTemporaryFile()
tfile.write(sys.stdin.read().encode("utf8"))
tfile.seek(0)
mbox = mailbox.mbox(tfile.name)
for msg in mbox:
digested = digester(msg).value
if digested:
yield digested
if tfile:
tfile.close()
def ping(client, servers, config):
"""Check that the server is reachable."""
# pylint: disable-msg=W0613
runner = pyzor.client.ClientRunner(client.ping)
for server in servers:
runner.run(server, (server,))
return runner.all_ok
def pong(client, servers, config):
"""Used to test pyzor."""
rt = int(config.get("client", "ReportThreshold"))
wt = int(config.get("client", "WhitelistThreshold"))
style = config.get("client", "Style")
runner = pyzor.client.CheckClientRunner(client.pong, rt, wt)
for digested in get_input_handler(style):
if digested:
for server in servers:
runner.run(server, (digested, server))
return runner.all_ok and runner.found_hit and not runner.whitelisted
def info(client, servers, config):
"""Get information about each message."""
style = config.get("client", "Style")
runner = pyzor.client.InfoClientRunner(client.info)
for digested in get_input_handler(style):
if digested:
for server in servers:
runner.run(server, (digested, server))
return runner.all_ok
def check(client, servers, config):
"""Check each message against each server.
The return value is 'failure' if there is a positive spam count and
*zero* whitelisted count; otherwise 'success'.
"""
rt = int(config.get("client", "ReportThreshold"))
wt = int(config.get("client", "WhitelistThreshold"))
style = config.get("client", "Style")
runner = pyzor.client.CheckClientRunner(client.check, rt, wt)
for digested in get_input_handler(style):
if digested:
for server in servers:
runner.run(server, (digested, server))
return runner.all_ok and runner.found_hit and not runner.whitelisted
def send_digest(digested, spec, client_method, servers):
"""Send these digests to each server."""
# Digest can be None; if so, nothing is sent.
if not digested:
return
runner = pyzor.client.ClientRunner(client_method)
for server in servers:
runner.run(server, (digested, server, spec))
return runner.all_ok
def report(client, servers, config):
"""Report each message as spam."""
style = config.get("client", "Style")
all_ok = True
for digested in get_input_handler(style):
if digested and not send_digest(digested, pyzor.digest.digest_spec,
client.report, servers):
all_ok = False
return all_ok
def whitelist(client, servers, config):
"""Report each message as ham."""
style = config.get("client", "Style")
all_ok = True
for digested in get_input_handler(style):
if digested and not send_digest(digested, pyzor.digest.digest_spec,
client.whitelist, servers):
all_ok = False
return all_ok
def digest(client, servers, config):
"""Generate a digest for each message.
This method can be used to look up digests in the database when
diagnosing, or to report digests in a two-stage operation (digest,
then report with --digests)."""
style = config.get("client", "Style")
for digested in get_input_handler(style):
if digested:
print digested
return True
def predigest(client, servers, config):
"""Output the normalised version of each message, which is used to
create the digest.
This method can be used to diagnose which parts of the message are
used to determine uniqueness."""
for unused in get_input_handler(
"msg", digester=pyzor.digest.PrintingDataDigester):
pass
return True
def genkey(client, servers, config, hash_func=hashlib.sha1):
"""Generate a key to use to authenticate pyzor requests. This method
will prompt for a password (and confirmation).
A random salt is generated (which makes it extremely difficult to
reverse the generated key to get the original password) and combined
with the entered password to provide a key. This key (but not the salt)
should be provided to the pyzord administrator, along with a username.
"""
# pylint: disable-msg=W0613
password = getpass.getpass(prompt="Enter passphrase: ")
if getpass.getpass(prompt="Enter passphrase again: ") != password:
log = logging.getLogger("pyzor")
log.error("Passwords do not match.")
return False
# pylint: disable-msg=W0612
salt = "".join([chr(random.randint(0, 255))
for unused in xrange(hash_func(b"").digest_size)])
if sys.version_info >= (3, 0):
salt = salt.encode("utf8")
salt_digest = hash_func(salt)
pass_digest = hash_func(salt_digest.digest())
pass_digest.update(password.encode("utf8"))
print "salt,key:"
print "%s,%s" % (salt_digest.hexdigest(), pass_digest.hexdigest())
return True
DISPATCHES = {
"ping" : ping,
"pong" : pong,
"info" : info,
"check" : check,
"report" : report,
"whitelist" : whitelist,
"digest" : digest,
"predigest" : predigest,
"genkey" : genkey,
}
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,312 @@
#! /usr/bin/env python
"""A front-end interface to the pyzor daemon."""
import os
import sys
import optparse
import traceback
import ConfigParser
import pyzor.config
import pyzor.server
import pyzor.engines
def detach(stdout="/dev/null", stderr=None, stdin="/dev/null", pidfile=None):
"""This forks the current process into a daemon.
The stdin, stdout, and stderr arguments are file names that
will be opened and be used to replace the standard file descriptors
in sys.stdin, sys.stdout, and sys.stderr.
These arguments are optional and default to /dev/null.
Note that stderr is opened unbuffered, so if it shares a file with
stdout then interleaved output may not appear in the order that you
expect."""
# Do first fork.
try:
pid = os.fork()
if pid > 0:
# Exit first parent.
sys.exit(0)
except OSError as err:
print >> sys.stderr, "Fork #1 failed: (%d) %s" % \
(err.errno, err.strerror)
sys.exit(1)
# Decouple from parent environment.
os.chdir("/")
os.umask(0)
os.setsid()
# Do second fork.
try:
pid = os.fork()
if pid > 0:
# Exit second parent.
sys.exit(0)
except OSError as err:
print >> sys.stderr, "Fork #2 failed: (%d) %s" % \
(err.errno, err.strerror)
sys.exit(1)
# Open file descriptors and print start message.
if not stderr:
stderr = stdout
stdi = open(stdin, "r")
stdo = open(stdout, "a+")
stde = open(stderr, "a+", 0)
pid = str(os.getpid())
if pidfile:
open(pidfile, "w+").write("%s\n" % pid)
# Redirect standard file descriptors.
os.dup2(stdi.fileno(), sys.stdin.fileno())
os.dup2(stdo.fileno(), sys.stdout.fileno())
os.dup2(stde.fileno(), sys.stderr.fileno())
def load_configuration():
"""Load the configuration for the server.
The configuration comes from three sources: the default values, the
configuration file, and command-line options."""
# Work out the default directory for configuration files.
# If $HOME is defined, then use $HOME/.pyzor, otherwise use /etc/pyzor.
userhome = os.getenv("HOME")
if userhome:
homedir = os.path.join(userhome, '.pyzor')
else:
homedir = os.path.join("/etc", "pyzor")
# Configuration defaults. The configuration file overrides these, and
# then the command-line options override those.
defaults = {
"Port" : "24441",
"ListenAddress" : "0.0.0.0",
"Engine" : "gdbm",
"DigestDB" : "pyzord.db",
"CleanupAge" : str(60 * 60 * 24 * 30 * 4), # approximately 4 months
"Threads": "False",
"MaxThreads": "0",
"Processes": "False",
"MaxProcesses": "40",
"DBConnections": "0",
"Gevent": "False",
"PasswdFile" : "pyzord.passwd",
"AccessFile" : "pyzord.access",
"LogFile" : "",
"UsageLogFile": "",
"PidFile": "pyzord.pid"
}
# Process any command line options.
description = "Listen for and process incoming Pyzor connections."
opt = optparse.OptionParser(description=description)
opt.add_option("-n", "--nice", dest="nice", type="int",
help="'nice' level", default=0)
opt.add_option("-d", "--debug", action="store_true", default=False,
dest="debug", help="enable debugging output")
opt.add_option("--homedir", action="store", default=homedir,
dest="homedir", help="configuration directory")
opt.add_option("-a", "--address", action="store", default=None,
dest="ListenAddress", help="listen on this IP")
opt.add_option("-p", "--port", action="store", type="int", default=None,
dest="Port", help="listen on this port")
opt.add_option("-e", "--database-engine", action="store", default=None,
dest="Engine", help="select database backend")
opt.add_option("--dsn", action="store", default=None, dest="DigestDB",
help="data source name (filename for gdbm, host,user,"
"password,database,table for MySQL)")
opt.add_option("--gevent", action="store", default=None, dest="Gevent",
help="set to true to use the gevent library")
opt.add_option("--threads", action="store", default=None, dest="Threads",
help="set to true if multi-threading should be used"
" (this may not apply to all engines)")
opt.add_option("--max-threads", action="store", default=None, type="int",
dest="MaxThreads", help="the maximum number of concurrent "
"threads (defaults to 0 which is unlimited)")
opt.add_option("--processes", action="store", default=None,
dest="Processes", help="set to true if multi-processing "
"should be used (this may not apply to all engines)")
opt.add_option("--max-processes", action="store", default=None, type="int",
dest="MaxProcesses", help="the maximum number of concurrent "
"processes (defaults to 40)")
opt.add_option("--db-connections", action="store", default=None, type="int",
dest="DBConnections", help="the number of db connections "
"that will be kept by the server. This only applies if "
"threads are used. Defaults to 0 which means a new "
"connection is used for every thread. (this may not apply "
"all engines)")
opt.add_option("--password-file", action="store", default=None,
dest="PasswdFile", help="name of password file")
opt.add_option("--access-file", action="store", default=None,
dest="AccessFile", help="name of ACL file")
opt.add_option("--cleanup-age", action="store", default=None,
dest="CleanupAge",
help="time before digests expire (in seconds)")
opt.add_option("--log-file", action="store", default=None,
dest="LogFile", help="name of the log file")
opt.add_option("--usage-log-file", action="store", default=None,
dest="UsageLogFile", help="name of the usage log file")
opt.add_option("--pid-file", action="store", default=None,
dest="PidFile", help="save the pid in this file after the "
"server is daemonized")
opt.add_option("--detach", action="store", default=None,
dest="detach", help="daemonizes the server and redirects "
"any output to the specified file")
opt.add_option("-V", "--version", action="store_true", default=False,
dest="version", help="print version and exit")
options, args = opt.parse_args()
if options.version:
print "%s %s" % (sys.argv[0], pyzor.__version__)
sys.exit(0)
if len(args):
opt.print_help()
sys.exit()
os.nice(options.nice)
# Create the configuration directory if it doesn't already exist.
if not os.path.exists(options.homedir):
os.mkdir(options.homedir)
# Load the configuration.
config = ConfigParser.ConfigParser()
# Set the defaults.
config.add_section("server")
for key, value in defaults.iteritems():
config.set("server", key, value)
# Override with the configuration.
config.read(os.path.join(options.homedir, "config"))
# Override with the command-line options.
for key in defaults:
value = getattr(options, key)
if value is not None:
config.set("server", key, str(value))
return config, options
def main():
"""Run the pyzor daemon."""
# Set umask - this restricts this process from granting any world access
# to files/directories created by this process.
os.umask(0077)
config, options = load_configuration()
homefiles = ["LogFile", "UsageLogFile", "PasswdFile", "AccessFile",
"PidFile"]
engine = config.get("server", "Engine")
database_classes = pyzor.engines.database_classes[engine]
use_gevent = config.get("server", "Gevent").lower() == "true"
use_threads = config.get("server", "Threads").lower() == "true"
use_processes = config.get("server", "Processes").lower() == "true"
if use_threads and use_processes:
print "You cannot use both processes and threads at the same time"
sys.exit(1)
# We prefer to use the threaded server, but some database engines
# cannot handle it.
if use_threads and database_classes.multi_threaded:
use_processes = False
database_class = database_classes.multi_threaded
elif use_processes and database_classes.multi_processing:
use_threads = False
database_class = database_classes.multi_processing
else:
use_threads = False
use_processes = False
database_class = database_classes.single_threaded
# If the DSN is a filename, then we make it absolute.
if database_class.absolute_source:
homefiles.append("DigestDB")
pyzor.config.expand_homefiles(homefiles, "server", options.homedir, config)
logger = pyzor.config.setup_logging("pyzord",
config.get("server", "LogFile"),
options.debug)
pyzor.config.setup_logging("pyzord-usage",
config.get("server", "UsageLogFile"),
options.debug)
db_file = config.get("server", "DigestDB")
passwd_fn = config.get("server", "PasswdFile")
access_fn = config.get("server", "AccessFile")
pidfile_fn = config.get("server", "PidFile")
address = (config.get("server", "ListenAddress"),
int(config.get("server", "port")))
cleanup_age = int(config.get("server", "CleanupAge"))
if use_gevent:
# Monkey patch the std libraries with gevent ones
try:
import signal
import gevent
import gevent.monkey
except ImportError as e:
logger.critical("Gevent library not found: %s", e)
sys.exit(1)
gevent.monkey.patch_all()
# The signal method does not get patched in patch_all
signal.signal = gevent.signal
# XXX The gevent libary might already be doing this.
# Enssure that all modules are reloaded so they benefit from
# the gevent library.
for module in (os, sys, pyzor, pyzor.server, pyzor.engines):
reload(module)
if options.detach:
detach(stdout=options.detach, pidfile=pidfile_fn)
if use_threads:
max_threads = int(config.get("server", "MaxThreads"))
bound = int(config.get("server", "DBConnections"))
database = database_class(db_file, "c", cleanup_age, bound)
if max_threads == 0:
logger.info("Starting multi-threaded pyzord server.")
server = pyzor.server.ThreadingServer(address, database, passwd_fn,
access_fn)
else:
logger.info("Starting bounded (%s) multi-threaded pyzord server.",
max_threads)
server = pyzor.server.BoundedThreadingServer(address, database,
passwd_fn, access_fn,
max_threads)
elif use_processes:
max_children = int(config.get("server", "MaxProcesses"))
database = database_class(db_file, "c", cleanup_age)
logger.info("Starting bounded (%s) multi-processing pyzord server.",
max_children)
server = pyzor.server.ProcessServer(address, database, passwd_fn, access_fn,
max_children)
else:
database = database_class(db_file, "c", cleanup_age)
logger.info("Starting pyzord server.")
server = pyzor.server.Server(address, database, passwd_fn, access_fn)
try:
server.serve_forever()
except:
logger.critical("Failure: %s", traceback.format_exc())
finally:
logger.info("Server shutdown.")
server.server_close()
if options.detach and os.path.exists(pidfile_fn):
try:
os.remove(pidfile_fn)
except Exception as e:
logger.warning("Unable to remove pidfile %r: %s",
pidfile_fn, e)
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,59 @@
import sys
import setuptools
import distutils.core
import pyzor
try:
# These automatically run 2to3 and modules and scripts while installing,
# when ran under Python 3
from distutils.command.build_py import build_py_2to3 as build_py
from distutils.command.build_scripts import build_scripts_2to3 as build_scripts
except ImportError:
from distutils.command.build_py import build_py
from distutils.command.build_scripts import build_scripts
long_description = """
Pyzor is spam-blocking networked system that uses spam signatures
to identify them.
"""
classifiers = ["Operating System :: POSIX",
"Environment :: Console",
"Environment :: No Input/Output (Daemon)",
"Programming Language :: Python"
"Programming Language :: Python :: 2.6",
"Programming Language :: Python :: 3",
"Intended Audience :: System Administrators",
"Topic :: Communications :: Email",
"Topic :: Communications :: Email :: Filters",
"Development Status :: 5 - Production/Stable",
"License :: OSI Approved :: GNU General Public License v2 (GPLv2)",
]
distutils.core.setup(name='pyzor',
version=pyzor.__version__,
description='networked spam-signature detection',
long_description=long_description,
author='Frank J. Tobin',
author_email='ftobin@neverending.org',
license='GPL',
platforms='POSIX',
keywords='spam',
url='http://pyzor.sourceforge.net/',
scripts=['scripts/pyzor', 'scripts/pyzord'],
packages=['pyzor',
'pyzor.engines',
'pyzor.hacks'],
classifiers=classifiers,
test_suite="tests.suite",
cmdclass={'build_py': build_py,
'build_scripts': build_scripts,
},
)

View File

@@ -0,0 +1,18 @@
"""Package reserved for tests and test utilities."""
import unittest
import unit
import functional
def suite():
"""Gather all the tests from this package in a test suite."""
test_suite = unittest.TestSuite()
test_suite.addTest(unit.suite())
test_suite.addTest(functional.suite())
return test_suite
if __name__ == '__main__':
unittest.main(defaultTest='suite')

View File

@@ -0,0 +1,33 @@
"""A suite of functional tests that verifies the correct behaviour of the
pyzor client and server as a whole.
Functional test should not touch real data and are usually safe, but it's not
recommended to run theses on production servers.
Note these tests the installed version of pyzor, not the version from the
source.
"""
import unittest
import test_gdbm
import test_pyzor
import test_mysql
import test_redis
import test_digest
import test_account
def suite():
"""Gather all the tests from this package in a test suite."""
test_suite = unittest.TestSuite()
test_suite.addTest(test_gdbm.suite())
test_suite.addTest(test_mysql.suite())
test_suite.addTest(test_redis.suite())
test_suite.addTest(test_pyzor.suite())
test_suite.addTest(test_digest.suite())
test_suite.addTest(test_account.suite())
return test_suite
if __name__ == '__main__':
unittest.main(defaultTest='suite')

View File

@@ -0,0 +1,138 @@
import unittest
from tests.util import *
class AccountPyzorTest(PyzorTestBase):
# test bob which has access to everything
def test_ping(self):
self.check_pyzor("ping", "bob", code=200, exit_code=0)
def test_pong(self):
self.check_pyzor("pong", "bob", input=msg, code=200, exit_code=0)
def test_check(self):
self.check_pyzor("check", "bob", input=msg, code=200)
def test_report(self):
self.check_pyzor("report", "bob", input=msg, code=200, exit_code=0)
def test_whitelist(self):
self.check_pyzor("whitelist", "bob", input=msg, code=200, exit_code=0)
def test_info(self):
self.check_pyzor("info", "bob", input=msg, code=200, exit_code=0)
# test alice which does not has access to anything
# Error should be 403 Forbidden
def test_ping_forbidden(self):
self.check_pyzor("ping", "alice", code=403, exit_code=1)
def test_pong_forbidden(self):
self.check_pyzor("pong", "alice", input=msg, code=403, exit_code=1)
def test_check_forbidden(self):
self.check_pyzor("check", "alice", input=msg, code=403, exit_code=1)
def test_report_forbidden(self):
self.check_pyzor("report", "alice", input=msg, code=403, exit_code=1)
def test_whitelist_forbidden(self):
self.check_pyzor("whitelist", "alice", input=msg, code=403, exit_code=1)
def test_info_forbidden(self):
self.check_pyzor("info", "alice", input=msg, code=403, exit_code=1)
# test chuck which does tries to steal bob's account but has the wrong key
# Error should be 401 Unauthorized
def test_ping_unauthorized(self):
self.check_pyzor("ping", "chuck", code=401, exit_code=1)
def test_pong_unauthorized(self):
self.check_pyzor("pong", "chuck", input=msg, code=401, exit_code=1)
def test_check_unauthorized(self):
self.check_pyzor("check", "chuck", input=msg, code=401, exit_code=1)
def test_report_unauthorized(self):
self.check_pyzor("report", "chuck", input=msg, code=401, exit_code=1)
def test_whitelist_unauthorized(self):
self.check_pyzor("whitelist", "chuck", input=msg, code=401, exit_code=1)
def test_info_unauthorized(self):
self.check_pyzor("info", "chuck", input=msg, code=401, exit_code=1)
# test dan account, which has some access
def test_ping_combo(self):
self.check_pyzor("ping", "dan", code=200, exit_code=0)
def test_pong_combo(self):
self.check_pyzor("pong", "dan", input=msg, code=403, exit_code=1)
def test_check_combo(self):
self.check_pyzor("check", "dan", input=msg, code=200)
def test_report_combo(self):
self.check_pyzor("report", "dan", input=msg, code=200, exit_code=0)
def test_whitelist_combo(self):
self.check_pyzor("whitelist", "dan", input=msg, code=403, exit_code=1)
def test_info_combo(self):
self.check_pyzor("info", "dan", input=msg, code=403, exit_code=1)
# test anonymous account, which should is not currently set up in the server
def test_ping_anonymous(self):
self.check_pyzor("ping", None, code=403, exit_code=1)
def test_pong_anonymous(self):
self.check_pyzor("pong", None, input=msg, code=403, exit_code=1)
def test_check_anonymous(self):
self.check_pyzor("check", None, input=msg, code=403, exit_code=1)
def test_report_anonymous(self):
self.check_pyzor("report", None, input=msg, code=403, exit_code=1)
def test_whitelist_anonymous(self):
self.check_pyzor("whitelist", None, input=msg, code=403, exit_code=1)
def test_info_anonymous(self):
self.check_pyzor("info", None, input=msg, code=403, exit_code=1)
class AnonymousPyzorTest(PyzorTestBase):
"""Test accounts with no access or password file set-up. And test
anonymous default access.
"""
access_file = None
password_file = None
def test_ping(self):
self.check_pyzor("ping", None, code=200, exit_code=0)
def test_pong(self):
self.check_pyzor("pong", None, input=msg, code=200, exit_code=0)
def test_check(self):
self.check_pyzor("check", None, input=msg, code=200)
def test_report(self):
self.check_pyzor("report", None, input=msg, code=200, exit_code=0)
def test_whitelist(self):
# anonymous account are not allowed to whitelist by default
self.check_pyzor("whitelist", None, input=msg, code=403, exit_code=1)
def test_info(self):
self.check_pyzor("info", None, input=msg, code=200, exit_code=0)
def suite():
"""Gather all the tests from this module in a test suite."""
test_suite = unittest.TestSuite()
test_suite.addTest(unittest.makeSuite(AccountPyzorTest))
test_suite.addTest(unittest.makeSuite(AnonymousPyzorTest))
return test_suite
if __name__ == '__main__':
unittest.main()

View File

@@ -0,0 +1,843 @@
# -*- coding: utf-8 -*-
import sys
import hashlib
import unittest
from tests.util import *
TEXT = """MIME-Version: 1.0
Sender: chirila@spamexperts.com
Received: by 10.216.90.129 with HTTP; Fri, 23 Aug 2013 01:59:03 -0700 (PDT)
Date: Fri, 23 Aug 2013 11:59:03 +0300
Delivered-To: chirila@spamexperts.com
X-Google-Sender-Auth: p6ay4c-tEtdFpavndA9KBmP0CVs
Message-ID: <CAK-mJS9aV6Kb7Z5XCRJ_z_UOKEaQjRY8gMzsuxUQcN5iqxNWUg@mail.gmail.com>
Subject: Test
From: Alexandru Chirila <chirila@spamexperts.com>
To: Alexandru Chirila <chirila@spamexperts.com>
Content-Type: multipart/alternative; boundary=001a11c2893246a9e604e4999ea3
--001a11c2893246a9e604e4999ea3
Content-Type: text/plain; charset=ISO-8859-1
%s
--001a11c2893246a9e604e4999ea3
"""
HTML_TEXT = """MIME-Version: 1.0
Sender: chirila@gapps.spamexperts.com
Received: by 10.216.157.70 with HTTP; Thu, 16 Jan 2014 00:43:31 -0800 (PST)
Date: Thu, 16 Jan 2014 10:43:31 +0200
Delivered-To: chirila@gapps.spamexperts.com
X-Google-Sender-Auth: ybCmONS9U9D6ZUfjx-9_tY-hF2Q
Message-ID: <CAK-mJS8sE-V6qtspzzZ+bZ1eSUE_FNMt3K-5kBOG-z3NMgU_Rg@mail.gmail.com>
Subject: Test
From: Alexandru Chirila <chirila@spamexperts.com>
To: Alexandru Chirila <chirila@gapps.spamexperts.com>
Content-Type: multipart/alternative; boundary=001a11c25ff293069304f0126bfd
--001a11c25ff293069304f0126bfd
Content-Type: text/plain; charset=ISO-8859-1
Email spam.
Email spam, also known as junk email or unsolicited bulk email, is a subset
of electronic spam involving nearly identical messages sent to numerous
recipients by email. Clicking on links in spam email may send users to
phishing web sites or sites that are hosting malware.
--001a11c25ff293069304f0126bfd
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><div>Email spam.</div><div><br></div><div>Email spam, also=
known as junk email or unsolicited bulk email, is a subset of electronic s=
pam involving nearly identical messages sent to numerous recipients by emai=
l. Clicking on links in spam email may send users to phishing web sites or =
sites that are hosting malware.</div>
</div>
--001a11c25ff293069304f0126bfd--
"""
TEXT_ATTACHMENT = """MIME-Version: 1.0
Received: by 10.76.127.40 with HTTP; Fri, 17 Jan 2014 02:21:43 -0800 (PST)
Date: Fri, 17 Jan 2014 12:21:43 +0200
Delivered-To: chirila.s.alexandru@gmail.com
Message-ID: <CALTHOsuHFaaatiXJKU=LdDCo4NmD_h49yvG2RDsWw17D0-NXJg@mail.gmail.com>
Subject: Test
From: Alexandru Chirila <chirila.s.alexandru@gmail.com>
To: Alexandru Chirila <chirila.s.alexandru@gmail.com>
Content-Type: multipart/mixed; boundary=f46d040a62c49bb1c804f027e8cc
--f46d040a62c49bb1c804f027e8cc
Content-Type: multipart/alternative; boundary=f46d040a62c49bb1c404f027e8ca
--f46d040a62c49bb1c404f027e8ca
Content-Type: text/plain; charset=ISO-8859-1
This is a test mailing
--f46d040a62c49bb1c404f027e8ca--
--f46d040a62c49bb1c804f027e8cc
Content-Type: image/png; name="tar.png"
Content-Disposition: attachment; filename="tar.png"
Content-Transfer-Encoding: base64
X-Attachment-Id: f_hqjas5ad0
iVBORw0KGgoAAAANSUhEUgAAAskAAADlCAAAAACErzVVAAAACXBIWXMAAAsTAAALEwEAmpwYAAAD
GGlDQ1BQaG90b3Nob3AgSUNDIHByb2ZpbGUAAHjaY2BgnuDo4uTKJMDAUFBUUuQe5BgZERmlwH6e
gY2BmYGBgYGBITG5uMAxIMCHgYGBIS8/L5UBFTAyMHy7xsDIwMDAcFnX0cXJlYE0wJpcUFTCwMBw
gIGBwSgltTiZgYHhCwMDQ3p5SUEJAwNjDAMDg0hSdkEJAwNjAQMDg0h2SJAzAwNjCwMDE09JakUJ
AwMDg3N+QWVRZnpGiYKhpaWlgmNKflKqQnBlcUlqbrGCZ15yflFBflFiSWoKAwMD1A4GBgYGXpf8
EgX3xMw8BSMDVQYqg4jIKAUICxE+CDEESC4tKoMHJQODAIMCgwGDA0MAQyJDPcMChqMMbxjFGV0Y
SxlXMN5jEmMKYprAdIFZmDmSeSHzGxZLlg6WW6x6rK2s99gs2aaxfWMPZ9/NocTRxfGFM5HzApcj
1xZuTe4FPFI8U3mFeCfxCfNN45fhXyygI7BD0FXwilCq0A/hXhEVkb2i4aJfxCaJG4lfkaiQlJM8
JpUvLS19QqZMVl32llyfvIv8H4WtioVKekpvldeqFKiaqP5UO6jepRGqqaT5QeuA9iSdVF0rPUG9
V/pHDBYY1hrFGNuayJsym740u2C+02KJ5QSrOutcmzjbQDtXe2sHY0cdJzVnJRcFV3k3BXdlD3VP
XS8Tbxsfd99gvwT//ID6wIlBS4N3hVwMfRnOFCEXaRUVEV0RMzN2T9yDBLZE3aSw5IaUNak30zky
LDIzs+ZmX8xlz7PPryjYVPiuWLskq3RV2ZsK/cqSql01jLVedVPrHzbqNdU0n22VaytsP9op3VXU
fbpXta+x/+5Em0mzJ/+dGj/t8AyNmf2zvs9JmHt6vvmCpYtEFrcu+bYsc/m9lSGrTq9xWbtvveWG
bZtMNm/ZarJt+w6rnft3u+45uy9s/4ODOYd+Hmk/Jn58xUnrU+fOJJ/9dX7SRe1LR68kXv13fc5N
m1t379TfU75/4mHeY7En+59lvhB5efB1/lv5dxc+NH0y/fzq64Lv4T8Ffp360/rP8f9/AA0ADzT6
lvFdAAAAIGNIUk0AAHolAACAgwAA+f8AAIDpAAB1MAAA6mAAADqYAAAXb5JfxUYAAGrVSURBVHja
7J13YE3nG8c/Nzc7BLFX7F01ajcENVrU3qpVtalds2oWNWtTe1Zqj1IzLaVG0VJq7y1myM79/v44
92ZI/FDNoHn/4Nxz3vOcc28+5z3P+7zPoBivT+sko9V6je65nvWeP32N7rm89Z77m16fe34b0tV8
XZqLl/UX9nyNqMhuveeiLq/N75w6hfWea9q/NvecCaip16V5vtYke742v3PFCJJTvDb33DyJ5CSS
k0hORCSn/PDpPXaZ8nokkZxE8mtCcrpKn45Ze+yeJJWNSk+1dmseSjr1uW3G4tX5k7RRO6RqPaRv
FfuIj3mbZk8iOYnkBCP5o5uSpCcXjh7dN8otCjzOku78uHRnkL6iaxko+4eknYC5Qds8QJWldyXp
WM6Ut1eZgU9CdTzy5DKdxyzZuGzmrA+hWiHK9Ju6YtPBP5aXsB6uNXjh4iVfN47xVijewBGwy+Dp
mkRyEskvSXKFKZP7t6mrqUCZbXcO1HCNJPlgLoBs14PzaDSZ74Uv/qRrObBbKoV8BD5aX7tw7e+1
30saB++HXT8vZ+PUCtuDbFdbiJvW5QiWJP8wzbLK3mscDKmKqUbnombodaMkeG6RRpuqrrkjhf/e
PInkJJJfXk9OoflQIzjotPRwZgpjn4N2GBtD1EU+DFJf42MH/dnhYWBu1qoQwDb1ljQ+z92Qsnts
JPd+fPwHrcqeMnv2zA6gvdk1J4+HG3jZxvwzux1T5O4ZoKHmxZK2ubBXp1yd/9Yhbamvez9Omb41
XF2SSE4i+aVJzq7vyXrvbvF0jebe1NFkhvqstcbBpvoqfC9LVZDcH6SF46F5aa8ZjFAFsnecd1Ff
aM453dTnnAqxybMnh+bTcrsT4KjddZ5e0Qj5gYprw7TLY7Sml9+mBeyWuvbUUg/tLqsVAO21O4nk
JJJfmuTimsV3agGQbJmaAVBU842D3dT10V766MTPUnD1KvIBN/875r7qfE5S4IQK+jrfBY3AMeRc
pERvTWRTIEBabe+uKcOnZo08mEmTW0saayoRvsKE41+Wwr+HBf55KTiTo/bk0lJMVdeHW1olkZxE
8j8geYZ70EXDDPG5PjdMFxphHFwsL//fcFwi/TZZGyboxJX79wNVubsu6maPova8r8G45IA82hQp
sYEGcOVvnEq2maoVAyQpyoJ+RQ0o/KekcdtDMwKfatTNs4ukJWTStg80qfIx6a/3k/TkJJJfnuT8
+qGGJhrbPqps1YfbAZDsvp+LfIH875jwu3BIunns+GlN76bRAdpZDOqqFwA1NDlSYm996hZ+b88T
SZrVTz82zB+Fy5ZqC2k/PiWtByigjUH7Cwf6l6CwlrdWzyuyjHdOsl0kPMmWtc1yOznl+/i8JIVK
kq58UdQzp3evvVK49+hESHI2rRmu6gDkCr7kCMBY1QBguL7NpO+tHe9cDTntDrgGnO6l+nk2KrQZ
LdXZyme3SImTVC6vFHp4Rqe+Gt1PjaNxOVQfWl1sPgbIpZ+1nBxpoaYm9lfjTrekOz1MSSQnNMnT
wSlfHleS7dLdRuaShyW9Rfp8acHUTTdonAhJLqylK5QLIM1xtTb2rVYegKrBd9N76dt3CwIU1BWt
AmC/uqk11HryOHsrGRazFuofKXGO8qXXFjegokZ1iMo4MEclALpLtY3Rf56mA/CZ+g1XTey8vw1Q
+ySSE5rkAQwJk0LHUuVJUbyh2oUAB5ZL11Y5my2XqJsISS6qKUuVBUwNzmmhdTA8EeoI5h4BIVX4
RN1O/QQk36ZFGmfVQdrpS6CHhtpIbmhTrAFmqaynlhr69oQGGgEOmTLbVIb1ygFk8rNovhvJugc8
qaIZVh29zZdqCFBZ3yWRnNAkL2W6pIvV6NiftuETK5LhxnoG3v97bx2q6TAdEyHJxfRtb61rN+ys
QobY3uqPzuFS6y/5VYEv1Wh6eAHHcoe1tYzGALDSUk2LAI+wAy0MowfemhIpsYfaptM6Q3fYVkG/
LD8VKoXuywfArypcpdv82xpwVgHXpXsV8+trANqqVxu1BZL/qk+SSE5okr9nYphvV0eq3PFIfl9S
FwYe57OKQLEb+pk+iZDkIprvsk9S6KrIiZlfwIlQaWMGYIq8K0kh0mKnytbpXaYmTnemAYwbVtuq
YWe2XIh0NyqhOXYPtgJwOLi4pJBj69f8eiq7oZVLkgK/sss46cSVvSMzkvJSPQBKak9dDSmX5dPT
WulxsHUSyQlL8kaaZYMMMyx7DZ34Au8cpeOYLJB+nzYyPBGS7DykHHbvd6qTKsow2N8/5PRcbwC+
uZmPtod/X9rQhH2dbLYODjYXurbWpbtxDwtGnG1qk5eMRo/CTWjRtnikpxHmGUfXT2z3rktMRyHT
z2rb8RdJltkuH0Y1hSSRnDAkQ4v1wdK3TJSkMLPnUdpKFweaCmgbIxIhyc9p9uYX6mZK8+o+bxvz
UWD5yq8LkPyUvJNITmiSU5L+qwfSeFZJ0jGq7qCzJBXjzs0SPq8fyQnQPPZFuBslkZxQJG9geDMX
0qzVHIZLstRl8hoGSFIZbib83b4eJDe2zHFMIjmBSd7CQN2f4Gb+6aI540n9XYdSoQuYoNA/2lFU
D8aGvN4kN+oeLyTbpUha40twkn+jnaTf7DM+7os5GVS4q+9IW9AFMpxSZUYlQpI/XLVkZLWo5Dja
Q54Je87/Nb18dMIyPvS3i5i55Snf+JNSUQ5mqPnl9Jkdnklnnur2MXc2O7C5EoBjUsxIoiP5bxpI
UlcWWxYVyVttdpjkg8k1e52ZD6Rh5pmJj+TmFkkag0dZsAP40O9q6nmhYVf+uClNp9YYaF0AAJe9
GmCc8+7swwGS9MgRINXhud/9fF2SdAMgU3bnZvM2bVg0xCvK8stj7c1k3X7rLQB+nPKxRQprCyVD
PgTsK6YE76+m9i7rkERywpP8xFRTks64T4zYFTThqG0zMPHpyWnv+hXOVu7Hrxn8yH7PrlTwUZhU
R3fTASUPq95ykTrwURnAvEY/WO0YV+S/b/GYnt0HlQf4QJIlxH9401L5kmf5cMQey52Tkn+opDkR
lo1j2q4LVvPGXwcBclr+ehBStOKN0JL0UQPAW11cf5Qk3WqTRHKCk6x5Pxv4Js67jUlyIw0xNr6Q
5xMdz1It+KHlrsN62RsZi/rvt0CNsNvZYLx22LSAhSoCwC41B/qoSybXGZas9AiQJP3lo1+yQsYG
Fy0201xl7WGCFhkfVoS5A/10TLOgorawUsWAypqUSz8XSllqSoB6JJGc4CQn7haT5A5qa2x0VGP5
Wa49DGir5cxXFrBr/igk39VbQFcdcqtlORexitdHnwGwSvcywTDVhSGq5fDkypGtExtmwSPcFyDT
7Se2iKeZqofbtaD0AEzWu2A+F7ZXFcB0OtTtXIA9UE1TyhuedXmfHE4iOZGRfDXRk1xFJycdvXJ2
hGNrfar5TUJD6rVUd2ZoVOsx5/WwtosOAqYF2uwXWjqKK/3MblM72jFWWgQzVQm6qmUaTc74Vi4T
cOcUUO6s+tlOOPvEGXpYw/P6qTW8r1VXgxyB5SoT/hdAdY2qYT3jmF8SyYmL5JVudxM7yXwnhV3x
15yO6qUl5MpHN31mrE+GrvbkLS0BcPpNGhjJVi1JUh76yWIpwXQVgubqkEeSNBk4GTx/6XlZ/Y2A
FJZrk5f9ckErAaivYTBDLRTw593wx0H6TqsBmqt3LQ0GSBF6IInkxEXyZZYlepKpXSs56R486KyP
QvcDfKyBTNaYHnXTAy30JQDp74VFcZX4RL99Uak8dNHs8KPO01UUPtYnBXVu4/LN3YCTkkK31Yzo
X0GSFBB2ycjyoo3YXfF/Vwq8cPTYfR02+G2pjtU0FaCdRiaRnMi0i6z9Ez/JABxVbzW/cBmgrBbz
jaoY+4epvrGxTMUje3ezKtef6fMJmj1dReEzNc9udZmHw7qlsOWR12ithRWyubFLyQFMN54kq6Bl
dY25Zj9dNaJKmqtzSS0DMvo/Tp9EciIjuVb1xE5yjqwAyZ6c76jmu0IB3J7csusha3TzChvAtgAp
AAYbHvF8ou5Ox3ROxaGPGuQyvOuBo8FOH5/Sg4a2/q3UEeBblQNgguqPVwvr49BPfmoK0ExfZtMW
sN9mfQ8kkZyISO6XNbGTfOOgEzBQYzqq6XoBsFRVG9hsc9uV3v2jaSva0cUIuzPaN1ase6sJJZ9I
FaG7ambRGmwhJ+DY/XFIhYgxuYPhgd/K6om88UJgqsGqCTBKUgWAUlqeTL+Td5t22CeRnNhInmd6
nMhJHqdfmndZrlsebVVvdjAAdTQml29V4/AGrfaXdM++VtTA0rkqZ1/owxYdh2/PDU3D9QF8oQZO
utj/61GTF256Z588gFo6ZUWyur4wArBHGZ/3SasZojoAC3RRt+763bjd7GJIZv8Hq4O1LnnSGl+i
I/ligfuJnGS3jZL0ZyEyj8/oYey1/65OBEUfSbdHls2Yn3RfZYxka6aOGosgXgBf3csHDYLrcc2Q
HfbJCpU34lo7Gv2T+w0y5pITjM+VpDZ436gIUP7rWmcuXLhw6Ub5kl91lfR73SS/i0RIcmJrsc34
6g3oXObZ3vQVKzjE3NlF4We+H9qtYx/Dochs/cf1g5pVvMoWTkPuTi4A7kNtuYfM1lQCtpimnnNd
YrvYOzO6FkzyIEoi+VVsFy/ZUjj8o9MyJOVPfqrd27fx5yexHQg/fCA4MZBsORRNP1558uWEXvvx
mYf2DAxPBCQn5bT/N0gOnVAJIPXiyF1XN0/+bvkJ6UBOSDfbknAk+/1p/L+QL6L0uW1OfT066Gcl
6fQPP9yTFHjr9N41s8ZaHebOBkpqwQ5b16gPZuhhqR33k0h+Q0h+hFPVLv0+c2aeLnUsuVz6u7kd
gKn2dhe7mp+4MzDhSK5u5ytJ9zKZ9xxftewPaXx3i6Q+TLC9NGY/kDTBdER3qwH1FNrZyPNqhEgp
0HWYpA10ubo2UJKqpQ/b/0D+mQdK0hguqrRzcBLJb8qY7JZTklbgtT8FkO9wDz5Z5us7OzdvmxZJ
5z3sgxKM5HXUl6T2VMkO0OW+0/uStI1W1g4/4iNpCaPUEa9JK0/rJrnqftJ9vM/GW5KkX5gkKcCp
RH+2SLKkcb7E17pDDUnrTG9JTgX0xpDcKd9/XE/Ol14KX5OOKekdv+1fmBLfsl7SAvvMm76VdNrd
PeFIViGn29KvZvfkDm3fKpODdoYvzjW8rcf786ek3bRSMe5I0gn6RRU3jYWSVMalAVMkXaD6X3SW
cruc0zkPp726yvv/OsnODXI+k6LSk0d7AThGTvGqzqj0f7BLnzv6ZLBov0IR22UnZyZa1s5ZT1cW
HTH969rm/w7JhdPPrO2JecpgxkiqRm8m63x7UvwsKXxVWsYl4Izva7rqbh5TWQboyaNVOLBOkkRh
6/Gy5jBJl6mm+uy6tGDZkz18GVVcf36SpCbkJH+ItIrel6grLef9gKKmBdIR2v3rJJfxafQsLtuH
SapKunlnbreBNuMdYYy0M+2MguD++bdFMfX60gXACaDo7AybLArY1igy1WaXMAV1wm72AMN3412A
rIZPR8bbAXlT5ajkbnR8rxjmOZKkY9n+MyRnAdI13q2sTvcljaElPb5zoOTfksJr4TItIW0Xj9Ob
t33G50Xtbko6DvwuSfcoZhwOdC4kSUHk0lKSA2mm41n0vcsR57fkD0NqdpgojWXhXSpJqkZhOkr6
/dVSy8VKcjuf0tGTsLjbhsUyIfcrl95Yyu0Pi2RpwWXNNY3WXa3qp02mj69Ix+kr/ZUdKgcPAr5T
nTu75p6TVtmsxe9ZTmy+YymZU5a3gev3TACrLQ0A0zaNnBYiXc1jX9gOx6BTTos0MmfKwlMse/4z
JLsC7uN1lxKStIRmNGsNrp8HSGsNoBPQCrfV7GGX+6FrXknaCxyRpD+pbRw9Zd3wSKnQrm5eI1pS
FHDZH4XkS5JUiYIkcz2u3myWOb+kq6koHiTpHO3/bZId589zAs8vvlswOCfm+p7t5/rMMfKBcyCs
CsBsdf+iY+DtZJKO6UJ3jR0lHZNllY56+D9crEtpmaPgzPBH6M5jQOXd2mAN9z8WWpg6WlNMmg0e
2gVAqUD/AvC5jp2xHNd6fd9enrBWjY0iI+xViv8Iyf54rW2XijFXqC5JC+hP1UczP8pAuSBNpXtC
r4z0xW7vTePW5mAyzGmTGWQc/JOPJUnpUhmfD5GOoVHPbsI5SaEp0r/HNAoHdcNXqTJLuprScPK1
fLzl3yW5ULOOPpM79HFqs2TkJJ+leUr5+PjM/mrau9YEnCsBSlh+AXzkrYOndDLrMDWeqmBtf7uS
Fg9Wb8ZoEdukRTgHHf3lFoDjQav8DzUVTMdDSkuPklNIy2yuccfNBQIfnVB7j5mOoRfX3jFBOw3T
WMC11l1/+/8IyedoLJ1O4XzN0TNUUj9WOBaRFFqP6XqYpUFCkxyWyxx6h3clqQE1WCBJpUwnjIMn
jTE5yJTX+LyF/NFzYHRhv6SRdG7EhdZ078jvypZSUk2y8LMkyw39qySbFvj4+Pj4LE/lmPXdFNV8
2uTz8alnFxlOUt1wxqwLjFAfrUn/kTublXe1SlU18YW6XAzywO10eO4dfpdDixfRcl85APygAlaf
0GLARHWQNJgKEf7Lq9XtkOYZecXvPXi0GfhAG7R98R83wxTW5Y3VLgIDJeluqBT45cRw7aWbpIF8
15jR0tnMznc9U0vSanpLJ+4lNMmqzi15pAqV/rTLvpFukhbZlAuFuKcNluRL0x4fGcpEN76KevIk
pkl/uiW/2IF9QQVNhTmifIRrIh/utysuhdUwbfp3x+TCFZaPzJ/NhVKLfZZ09Wnytk/vSITOBJsB
Dod7AIM1woiE+iPMfrs8gDVqpYfn/CX1/eVqNR2rqTGblBrgaLgbgJ3fOYBeGqo9AQ/TVbQmEoe8
CtOsNSoJOFmCNBWoogOSQo//OK7MG2u7OOiU4qF03jnTfc0Af01kqqRTVL+elqI13fhGXiy4oeBK
/KgjTu7bEprk5vyl5szTzUKsfujkGaqtLslO2w42Zbp0pATLGyS7oYBO5N9LP0lh1/f9eFSSrtiV
1yVPFmoQP2qfGY6oDKH3PZwuqhG/aRf/uhWuik9twGX+wkbj5vnkL+9TLxKhuzeN/24BLNJww5Xt
7H3+CAY4oIFS4MlNG/X92mA2aKd6/KjsQDVtN9wsDIflbpqgWQM1vaLNqRM26GCyKw+MCJUnGgQ0
1SmdVdhc9zfYnrwb9kp/eb4jzaartPOdy5JU3O7mX+9B9gXSzBSYsnpQzaIlsDShSe6Kr444Onkl
p7P0KfVamZ23Rhw845aso5eJhuE/UrxTNvJfO0DzuW2LOwJvS5KqUSMtQ6RR7JA6wEmVJHQmXaS1
dNJ+bBr3v0ZyK598QFGfZqRZNtnuo6h2jD/DkwGcC3IE0+mwVkZs9JkwhxPXAS74L1MOM2TS1pni
bYvUbLmyQwG/oCLW+pQ7DB/QURrvdDpocCTJLTWUx9cM7/s96gV8pT/Uvt55Xaz4Bq+MnNxnkRQW
JMk/cu8PLW5KfpfCJOnsgHKp0ra8L2np7ATXLrrjK23Pa8oz2SLdLg4FfKMc9XHDruZWydIOUra5
aYRgOhdr1HvkZknStQK4T5fkWzdYupc+xWN920kbqlyRnrhmk3x+sPzLJHf3yQRU8amb/FOf9/nS
J0oI3Th1gre+uqXaMFjrmqgRwPcqceYcwPmA368C2GvXFHnwk/T+fGWn2s0I582rAVkg5S2/TupN
BUtIZHh2A43idGhyYJP6qTeY/7JMV3OcB4YEN0/y6kyQFpPk7z+4LtlKmCl8/97ozmt3d1m9ic4e
DpZ0Y2jXSQfDohwPvxhlifJyVJvi2u2vfLexkFzf55ve3Umz2MfHx6eDadIyuyjrdU/C9/4t3Qp7
PONnXcvQ1Kja1EE9rv0BsEdPtgMk009DVIhGUun5en+2wiIyerbV76Wz79TgPvoU5ioy/0V79WC8
+kEVy4EC2mZisNa1V3eg6pOAzEkkJw6SE3OLhWSXIct9FkLhIcNaT5ifvHGbqIvNXscVvqttslb3
pcOFyPyFM0CGzbWuLAaYLsOs5vT44ICQ1Ji2nXKbI+nvCpFrLCslaadrf5WHZAetpYSBnmpDxgeP
27R+GFaVtfrrkO7kKhY6zkgRMDyJ5CSS/8nKCK6prWt6bulj6KhpnABSeeeJVusxZQaAT+43SwlA
7typ37bmf1s+t1FUvwm7RrMXtjaToaUZcC0TIaTAlNzQLFQKagkpF1vCNueAVCYAp365kkhOIvkf
kZxgLf/QgUa5KTenpPzJSSS/xiT/1/2TT/d5nERyEsmvP8nh+dwfJpGcRPLrT/JvNNBrRPLJqa8D
ycYszcn++Vg5Fcj0/ztUb/l0pb6qU1ctrpZEcozWy4iqeF1InlggMZNcbPLx07v3n/YLvZvb1PCg
RRdHuUG2z1q6QZqCZas16vbN8sVWA0e1ftNX/3LyrrTduWHfRhG2ulI7rDEhKa2BJ75Khlv1XFls
doqSByXJ0hLytstuuHoM750GoMjEXybFjH/K1hegaJNPy5nfbJLDczs/fJ1IHpAmEZOcL0ChjxRy
/9yBza4TdXPD5pvab+4eKl2pstF2yt+GXeGcJCnwyt7Ve25LWpE2tbF4ctlS1uBtSdCHANx5QOnb
klYbsFcO0KpKaYof6k7bUAX8VgnzMknX80OXMEn+BU3DbMla3LIbFR+y0OysJP2e840m+bAR8/na
kNzd/CDxklxUvxkeOykoYpnkCG4HNdByd+h0Hdk07zdN6/tJZU/rmt/7dStvuJgKSp2ybG9ezNdy
5Uo+oxyUNRUiI/SkGOBiOZT+TsDYXceMbIbZ7gc1sWbQCt//41k9fmuUZl1bp4Om6pY79bK2vF0n
h7ESMqWF3aGwhlBZm2kQptPfLzyrvxzfZJLHMvu1IrkHfydeku38npgAaqtIQzUwdIOQ0LLQviH0
1nvR3/vfP4Asl0+XB+ZoTOiJZNhvkI9Ni+gsnU0DJbRwqtpCZS03KpFYU8PZnzzrCk30S/BanNgp
7+PWag+NjR5Bk72ky3YM1KfufpYWgPNWa0bQN5TkDzj1WpH8BX8mYj15mzIaNDX2Uk+gikK1PsJf
rVV0kpcHQk1VAIdeYX7ugzWGSdrtEukUdEMbTHyswX7+TpBOvwN5w0852FIlNgC4KlUFumiLLUJq
qioCTlrxuZ6oDjNU/VNtM8pXN7F/k0nOmzz0tSK5P7sTMcmLNWDEzgG0UNdyGpG99DcBIVO0fvja
3Td6Q131jE7yYqXCW8t3H/XXoxrYHwweoT9TRxytpolb1ZseGqk1gCn0PNBKQ8F+9LQFC/+85wiw
X4/NwIcKUzuYej1o1EV/Z8BdG0dpsPYxU17r1fA/YLsIcCil14rkIUawdCIleaKkkJHU1lAvhUm6
+l4VSQp7FF6cmnqqSvUMvUV5BUqPp+QASlp0NYrbWnEty3QjpNAQTTSCnG49ANpqILj6SdJaAM7r
pJEMXyoJA/7UVS02nPKXj1LNnSo2XZWPqzB4bzj6c8c3meTzicua/HyShyfqMXmQ1lRxAS+N9pak
H1yhzRc10jg10+CYJA/Sh1TSuJnBmmQCOKXCUVO2aA8ttHmIBmgVYPfED/DSZsAudVYZXvbJLfIF
WKkrKg7Ut/6CFNCiUarZWFvHqeZv+gDqWyQ/uzeY5G10e93G5N8SMckdjRKRXhpSSxM2SutSkNkO
aKZBMUlurzaU1Gjy/KFeRiKWHFEPh5zH/KsOqvbNwMyQQ3sB8x9GKvymMlIAfCn9ATS2nPpCw8Hl
L/krA0ARTR+lqnaH9JOaDdJhM5A55NabPCavZMzrRnJiHpNbGeEd5TSxtrrjvVMH0j/pB3ytj2mk
p4Kcm6gf2TQb0voF5wVWKy1EVlK4dh8K+0s1e2hnMqyhTgUfhk31JM35sPMh3XO8M11Xz4Z5JhsQ
4lcw/eOwaa13aNaVy8YSjWaOUm3qK1x93Y5rODBQO99kkqcnMiPcc0nuy8FETPKnqmeUsJn2gQaD
eaGG/2RZXKbU/dBcNH/adlFJ3+Kq1UA3rQU2WuxIdW287fC5+0BPqaZprW5tDgs0Bux3Livs8F2N
fvuuJF0o3FWP7up+eXj3jqSfHK9ftEvZdO6WPPpxqcrBPmkyRYLC61I1PKRSsjeY5CFsfL1I7szh
RExyM1UDyKgfK2gGkPru4/yHJKkLdIkIx7O2gpoEPw0G7LZuBb5/BL100+assekGYP5FNXEc7a9z
tjSI7p8f0pPxZrJ+s3zpp86Y+54+NTUzQPLG/SrZsUyPQ6VrmXbN7HfQETpJk6GrQg+GamifoNpv
Lsk92Pp6kdyEC4mYZI/dRnaJ3lXtO+QCqDkMuwZz1tUBcszPGp1k+/bvRngc2QGVOkDO5RG+QenT
AyRvYAacMkWNMXH+P+5Baacd3TG4REScidOK+1WArk8U2Mf0x6P6bzLJP79eJJd0CknEJCfalqxA
Mkib7g3WLtoYyS9fH5Kz5FQSyUn+yTFbC/a9ViRfSVxu+P8+ySmLlS9RysvLK3fmt3J7ej6tR5iT
JZH8jPbRa0byz/Fb9STeSS4WPQ9f+P371y6cOXn+wl+HfXceOnH9YUi5JJJjb58lKvPs80n+hmWJ
n2THjzz+KWV9n3fFiTbkx//wRVpbFEkBt2eIM+cp4PAfIbkjO18rkutxOfGS/P4GY13jc30dtbTI
4qhcO7Z+OtopHak/HTDo887DPwdaS1LoqZ1Tun1Up8mEWK5oLSXSIETSrVT2zces3HE+NCI5i13U
IiMlTf3vS4988v0nSO7GlteJZEu6bEq8JM9QIYC8AY/eGbRlZg7wrgL00WCAglvzANRUNzIsP7+3
Ldhbs3s3vGPIuACUkqZHOHaWUcjlo9eCdPOhJPmPCpaszLrdDmyUcdj35m6S5P/HktLtD6xMj3la
sO+YNNBkmQtw73x9/Tnxu8u6m5Oiw3674PNmkzyaFa8TyQeiVZhMbCS31TozOB5Q18OSHpZk189A
lrBLAOU1AqCr2tr9psfh+rTRrUyA6VfLfG2uW+eT9o3yAM7SnooVvGm889jJoW1LOwMN5FQ0XFJP
ZknWFe+2GoZR56FdrhRgWqjg8FUMV7h07S3WqjG46sgueYL9MC1ZaVHYtUVvNsmTmfE6kfwlvyZi
ks2+6g5TtWCWZuS8rF+5uwlgnzyAtzQFYKAavqdtdnnD95XWMKC2Vs2M6gPnL0m7GKVHOcDx7Uat
mi96ACsk5XB/LKkHACNlJJMdLW+jgvu+dL025Ak9mrVOT8tph9/1I3jId2u4C1BA0vaGad907WIe
419c3qMrCUyyJUfqkERMMnkCH9VspGP5Qi67YL75xNGIWPJVciCXFhghHZW+1qfwm7L7XzXDT3rv
O207ecGWMPymJPkySuepcyjgifEiIvMDhdi/I0nfAjBWTUr2HlmSIVp14GjhFA/upAMmqRWwRE2u
KywHWeT7vTIC70na3i/bm07ywpepttcoa2jCkryP3krMJPPRY/+7AQU+Vy9gnYpqHcCNABOQWisz
lGkx+LTybVAB1wzj9MliVcQj/JTpKynobm0rZn9J0k+M0lHSpIEOkjaQiXqWU9SSZC0tUl0WSfp4
iCS/oh30JcClB65AI40JDdYyPLV2tooDX2neH1JI+zec5LXPt/so8MhPc4Z2H3Tme1slA8veLRH5
s3cM7TpyR3jMkx7O/O7CC9yiX+npL0Fyc+MP/bwWmmAkM0jqzxKVBObLSxuB5DoCkMN2TmprtZ/V
NTWDTzWCTzQ00ubwuyT5MEG/kr64d6URktYwoBLfrqWPJI21xqsentWunZZNV+vksE5vARm01fDn
XKGDO9TcUwtHqArYHQ9JR7aON0Lzvtkk//K8GqFXG2e0OmfN9Eh1VZJ0+12w8dcZgK8ky5Si/a9G
5odfnRbsvo4Us7umZ5nu352QfpndOH/2mtOtqehC3nv6nfD/SD5vX/kFvvWV2g75YsRf3558XpLk
v2d0i3mRu//aNn+X9TEcWvuZ9r3Zh16Y5OI64chP8gA2qYS2A8WMQKWSOr9mepdqf8ruir+v7/qJ
9/xTPLnjNFfVqK6vIl/9W4yBd4HWMMkqexv9rqd3+oI5irBdGLqM1vygQsB+OQGlNR+guZbJN8/D
wGZaNUgfQhsjVLWPOr3ZJJ97XvTTUYeM7k7DZ6w/cORYcp+TacZLIe+Qzuafv538Ow6saP2H1Aeg
wCXrWUvNLsMmZaspaWaOQcct2uCUq3pWoNJZcMqeGYosC5cU0pDKoS9Ochd+ef6XvpydvOYCYdLt
EZGm8kdvk9nv+NSmuU1AQ9veOx8AFDcYr045XRrnJ0n7mudrbatbcSBEwc9+2mOQnE7bYZPSg/OD
AHftB9oY9FXUSKPOY+q7xw1kc81R273KSPnI3PTgK0n92azVRoluST/TUcsws12y/s7kzQtU0qw1
ygH8qNxAXq0BXA6oijZRKyxAP3RXS8o9elIAYI4avdkk36fic4SEqT5+hlKhMaQ4pt406GTzoGvM
98bGQVNqJ6dM1DE+7XVw3yMFhlkN+UWvvY+/dGJOXvJS86F0611YJ9304t1HL64nX3Wu8vzvHPgO
wzWIU7qUFVPvH4MfFal1VWpLKlaVgNTeny/6O9jaNaikqVHbos4UeLL/3atSBQ6vY7lk6WVHFtr+
4OYvhXxhGqRQ+yovTLKHtsN4jctWYb1mcOeBmeR/GfaF9wxe16jA8esAC1S8QPCtJzehovol827d
b8rwNGAsuQ5mtzaTr/ZnvQeMGrVxHZ0VntY6GRxpFBzxrwAr1WSLMoJTD/l+kKdytauBuSjmqxVo
G7QM05zPNKFfQFj99ivMxaZazpWbnOFNJjmI8s8V05O91q3ekHyGg+fDEmarduDFnYc/zVwepHps
DDv9OIfJYP491hrHb7oXbulaj7dLZwySHo93SN6OEZL2JXeijOVGfhoHvMSMr53pBUL4JvGZdMb9
kqWkUbJ0JxR++LNd0Rb4pjZFT5Uxl8+ka+tbMnMgN6UhzN3CaGkYOQ/o48LjOKK73uQ+ozsvMSbj
fwnSXZOk35MxSzMHXtAPRnUzY4FknBpskyswXYWZIm2EarppPFz1gQOSNIRfdNPKHUUWMlmqjHGp
pdbYlPBNB3TSfpl2bLlmMaZ/mqegW9I6R+4vBfpaqo2U9LAeSyTpZLH7evdNJjnUMe9zxQxjnXWr
Hj0cYbI8bK6VDUgPUPK+c/pwSfWtIfzpkk1bubD9h6s1mJ8UKG94V1sbOpF6y2bG6e5wF9NSbxYU
oaPlJWwXR80vkPcr1NP1lvFaoFTlz4o0Ii/2vJOfzXnNj7Ok2ulzPErfqsb88Sh1K7hLmsuYnfTS
YXP2m9LhQYPYdi4Ptf2k888ueRyT5HOBdpBprM+C5s6Q+bwUNsFwey+rSQBVNXtsaHpglkqQ9o76
QOZAy/F5XRt4f2wH3JGkIwPuSsGnjx3evWnj6uPX5vlL25sZuv9ya8DgSVk25sH7ifTg90Of1Z+1
dEIlU+fTV3d+aoYyeY18ockHzeueDgot3TqvkVOK3X6ub3ReuEyZnytmCj7WrXzcW5K+hUVp3rLu
OF4ka6me0yvyCc0kqSlHJUktrH/WoUUyhEtaDJ7rIVPvG/qOL73scV+u404mWltewgpnKet09vlf
2ZfGtqnoIknhpeGTbiZq3DUVlStgH0WGRwZJUniy/OnzSFrGAF/6qYpRGlsT6ZveNMIi6Vd6vjjJ
m0Oi1nJ0/6i57YhzsxwA5i5vpc4CkL6hHRTv7QKkily2sPv/X8+i3baemdMDuHumfnFv+zc7V6dn
8ueKmcwa64BnzmJsFHMLitphN46MlKSK1tCk0JVLxo3/+YiTg1GW9wy4DaSTJI2hEXZf3jMUvnUv
Y09eaVihntO+sllVSnFLkr6B7qpud+AAbYNx/2JslEjycEobG8lzOL0taS4jVzPwBgWNvQsAw4Vn
RfRS2P+f5IypXsnj8rlxBOeSvDqf0UoQ9jwx39hIfkhxY6MTRg3gsPtWyNIySdJjt0xRhQWaoKsk
/QmcS0/vMOkrNr1PseuSgkqw5yVIDsmZ8e4LfOVWtuXsjB6yLol11+0jWsXgULyiKyKUl7Ty86t4
4yXpCxbOY8wKm2fHGiDzIUmab9i/XozkV2z2S7Yv27hxy+7de44ePXr0zCW/q38fPXr08M6Ny9du
3b1548/7hieR/IzmbUyJ/18bxQZj44HN0PGzdY1kaIonklSbdvSQNMo6bIZvDzDGsreMvDC9MHPy
VF7aS19wKLApua9L6h5rCOGzSJ7Byhf5yp+zy7r+YORZbg+tJGkW40Jtz6GtOeSQVMnlS3qbC0jy
5uhMpk+wYbuGTm1NaY5KOpF8RbyRnBT99I9JbsSJ54mZzdfrxn8z6TvdtJFseYcFxrt8bKCC+5Lj
bvKUfwfOc0hzW5J0gRmSLmc3rTC9ZVHA1+Z0vdmgO/nxUT+OyNKT4mFSdzbc8v7+BUkOzl7shcry
fsNYY6MIDyTdS2OiriQtZqicc0Tv+7a9nzQWTEfzON/VclMhzWD6RCYbR1cxSQtMnnckBSuJ5MRP
csvnZEI5NrZNEWv8gR68Y1NV/07mekvSBXecPd3J8bdGY7bHdYfVtJei9JHT89IyRHWo3DQTaQ5s
YYp03FzA0o9fJTVhk/QtSwdS7wVJnmOz6z1PjbTL4SfJN0dGvpWCa/MxJY11so7K4xT9YejPt1Jo
NTI/mkzOSuZke7SYkVv4SJJ0Zg6zpCF4B0hXNgQmkZzoSe7wnHWzduDwdrPRSzcujZbiZd8SSdIf
nbw983a4J8nn/bJtT9qO9gVwmSz5eUPKjtcUOOaSpMZsn8AiScuZLB0qdvjaUL8XI9mSL98LVkrv
TbbhE5s6OC5xduk/siglniTLIkm3KasPWfZd789bdbDFT12xT/u71riYeDegg5nC+6TdNAxM47Q6
NOxEa1N+lkph9fA6c8rtmSnHkkhOPCR3N9ZHn9meHDns/9IXDl/cpsFgwwX01sVI96Kddt/fHvJI
0o0eZ551auwk74hYun1eC+tjD+TYovUpgHoPlS2/JKmC3fGImjMRRhkTKXDZ2JSJenTdIik0Z2qt
ccDZEYp+zUJJwS0wDcfupySSEz3JXeIzdcuj53eJneQa7o9e+CIXVi3eFy7pjs+S45J2HZMkXVwY
2qN6+7m+B44evR/R9ccahT49oZAZEc/q+RPSXx28322xNPRYYcMK8tNg/3kHkvTk14HkxBWFESvJ
J019E+cvnERy4iG5p+FilbhJbu98I4nkJJL/P8ndXsRT8v+2oDgn+a5rWyWRnETy/yf581dN3bK5
SpyTPClR5UxOIjlxktzLtij2T9vHDeKc5JKFX03m/GVJJP8X7MmvWEupccu4JvkCg15NZr26CUdy
3szglSaJ5LgnueGrZtauXjeuSZ7AsVeTWapWwpF84gpp1TuJ5LgnuTzhr3aREs3imuQPs7+izEz1
E4zk9JZ9jp9oaa4kkuOc5KJur3iRXB/HMcmW1K1fTaSfuUsCkVxl1zmFS9KYJJLjnORcGV/1j9kq
jkk+82z34BdrS1gUryRn7//DqJTgtrtpo4thunxM13umSSI5zkn2yPuKF/GIa5KXPd/v9P+32m4P
4pFkh2Ehkg5ntxus0djfu+vQJVoO2iSS44jkIMq94kVSNYljkoc4hL2SxCNxp1zERvI8netYbL7C
/XQ9NxV0dslNPTieRHKck3w6Mo/JPx2T49p20TT/KwkMLeV6OR5Jdrp3PAWYOx0+vzS3+wI/SQq6
6mtKIjmuSV7Hq/rmZKoTxySXebULdOYrxeeYnCF9xGaah+FB7derSdLKSDyQPJK5r3iRbBXjmOSc
7V5BnH9b6gTHK8lRW4awrVx/6JhEcjyQ3ORVp1PyjGuSM/f5p7Ku+g5KQ4O4zNz5HJIHqENyHYjt
iJtDEsn/LslvO7zqiOVZLo5J9hz+jwRdHp4f7EqvtCjhSE7d0sE8+aMoO+qOGjb862Fjpi72KZ5E
8r9K8h1771e9iId3XJM84h+IudLczlx1yOqrcfwLv6wHUScfW0si+d8leSVDX/EawdSPa5K/fHkp
OzycvrwaD7/wy5Kcv2aVUrlyFyxYqGDqJJL/VZIHxpoH6GXaDdrFMclFX97N/leXAmfi5RdO8upM
LCSXzhH2itf4g35xbU9+72Vl3PfMdUtJJP+XSH5ifuWwopXMimOSh2V9WRmdXE4rieT/FMl7Wf+q
1xjCT3FM8s+cfTkR+80TlETyf4vk/UWDXp28S3FMckjKqS8lIbxEsdAkkv97vnCv2nJkVxyTrCYf
vpSEtf/yWyKJ5P8Eyb/TNs5JXpX8pcbYGuWVACQXvPQvtJOuEa5GhS/FVcufRHJsrf+/nfglFpLD
yl9/CQE3HZYlBMlF/xV5jSMGzrh7ixZ+EZKDL/y2cdaMG68byeFn/qliGZ7RIzTOSX65tt3JP5GQ
fO/33478tnHH8ulfD/98+ovJ+zvS2+huvJMcdP3X2YP6fPRh9RI5rLV1Pk/UJH9bpETFiu81b9Ww
WauPPqhUJKtn9uxOvP0Pr7DpX14X+Yckh++NpPeqt1/Cknx60ajOzWsUSWMXTTl9wQyNXSJOOBiv
JPt+UbOAXQyNum+iJjl9bJMAh4B/doXqT5W4SyCSP8Wl8eagBPiFYyH5jlPEr2qXu0qTFh0/Llkq
O24vGMF+3t528sL4JNnf+i5IVfy9hh069h40bfnPf94uSuvETHKgHTX7dfy8Y/O6DTp2/GLsvJXb
dh9bBCf/0QW20kiJgOTwzAD25RdfSwwkQ41eo+buuq8HjyPegzi+qMT6kR6h8UiyxY3Oc3efjV5y
piZVEjPJT+xjhirvhkP/6AJejicSA8lbcF/bvihgV3na6YQmWalZ9XS3RRD4wvqatX0Qr9pF+Vje
AV3Il6i1i4yMiIXko/9E/ko6Kz5JfparcVWaStr0DgDlht9KWJLLGLUJo7ZfrbVPX6CFZbKSnC9e
Se5B7xgdB5MlUZOcz6iRJ0l6fPnsod27Dy+GaRs33ntZ8dfTJL8ZbyQ/mt84nbt3l+/2hETff+io
5RjGonv4vj65AUyVZl5OQJKb0FrS+YO+vr6+O7evXbNmz5EJ4LN+3vgnLyJymJVkl3gleUmU+uW3
jvmunT9/zjAvnD//uGVwYiU5sCiFWjWsW6NUvuzpnKPN+rq9rPj6TFU8kbyimqvtNot+Ni8qzIVx
yopHhGfGgR7FzAC5ZyYQyeFHK1GwU+M8sc2rX8jQvdfW2y8+Sb7v5Pj7Vp8JAz+qWvSpIq8nEifJ
H7s+exGz+0tKn0KV8DgjOehhlH3BLQFyRNxptSiWFt+KzoC57vjfbIOH/5axnyTDI2FI9o3xA0cY
I9zyxFxECl7bIpuT8zt9LkbuCs1i7X8svki+Nm+gd62n7tslZdpsGSGf90d3EifJVQHSl6/ZoGXv
LweOXLhqx+69J67d/MmZ2g8exiriyeLPvIvk+6DLqhg2rrnmHHHxJa0k/5EnytStNxQvmHZwHjCX
cgH4NOodpsfTDOBS6esDNm/r+U/XQ40vkn8w20OZ7mOXHrx+4fr9R5IUcD9Qo2FHSMzTl2azZcno
Y3sOf+lVwbpvTXyRHLmw6FiweqshMzcdvRAoSQEusSjPEe3y+mlzfwtIMJJ1+4Aplpfc15hi9Wiz
TPKwfUmPgdG5nWTKcFxxR7JW5opYot5vwj5PPt6+WwJch0+rZMYUpbTqUtxuhfzav5IbgEfTudcl
qTrDE4ZkhU4nWczZQ1AGYiaCDGoFFB04ffHwvFDUOgTXjsBqfHyRPCiNZxHou+vK0/aVBlR+hoy7
w403pFvtVWEJRLIWEsv68l2IrTTznepg59158Nc9qjhCsolRbAdzTZn+VlySrNFFbFPQD8jTDzOc
flgUM+3Dzr9F6shMb3WtjIQeGVvBEXDoekon/6lZ8V+Y8ZWmTSwdOxMzk2RLeMuoZB861hXHxdap
V1wblGPTk/vhHMsi5AhSxy5iRSqANM4A+X9JIJL7xWrudqDftPwpyn62IOpjGfAO1LfGw90cnh4a
R8xjZ5syxVGgXOSMr2tZYwn6iol5f9sB9S13itplMNcLPmgXmVcoyIXlkXe8vjqA2QOX0AQi+bIp
1tDIufDthO92RFXhxkLNCHxOemM2BpOsERkF4pHkslSPpeeC2L0/wrqAudX2e7KcmV/FhLlbSIKQ
3DqakmmbuUZmru7pF6Uro6KYwTqaIszHP9vH0YgcleSw1mX8JOk7kvmrE0Bv+RXF067yoyJ4BERO
9aPZj0+Negsgy5zbCUPyIlLGNg9eavUKqLE5wkZholaUN3N4LzzuS9IE258idzySnDWmDVzSNvhT
D3du3nsy6leyNINSEU4Kh8tAeb+EIPn9KObkiLY9yqzVY4F15zqeUvh3Jjc1uCBJt7Kk+EtxTrIs
HXL/LqkNdaRzDgB17jz8AGiQB1Zb+8+L6f94tQ6Ae3iCkNyVqrF1HBuZHfy8sacG7tH06fDS1Fh8
XbprBuwKJ4eg+CPZTGzOekfgo+L2AOnHRY67o+HDKLcWNtqRFglBcs3YSB4UmwdUIYo99drY6EAX
SWpi3qi4Jrlt91X3tLbEI6kynSV1BCDj1Gv5geQwzNp/AAVjyJgFVZy9E2ZMrh67K2SURETpVt6S
dNmOp/J4HAIc2oSoNcDOx6MGhccfyY58G0vPw1GgeM+mGB1z5L3oXOx0oI8l/kluE5vTT+loJJuW
S9JPsO/pfl3wlKSpqxXnJJ/uX8Dx/XnhkvIxSdKjTBEeZm0LQERuglbEDBHZAOdvP0kYkovE+p62
ZI/4cTHhvlcah/3T5qLudsAgHXvazhgPJOeO9flbY73nrJXcIIfV7d6LDDdjYMG8hJjxvRuj30kT
rlGdU8tKUtNYcoRvhCNxzEYU7eJQvzwPJHkyTZK2mG2qfE4AW/h0U2IWdloP8RTtEJPkwlHnFhHt
N8g5d34jp2o5cS4J2QP1HjEzQ/b79AMKSh8AqYLjleRGlIil55RoA1z5UEnaEouZK6BA3Pk7/T8r
XLoY/QaCJ2B6p8si34WA/V0pPAUxY+0DnBgTfyRbWzYmSpLmmKP+rt9HmLJikjwGh6CEIrkgsaWx
a41rx7Tlek1tb00H98sDx1g1005kkX4F/uUsIs8jeRrmWKxwnxqrNu/17F/O3rbY/iFFY2oSHckb
/yQfgacdHx+mBMgz5Kyk+x/jAFuk/cRWcrIUjeOd5AK29/VPeY1ajV07RnEHaBxLXrrqvKeEIrk6
TWN2O2ZPq4EtDA0jZ2Zgxmq4GIu8bnhKqgDkssQnyVdNsT06OQD7Pvf9d4xpmRtoIOmGObaOQ/CM
f5KVnbFPdRuOnXePLX8e2f3juGYpAZgmTcc5LLantEK8k1yZ9rbp/S/jv111Tgr2InPkaFf6aRFn
7ZmcYCR3j214qm5T8UuNmJgPB1f6diWXYiU5j81kZ0Q/HQuJF5JViYIxJpgHwBUKlLG9C1Mb+uXl
xEJyd7JH/3VuuVN9R/vI93aawgyVesTqIFuLSvFFsu+MqX9bX11Pz+n8k0XaB78ig24viqa9tyHl
/Whz65XxSPIGTDFSJm01QGi97Pyi8pBsdTHavRt7itNO5NbNFn1TAs2lkwOyMiR+SN4OM2NXLqK0
+9IoUsYism9EDo0bE+7GG8mHTdEnmmE1ot9uxQst6SJ1jGVmKOUmso7S5cmP45Lkv07sMPLGLrGu
pEYGnU6INCcrPYxJwWdRJJx2jLbQu/kd7PbHH8mBKWKoFzczAaT8rLW3G5jqnFVueuYl1oJUVfHW
PuPvYL4pdyBjaLyQrPdxe0rrPOEA0esInpQ6Els6hI+tS4RXuzrEjfoZe5aABqS5Fn26lyNttrfz
pnMBU6HeB6QmdJJ6USYWfQ82WDcvtDHHQcRIbNrFRZgn/V3fzjZFCs0De63bD1KBa7RQIUtV3KKs
+m0GqGKJN5I1AlP0ZHuh7+FYzhb5W+83KcieKelidXF64MRg6UOsKt5MM7Ayfkg+60rOaD5i4d7G
fM+WSyYXHJOa4xWLyDzWEPscgN2xeCP5hCulIx1w5tnjZSjEgdfuGjP+snwjjYjNw7cTyWwjowfg
eD5OSfY3tKCeUCgsNAsRqQy+JNISNNBqt48EuXfkqokk3asDsDb+SA7Ki8vWqH0+g1E6N+MT70qf
TPaTpD9gl0esasMPsE+67AJu4CXNioun8Bn5LmZD0SgjnKU3eE9Z9lvA4bHtPny/cd/Nl+CM1DK2
AW4f1tDFWSkgVg+OuCFZ35l4yxpLHTrCDtPTKrwHq6Tvn/JnkKSLzpEG9AluEBdvkkiSf8000Gpn
g2GngSJGj+9NUcIvBjzlN3a8ItSIPg8YD6S9EW8k63hyXGZF4BfUBZpHh3EyTgHZY1tq1UdkM+Z9
dL2wYL8UkhHYEj8kazjkPBfxqTcQzcttHfhLw2Mx4qoOGaxWzysZ4sYd9Vk5iCaYcOnxe6Dl2vel
IcZq73U4Il2JJVinDskiiTiTEuKgIHQEyT+4Vjcu1ggwtQCrMrPDBYgoNjkOikARKyoBY5yg3lNu
tpZixvwpvkjWL2mhqtVV0PctKPOUP3olqqoKDWJKO+9IT0m64WazbIyBWF/ncUGyxkJ668vrYTMg
T7Sjg0gnaRvEcLk5YGK0bXsW4Hoyvkh+0KE0gMkal/P0wugy3IIlvU2zpw5shnFRPo4B0t2MK5J9
zP2tVsBvwQmg1FVJtyc6AZGFXX+GM2Wg5z3J8mfPNJBqWoy38X4z8EP8kay+gEOreb7bvykBNH9q
lea0HfP0BVnDYzHWOZ+z8mA6ZShYqYEd8UPywzYNnaHy1L1/7RqaCTCeqohWghqS/F3p9ZS8sHdJ
FzH3DysPlLHEE8lN4K227hEz0lYxfs/akjQKl+jqhX82ikf99YPfBmrEFcmNZkfscmaMN1BleO+m
Bazzj44RdgE4NApwKlgqLWDXMrZkiD2B9DfjjeQpkDtd5Ix/+VPntMP9kf62i5kVYwn0t75GPLEW
IZwElA+PF5IbwKoSUU0VP0c9eslsmOkaku0pa8qgaF/xtLuxHhEfJP8IvKegAwsXb0gJkDb62/h7
60rwLbenbBPNsNsbbcdvDsCSOLddhDswKnRoZIRnqqhx4Bfg6ChSGMZ7jx6xv9eCCoLxeMYHyeeS
wRD/iUXswb0Y0OGpIdmJXpKqUvgp14qHGchu00MG2+IQHycHNsYHyb7AivBlVR2B9GmBDNGIHYzz
HUna8zSna0zRTeOzAKfT8UFyeF7AyfjJhpbh6SfovAd5ja/QC9O2KAdGEWOSMhyias5xRLI8GCLZ
k9HTFexoXjhq6r2LsGsg3tend3Hh2fGT201gWhs/JFu8rVPhwEvXw+oAyaLNqANKkuqmpFOpn7LD
WZpChBv+kywmq6LdC/COD5KrAyMkhVw6f+2eG2B/KspRP3dbkrj3SRXVsX69G9miWe9CiwCVQuOB
5FUA/GpV8h2AdFHu7EIeHK3OnA+ykTwikMcy0ozX0245QbmAJnFOcgqmKQgWbzFDJhZ3zB1Fgw82
s/hrikgjgQX/V2rmgHgheTmAs+HNuwGAUlHS4AY3gKWSpJHYb4oKck+iBgB+Y3ssr7gBB+Ke5KNA
RFb3yfCU5tgZlysR06OGkfuXu+D+VMjkLtO/7wAVG8nhhQCYKylQCn8b4N2Ixd0z2aMkiZwOLmMN
1cO/JeSPmaVqI0SulcQhySN1H2amA5atvifLqShMZmPIZPJop/n/Jmq9kzxufH5jWeMz4vB8JEln
2yUHKB4xaIXUhh5WAxIk2xn5Z+kM+aJ8rTspc0X5QerHPcmdo+aiq2MocpEOOqttlAc3jmr0DOlp
InOM2N+OQMpLcU6yD+QD+kvD7T84dd8lOUAhq6vAMndMg209H5UCyDnx1yPbBqSHIrGpEQ2ADLfi
mORM9NN9yAJw4en+Zei3lmQPc0EsBvDINgJg8D++q49ylWzayLtwUS+v97y8vN9Z82ySp+OcMVI5
fmIo8Dmtz9jR0vCBYe3eYQ84zbBO5s54Q6Gob3PNtkWHHzCD6R/PRrYWzv9BixrFC5f2qlTBy7ts
vdBnkBzgDpDT+un39Ia7egfrI/hjcnIZo10rIAPUuyIp3PddKHQuxjVveQD5/rH/xZpCBWu2eL9Y
4TJelct7eZdp9iySy1G2HlBDs93+1953h0V1dV+vKQxVKYoFFcXEGjHYG2qMxhqV2BWxxl7e2Gvs
FRvGHkti7LHGJPZYYnujRmPBjmhEbFgQBCkz+/vj9joX8Ps9r8/D+Qtm7ty758665+yz99prA16H
1jK7Vbfhp2JOLw4B8gjJ0W5A09Z8ocYg1RqMG24AQrLNv1hbPrhVeOOQ4FqhDUND69cYoI7kQEyj
hywZQcE7roKx14CGAOCnE/zJDAYAtko1OfrQj8vXbjp42WjV6k0ZlaajJpIzAjHQH96omBjW/G+i
N+3yugKAtcehk6cWN7MAQ5jY4t38gFcIUHPHY6K48W5AI6meSOq3bLdiewkAPsz3djw8u2v58o0/
n7tjlCLXVmb5PQ0kb4PFC7BwP/JvYWyPiJGnH1zaFWaC+xluIYHr01aAW4exnf0AdFFrIPAjAExg
zX9wesfy5Zt2nI8x6jo3kpn8Qh3JV4Afy8KEQnesAFCDLo0Xf6qGoMay1wR8R6c6+wHI318rmR4J
gONQZsb++fPylZt2XYg1anRZqc3mDFUkl8BseiBtWyQOcw5PZ2WsP9e7VCHAYzJR8onZrcq78Fd0
/ajl+D23nYY/V8pu7hpNJG+C+Q+gLVwXAAUyiejZRQvMhfjS9U3s8Q0AVEzpCgDe3gBsi7VCbecA
uI4gurt+YA0htmf2q9VrxRmnQvn2wlLDy2jNyfXQsChEKiEzpZ8ryXjqR6wA6pF9JqsoWHyb6lUX
AqgbS45ba/tVE/pzm/OH9vn+L6ez3jtv6aWr2tWRPAZ5UgujItANANCSaAuA4aN9AHgIJexEiYUA
/EJEjrhYnfDEWACNH5P92sqvqwgmmAt+1u/HC053WI9l/QAaa83JMyiWOUIZSmuAPlQKMKsExsUj
Bi5lol+ta+mqpogX2HPaGd1qo6qyD5zUQrI9GG2vA/OBuoBHOhHZawLVE8YVAGCrvThRvC1sQnTw
MzMAWNpo6zmNRNGuj8+PKKHaiqDRiI263YR+lX2ghYaffBRY7ydW8Iqe3BVAgaZuAFByPjNXX83L
KWE+jmpVs9HowxozVgd4TM84M7SImsmuTUdu1XVHN8g+0EXdu8gohJ7khpFsdWRILFF9AN0oLebM
JcnKPRFwXrR3zh5q8Vye8Ue/QmpGe7Qas1NXCSFSRWNRBckfYQid02rhEoH6jjIoHgTgW62J6TXN
3IvpmTN15B3h/uVuzYqp8/KDT2gheQ8sN08DBwCAYXeuAlCb6N0/F+4KU/+7kgAY/tiTHYuXbNOe
Kvq+aOCRfreyXnu9UtO0o7dNZcc200ByTQQ9NgmrKxElNgNgTUyPPn6GA979ILjwm1nt8duTykXS
r1bQM7ncnFjNj9eUHdtJHck7gAvJQCXA1KwUmhLRQwuAporzxecBYHqlZ/HeW1Tqj6DyGf8tpWOz
OWRhnOYZSssOHqqO5GAMcHwJPuQiHZNRegNQB9Cs731Ko2jwHbcyz79w0nExXy8N2eVp8iN/00Jy
U4TRFmA9AOAAET3JBzC0IPFg1u6ZTvc+l6jsytlY8l8nhptqbVA/wVub7MhQdSQfBtafEzm3RM+Z
ZUgsp5TyKcxdACe9l0/R0si+2HjQicnmehps1WcmlWVEBcktUIX+BQD0o2KYTESToEpSGQQA+q3P
e0dQx++6mvdvdGK07Yvf1U9wV35kD3UkV8aAFewRyo3yPPj4AYAncEz9MuNP7t6w+v5SS4nyTruH
un8Ta2Sa0ERyggX7aA5gAWoxK1pvAIBst3yZWRw2OfFxKaap/dtar4Jd6jvve1p5u5qzvwfGkNwW
pTJ2AiV5WlNKdbibpW170prBtLqlymMpGifSqPXtuDnxJSyNTE5NrqXaM309DCH5phUraDUAKzq9
sGADUVoAAFRRgMwNgF7DlPuN3h7L+2bV4DsBbo2c3+cGh9TO8Z1BJFdFfQ8AMMNd+XMtZz/bBFBd
QVbfS+uQeIro4U/duud1bqlHV2WfimcW+VF7NZC8Dj7p1IPx6IELRBeZH1S6LDmqMWc5o4/ka99S
+6XxexyHp9TyM9DDN2SJ0jvqKz+otiqSn9iwkJYDg1GMiQSm1IX7QkDM9HZEAEMSbOCLKtXG1DG0
YwJR5r4pbWoZMLnaaqWj3REqDpESyX2Q/98IAMu6ov424DbRT+oSZMxiPlDb5rSgydRztj09fe/Y
tlUMGF17Q6ZTJw7d1JEcAsACoLpa28C1AGoCrt01mmKk9Hr02kGvhnvVftXHUFdncwe5duNqxTF/
aSC5MzrQBleY+jREZeAoUSv4lQRwSxmkAsy6O+ITD2joxJQndLAGjhwtZMjy4qtk9zhD0bHuK1Uk
T4L7K5qMAgcBtIgnoqGw7F8HFBKYWjQVaJm5AlASn/jxw683vK+Rg/ZUMp/b52fI5FLrZXNTSl61
6U2B5FduaBgIABdnIagzgomoHkoBKCSz6SRzktUaJo9rmU5r8zxNeeHYWMZ6a0seQ0aXlytbv1DE
EUarI7khYBoNVGmiJl/5O9B9NhD+pboe22NKekLJ4/I23eGV32iHco/h0iBpPcURSepIdgRg3lQT
0JdJxByjI8AsC2RiTklMWZ8TBcPfCk5KO0EnavrM6ulvMmp58CnJOfYpDpigiuSP0JVoLAIdA6yA
39fnD5kQSQPxaTPhl9hrQeUUqqPq4HHjjH/yczv9EeK/oEMBw+3gq16QnGOL4oBINSRvBABLBBCz
GhYfTCW6CXQEFKt2UxS2ArigYfLvpgGOtLabaXf5wBXNDQME9aUc7cWKAzaoI7kF0PsgEFlGQY0l
opTKXnEj4XorWM45Y8b0+ZS6uECFA9uLIgujwDJRdDdB4Vz4a2SrTwLFAGAO7QWA/2YE49MzCmLn
eCAQ+arqqsq+uJ8Yf4QuNrONutXdnAXDTWHRsrSxdHyvhuS/gFYHbw9DINHTJV6AxQ+eR+lT9PuG
XwRjvFDhOcWb9NJPV648IDrzhdukmx1MWTDZ3DFGluKVjm1qSG4LoPCBk8CDA2BmiknwnCtKfrEj
1oTuAKwaiSSHY6VlFdGx2p6zrzTNCkAsvcXhps8U759SR3Iv+Cf9APNNq3pdTSZ1wCDyV9VjI8rc
UrTY5pg2yOIovkvuDIjnEQ0kDwSA1sBWJiUZtwbYtQxWD0lh4U1XfBWEvhV1+yR3gmX/o86WnnGr
/bJouLU7v12wBynePaCG5KMcpoKqtBs240tfAHBP9cDcWfDl7cHfRDsAWIdrpaGX5tkSG2btE7c4
bxZNtvXn17g05WR+Vg3JBYCOL+gkEPsIgL+d0gqg52goyFbz4DpXNaHGjP71r9+8f7OJbdjTue5Z
NNptGE+GfqZMUjxUR/KDvofpUaP+MZo+WsLGZ3azRhBujc+ypFluyPpo8JdW5EJknxTJFQHLunvA
30S+AJJLoB71QLmKPMeM2dJ4nQR2N8JnehnnZ/GOAi2vXq6WDcPzT2OzPAeU711UQ3LGxCpiR9wG
AC73gD2/A2zNcRPAO+LgdKYjwzcaaZzMZJ8Oty5WyobJBSMzRBkj6YhRQ3I/cx8H0Z/AFSoIdCfa
Cfy3MwJE8hFERI4gtBsMAF3V6yz+bV885qlnj9jjpbNhdPGV7MK9SPleOul1ljyvIh7Kj5caNYVJ
i722lkO2hiXsHyKif5VLZVt1JL80wX0b/Q3EEtUCApYCx6k8InqLpUPirQgaBLxcBYTq8cXSUj37
DbVkz/ISUanMWqYY0VpcuEdnGiEoclDLsmxJXPU9QHQyL50V38EKwIJq2762AqZmh9R8jHevXUb0
MWfP5FIr04lEvY6FEacau7ATEZ0F/qR6wFai5gih6mhnlTbWe2dF3gIoUxLIO0EjV/A23jypqyl7
RpffaFffR1lIF8nH1ZuhMCMOOKzy8kX3Pdm8tQBg6p9ENEf5eg91JDvCCpwjugicI5oOlCuI5pRs
wqJZcBOiTXctAOC6P2m4CV7j7mt9n+SI7h2zbziCTrLlJrLxQJPVSaOYKtj0k0UAoNMK2N5RFbTm
F8YpQQBga7p8eH4AIT8oAmiJYUOb5sDkMueJntuUr7/W4sIR3QZ20DjgPj0zYwkFo/fHsl58kQBQ
8+VwG2CbohalfVl/Zq0cGF35KtF9JcK89ZG8Sq+U+5pa7unZIlspE3Iyih5R2YOIYlMqTPunwBFi
RJDNF+kYcEpaj7zjm7omAC6fFARgqr1ClfBxsXT5nBluGpB4WeUMz7SRPJbVdrtoQkS9ejHDUIRo
BmzCsmzfz/56o5v6AShcq7nE2btYKIcmW0Ykn1T5Iu+0kfwU2EQ3qwwlWgXzEyqLAfXQUjb9NbIA
nv85Fu4J+C+W3+vMMz5lc2a0dXr6b8pXC+kj+UugqyaSL/AuHRERPUwicixtlQ85Hp1iJrrIX5ui
h+Q0pk1caaYCfDG80l9ZZFyFGvDl+00iYIZ4Yk6LIaJrMy05tzxg++kQBSxIG8ljWCR3QplMIgpB
S6JYs1gb9xmwelE18S+/kYgo9T4R/T3F7Jdjk4N+O6LwBJk9pzqSX/N50s9RjygQoyNQTkn7BeDS
43hHG1Caj0m9iSOi08Mtvjk2utwfuwPlr1XURbLDF+r6luzvABH34KA1756kyfkKak9ZRQxbmnf5
KXkMYJoeklPdMIWIdvnDdIFoACoSleLqRFg6hRvmJO8bG8YFBs3h/LqXGYqwt5s76wC5gKthyxtf
l4fhzDpIbsroW8S7MjfSBdOIKFSszfoYOE0U/11NBsz+Ie53iehdCMLT1rVx9da0o5DVsMlhN7rJ
XvHTQXIiE1YkemzGGqIATJkNmyw7tBfoMtwVsLRb1xhAaaZmOfkj9MuMauGnbVkx43N19ztyFvin
uki+qEdpsgeKM5IvAgCXGWZtW9rdtotIGK3HuOhaWn1fT5siKquJ5JOMKuG7QviKiBqgOVEHTj6J
GcO5IMzdKUEMY+TjJ0ImDfWaaRsesC5NREzwXPqpruGe0zd+LP24DpKLI286EQ1B0XQiessohKyB
SYj2ngbq24mILk8pAwA17rHfBs3qa5tcYku6iJbgs0I/TJB3/tri0l2VDpKvMkRgohWwJhCZEbkL
eKTgW3rFPR2XBzDV+LwQ4LmBWDpMRDlto0v/Yp8u/Fd4WaB+tGhpVEEFPVkbyRPgYoIG54sOAcjD
e0HTAUDbTPNcCQ22ZCKd1w9vWAadkvDoNukhOQrAz0RbmdBmJfQiWgibODPdRuBZ27fUrZG00Ks4
G5t87alveeVYonQBnKsoY4T+hrb03kjx+llaB8muwDGiODemei+Wqc9MchfF8M8LzNnEM3srMMVk
8S76Jtd5xGU1weSI3vXXn+0qHJwq3qpW00HyfsDykIioMWoSEbAiRlSfzowZDBf4dWRJAJ9OsCGI
iG6Z9Y1umkD0wof/+Y/Ri3D9abnmn2PFAekwXSQ3R8NgFfopM/qJg+IZxfRnqq1ERPaPODtPEVFi
PScuxuzNZYT/DuohuR+A6kT1mG4tRTCY6L9SHaGeACTN2G9d10x7SsbnSZJMTTMHEW12EptptK+P
VUIg0kByBoBJRDPg/ZqI6AzwJ5NMExQNrwCw8VP0k9PqjFfpaJkq+V6diIhWOlm4vzwQIXyp5jpI
/p5lVCS6YiERWbDJ4Stv8zIeQN63RJT+Uzkg8nL73zmmp/bo4SAiGsf9O0yNSy8f7feFCV+rtx6S
k1yxaDw8Nbokl0ETQURmp/41l0npg4zsz7OCTizNM2El77xe1kNyU3gADy8wfnuaK2YQZeSXFMOO
ADQE1T/RDxOzmqpszZg/w4Cf4syNC93ZkvuzjTaSHwNoQRnFWV3nHWy8ZSdMvBrOLQBy2We7/rpb
gd1fsXNLIKMRO8yZyY128QTLXjpIHg0gnMmnxBBRcaygqvKyosEoYGU75jhmshq1b7x0r85G/+NY
z78cw8kNd5Z1D9vFt1Abr4fk48C5jfK0Op+rAdYH8tuqZrpXLMkGQlnyfrlUzeySnIN4cjjrLsfq
Ibk4wk34sT0C0ojoGBPoaCVB7mhAtYkL/aVvAKc4ecIMEZkjw2nix232vrKiQLg6kmMB+GZGM8kT
ojWwpBIRpecXIrRXgErII2VXOWHUc8x0JlJlYlf+1OLOTPZcvJvdZn+jg+QIMDTOHqhMRBkBmEfD
ECS9p6NRtTP/m3OkPf2Ln5IcZmH/TXBKlPNZt5HdxM/WQ/JCuKWfA46qInkXcOtrToAqQX+95UNo
Vz0B2LgCr7QAp1B2WXSvFQBxM2clku1WRIWgtSsjXzmXmUkXwlUEgLkI8OZ03yRjqP6qkC7h8EZw
/y1wvruu/nC1L8CKSKkj+QYAXKIZcznueFFuTsvH8Z7vAQuFRojM6KYfCcyUMHj5+XySAbpZ/Hde
YDweTSR/CQvwhKgsxjDJsSr0HSzSmtfpKH5Grs2nP9XxcTx7HQCCUlRf50a3ejLDBoBJNGohuSMq
01sXtqGZfHyNUrQBpnjee9IZfDXpOjOAgjzfb4iBeMtnB6I8IW7Dq0TyNeDgZJhgfsg498D3RNdE
tZ5Ei+A5CAFKknmGPg2ZvysvygPAQC5hHGcgXuQ/5c9aYFGojuQzgE2U8ongkHzNlO8hl+YDDnaG
i5jR+NZb97K81m98SUD0EFw3Eg+fdawSwOrZqiO5HMJM2E7pHtjNnvT4Gbl+zxQEUm1pn6cEfRoO
vwbdLQJAQNwpA0YHRh0pD7YOTwvJZdGHqJKGDnIdRNAVrpFBmH5ShqMwPfHIG7m5qbl4smQBdBq8
rwFYr2VqI3kLcPccUH8M67+bUIGIgsQCSHNh/YepW5cOJ+V604RnrtmWafkF9niwEcNdWrkAg55r
Inkz0BtuPFexEl8e8BdfuvUQOPTYWyLG4MS5WCxM3W22TvQVpH8DjZjs2soCjHmpjeSCGB2AcUTR
i+wsG3tM2sziUuX02ShAG4Abopd+1r8sb2Qj9Ng2Ko/pAJfQ8DFitEdLEzA3SRvJbz3xA1FbdVns
DHfMIYcvuxL5617JN4VfLHYT0RRewSxG7eDAwMDAwKDg4ODg4ODiAp+nnjaSJ8GHyJcVms6wIQy4
RDSKCeVyEQpXqq/SMtCJm8A1+rzrXiuT6FEA7xB2UAFBQGBgYGBgueDg4ODgcsLEaT2uheT1wA0X
oW9oPpWOFv8Ch2mphG7mxE3gCoj/cWlMRDF+/GaiiYo3z5hcPjg4ODi4rOCUuv6theRMK5bUFZTG
VgMIVtzU7wFK8pL0UPqPvtFcGO+gqT0RXXSpT5qkSLgXCQwMDAz8JDg4ODi4jKfg59/URPIx4DzR
UJnOurCpPkLUlOmWGuds/1Omab+RY7+NKuFNRPQE7e7GP46NvX93nQoepJGSzKT4e6c2L5u3qoiX
NpLboybRZ6zC2hngqBemEMX3FpVfLoGZDqgwniKc8BL8a3QdNmrCjEFMBrkXTsc9+ff+vXsxKls+
WSfbtPgnN7ZsWrCkL5ZoIXkBrNQGH7HZ3HdQKbiPBg6QvRJChDKEVk5MLliz2/BRE2b2YkLwbS2X
4p4+uB8Te1tlyyfTRHz36PH1zT8tWBqBn7SQfB3YOwmenHzNRI4iJd/dEXWU4MZJga9LQJ2eI0ZN
nNmeKTyu6XEj7un9BzGxN1RqSmSaiKlxj6I3rVu4rBV+0UTyfNjeEs2Fq5pUzwy4JhOtYBQydxlN
MnqzWNMZV2937DsrMmrRopnjevXo0CmsVdsBQzp0aeIToI3k0hhMNAGuaQxmXd61U7SE3gC8cdRR
doouY9Ty5WxNmPYId8zuPDpyUdSiOTMG9YgI69jiq/Ax3Tq3qIb1Wkgeh0J0gJ+Q7gBwkUc8rwAn
iXaICnKpkFGTNxEpBXAkY2Dm5M7jIhdFLZozfUCPrmEdv2zdbUxE52aVsEcLyYeBy9eFkE47lIQy
dfYDbESbxBwuu6dRo38jpYqTdExNG9l1wryoRYvmTOvXI/yrji1bfD0mPLxJeRzTRHJbNCCiPVKe
EDc+RnsiemLFOpK3DdcbcURsIbTWiG6mG3FU04WzYTnRKVayYAhCaAXcZBSsrUAC/SJrSENEXkYN
H0xEt2x6R0Sc0HjjTy0kd0QtygziqNd/MJEMhR9/gcheWvCUMw3f6wlEdEGXbj14r8Ybl7WQvALW
VPqE36bWRkSwUqTqB1iI3niIWk0kGDY60un2afoGjTfuayHZUQCTiOi5ShsfoutslOULtCa+r5kR
FtPPx5xwgNsqnfwvzOKFV4HkGOAwUYYXE7ypil50QVEP+SNMmUSKQr4k4/zHUcdW6WdySilzlgWr
sOhUR3JldCSazTbxpCjYzIqWyYeZh/97AeNxhk22TTy+RJ93Vl7phharADDED1UkD0NJor58ZW8x
TPoO1tdq3gUNRr63IraGweEx8+R8/Qm8ilJX4OOSACzPtZB8jl33CqsVpa4CHhMRTUZ+O0u6MGRn
fr3UuwaG+tYAqm58ronkHUwNV318QURJFiykFBu/V2PHciCVaCNcpHIEGYatKGbKsuH4ZLgFmHxO
M3aRD9OJYk0sp2QIqn0kqdli9/xviCjVny/+NT69BWb9XqPKYBMQeUkzdtEWjYn+YEq6iN4A2y4r
2wtFwUJEd82CfNWd/69G1+8DuEZFa8YuhiNfGkMVUelb2IHtQHsEuEY03+g1Z2Qc2Xl2QZbM9FlW
ChDvgxVIngJvOxGNQwE70V/Ar0Sfy1PTK2CyEyX5YpR03bEZtKLA2/u79t4pkiXL239jAtzeacaT
45mwYG12uWmMjl0VPSSXAxkMlSEvG7p8axgTmXd2/n4ja6Trnn3B0ZPVkRyCgUT2oqzjcAK4leEv
5c8S0SJmP9RM2JY8MmpARce1nfsv2LJk9NBO4Apg1ZFciW1iOVJUoc9vzPOwDXRS3fCdU9aFEBlO
ISJ6F5AFMz+dZpNS4ZRI7sGwdE4BZ4kWwe0N0XIhhs3dXF+GfuEjXQqNlkWuZcuGjQ/rxJrgY1Sq
SP4NeE5Ey+DygogyvTA9Et6yGz0NTMzmoQvfza6gQQN2CNEFg8N1WgjA67ypITnJBWuIaDAKZTJZ
yTwOGgJfWZ3vbKYV5m7BJco0uOMzn2azbsaH56xSQg5LFcnJLizFaSugUP88yRS/E1EjtCZOn9bp
+J2yCoiGoy38llYLyVWYNdmeD98ShaGBxD6en1qUiOiJTcbb6mTMjNpMdjDReI2G50yGdxGqjeQZ
DG4fMM1abgK/HwdkGqBDuERuOxR8ZyTvy48mTD7yqYfx1W92SXFiUw3JZxgFgePs1nk4ShEdVVQt
D2SorGkFBFWG2sZMYKs1Y4xPyoXnFRGlNlWRvJ/b6F9U7vhpFqwsAf9beGaS3ZjcAgfBNKPKM6YR
X4uCMxpIzvBk81rt8Dkle2I6EaXmkXV5GoBPiIioD0cVE1FpDYxj/Pc2OMqu94MzJLdHDfZJ7EJE
vwJxyTamAZ843l2LZzr9xJWxGprd/hYqe4yNkHV54AzJ7KyWWZSJXjREK6Jki7wXYAs20DQSnonG
CRQAXLi60P5Gja65zAZnSJ7EeJ9ESWYl86Ilr8T4CxCjmvJSGbsEXBnbfC/kA7hntZF8m8P5JPg4
DrHNGavK+nuHsYg4JmPEXTRkSHUSaBCGRp0lHM+gtTaSA9lg1nh4pRJNRAGiFvKEaigvj1AbFRzc
VzAwvhAznI2MxvO4ibCbNpJnsCJw3Znez95YRETVZPf6pTvL1Yox8aU+ewzZ0E605hvbikznooxD
tJFcgwfrx+JWX5zbEylwbHcwSjnOKTV82P+EsXDBkhL831e0kXyAK/H+Dbi0HLZkJjv6kYyKxXyb
jMIyZfOyRiwRcmH1DFnev6+M0KOG5BfACq4u5ABROKoQrYVZWmxWkg/K7uFIiXZD2wyBiVbR0Oo3
TMh2DtBGcigLtpUwvyaKZfzFwTKNw2UwsY1FPuP3fO8Mrdp87YmjmKGo1kQh3zlWE8nvvDCDn8/q
Kuv7zgt7pn5EyUYcyNI80TnZiI5S7SjRUdHaSF4IC7PjSHTH/AFs+d6Psq5UX3DdwkbCP0XqJxkY
K42SHhgXOVKUC4zQRPJ3wE2u7ncyURM0JboJ7JOxW7j1MDOQaxo02ojJWwzyVpmRd1EtaRJIHckJ
VvahvgNsJ/oRuM/4ofektQuNBeYv5+T0MGL0viztX/IvCZamgdSRfFoQVx+r0BWNgkuqUFcUTEQj
DeWqwxYeenw/euegIAPP28gJ4rjiTW0kt+Yf/EaoH8qujY+kkHDwnTFvmqVsg6dGdtWmcuM3/xMf
eyqysYGtSOlNJRV7GBUkOwL5vF0b1CaqgR5c4l2SgOLnqXlwYSo/77kY8ZMrTtx67dG9E3PrG6iy
rrhRLEb5H00kL4U5nvOM+hH1ZBoUJVslqmpXBDptRjG+ZvmsoUm22rTtN+Nijk6rY0D7J3SDWNBu
kiaSl8OFowSvBeLl0eTPRXR8awrRqyC811F0t3SLru1dpHnyZU6z4OXJZcmKShoxHBXaeYShvENW
CPw+h2nABkkGXNO7OCv84IvhlvGvC5YS0UCpfv182BJ5b8Sda7g84f2abB75vYQ9rO1dVOebIPdF
acrMzz52wUwRHXE7Us9k4W+eB9n7/RptnTxPkojX9i76CDz+a7IW90RBIpX184xQ/Fn392impXuU
LLyhjeTdgqfzN8Cr1rfCV5LSan/O/zxngqRFgaP5+7zBHy1rJp1ONJE8CvnShVt4dgEsD5gNtKh5
nf1jcXp9IHyZ9G9GvfdpcrmVDaRZNU0kRwM/CHWbj45yMZ2vUUbklvqJlpVoE++wp1Z5n0ZXXSOL
62kjWSR2mSinOz0T74HSPJgag72292Zm8JnJ8pdiNJHcGR9zc2ymD+D2lstQivT2b1vY1YdxmWtJ
vk1K3fdmuG1CtHxt6qOF5E8QznvDeTG3DkOeTvEQqPJEhyU+0lUTJ9KX+P5Q4T7jknwHOVILyWOR
j5thn5qxpQOKZLKxOdNjMcNF1E6gpsAvel7mvRmdZ+Gf8h3kdC0kvzQhSlQnMEjy5hGJ6FptNpu9
7j2Z6TUn/Y3ixXtaSM7wFmWym/CVxbRLvEvsBlehrHaTvHPDm3LvyfK60TRc/lpfDSTfEVew1EM1
brqrKTSDJQpHMbsk68pR3BNKvCeTG99RyouO0kCyvZgoiFUBbdy4LFO0OFQSLKlniIQbn1SN9X1P
RreLp5YKHoQWkg+KCYZtZbPYeOTJFBtre8MWIr6P8dl1ogzFd9b0Lg6KqgRpCfh7fV+0fY82ix/F
N25ydfP7Rd6H4T7fZarUoGh5Fwvhlkrieka2ufAIFBZWC2+hxRlzgz25Ittb+d6Hyfm/dxB9K39V
y7s4Kq7XGwTA8i/vbQ4V1aqL+2vFmNhQIxHR33neh9EBG9UyLZrexQTks4siVR6SN7+UxO//4bLQ
me1ybqb/WvbZMYrkofDJFEcHuY2dw0sQ3O8Bd3EnqM5SoS0iOuaZY8NNYXGq9EUtJNcQ14es590Q
OiRaS7bLsqsxItbnr245Ntnc+alqFkILyb3EbVo2A3ywjdoJEYA6KC5hYTQWV6ZusebYaEvfN2yy
0SCSGzFlTXzGRdKzpbxkr5rhxuk9ZeQUyuY+r/msqHRc1UJyqHA/idaKSovr8Nmx++7SpsRHJSlD
BsreObT848MalKRwdSQ/hlg/+zJ4zL7z5eP41BalpOrfrVCaf+E3jxyaXJ4VlVBUaAxUR3JKHrHM
wi2ItPpmwTONL4ZeKytxMItU49bnFMpcs583ctHJcRpITrCICb6pVsmWLwFiXVSi+vz04ojyyomZ
LXm8vrIYnJOfmsWmfSXqR9KP1xNpzeecWCuDhM0WN27WydEqvZwvURlsbE7+UVL+lmFBEQ6i3fkV
44mLaJ/KlZUIkmCXc7TtC1iXoVXLqDEnHxD7cUSBsNwXuRSnOacjQEqMS3bHXNG/ZyrkxOgS2/gH
uaHBOfmAqMU9EVUS/CBmM3VRGpF15b36B22yrfJc4RdJOYV0aOX4doolkl66Aq4pwoqdwH2XIfLE
jsdLRYXM6oLZNdw26JmEGi8ZGjm+MIno8Fs3VouNOcE9zkxXWasOeymOFEFElLk42/LJbiNEN0De
+1AjxzcYhcXbzxCR8lCmG7fClJJLf1F7VvqTe2inZ9uTyzvljXCembI3tXJ8U+AlVjkJ57SG2Gyp
h6RKLlbSVudc9pT3fZZK6l7Hyd6+rYHkIeJQ5mqYBSnEO2wtg70y/N/IAjOuWKMsHkienL2QeFNJ
qlbe7KCXKpIfWiQz1W4IigYP+KDt58p2h1PgLe7K+npM9kKfbSQNZB/IJp9hqkh2FOKSf1wS1SQ8
DXXRnoiILilKn2mnvBLt+dBsuRimLs9JujuTjCkaSG4n0QqhObClidnpMjn+UmKNFHKc7J1lSPjO
keoxKeqvNbyLzHziqEQTNCjJrx7prsz2aD2kHV0Yf0NV7TAhKutrXyN5/+D6RryLYbDGS5h6AP9P
Cdarv2biyfViDoFUEuNpZNajtF9elJ01xIh3cU566Z8gDiOOQEl2zQ+Qd9dK81eUdD2YmuWMsKnT
TZJn5wx5F+VEIk9MZlXouXZXLMtNTDGfp7R/c8zQLO2grH0VIorpboZ2fH+LUwfPbFgsahVQH+2J
KLEoSitEtH6B6YoalCljQ40s3eDSyo7mEwzs+OwBYi+BHpkhCnr2Q4CdIdH4JSrOXk5RyZy+JiRr
aSdll6PBRnZ8XZE/VeJwittp72dWzXNM0YB0DEMBRe+41GXls2R0tZOK00YY2/GFShnfKa7Chpp2
wyJz324qtKApZeeQT4zi+Gu1Zlcyj/6IOpKnwiNFFMsy/bsOJs6V+AYlGF66QtiH0n1luR7RuBrZ
xmiAq+x2FSWQw7I5Ww3Jf0o1ZObDwypwwHcxqL5rlsjm8nsSF2WXyX9mtzLqZXz6q0ovNBknt60a
kp+7Yqo42AMEssULRESvrFhOREOQX9mM6LTaL0COv6Y2N+plVD+k8jutVfPiVKJwT6S/USURmXo8
TyMRkXzDVa51oqsB5z5Io2nbculh69WRXFsgZxM1Rx2KEW7bargRXXIRmoJJWC6+qaQ5XkYZmDFc
Wu9RFZbOkCqLlVND8hj4iaap1NL4qqbQmSg1L+YQ0Ri4q8j9xplVZj0iehZZysA+r/1+NREeSpb+
TjXVkDwRnk8kHLJKa2B6Kkr09iJK9VZtQhuiig4iejjVQKrSs8sfqs2OE6TPQRNtpr2UkDw9VuC7
mpfL3/5WovAq8pEuzGugN8H59z9u17jifWmg+aQqkmPNPKWF6K0NC4mK87PtH8Bzagi/Rypnj5YI
eaqMx7v76CkCWkJXPtX6aHdpZFENyZXRXfSBBcCBUSKaSBfUI3rpJ2riIhp1UVWjz2vczz30OPjW
BusStExuLTmyqxqSAyU9yh97YNstk2iuHYWCdvpVEmPix7w8EzXv871NXfU0BV2bbHqj9VFp3HSQ
QSRLxivFM3LR4paodXTGte3jWlRRlgzka7Hgit5VpvXr2Duia69Bozccv3dbXLEkQvJkuAvX3Q7E
EA1EEPvSQ+DPJya1KAUR1UWI0+/59OjSbnU+VjBlTZUH707S+djlfl3De3fu0W/o3F+jY0+lqiD5
irTLYSmE0X6BDk7bi/YiWgDbVbWz/4RiKdrXjj+yOLxWSUUg1Fz9m191PkVn+4V37d25R7//zP/9
+r1T6WpInuwjjndcdqmcRh+JJBe2AX/RPyXUUZSmf5//PTS/Y03lvGGrPeKQ3icP9O/SrVfnHv2/
Wbz/ZszJzOwgWWUkOj0i4crBeX1b163auEVoaJMOk/ZEZ2b3WiIkfyZWxJ2Ohkz0mIvDff1NgmPV
SNWlif7xCE03drl3MX+tH9OpUZ0aLRqGftay76qTL7NhsxTJi/h6XiKitCC/G5SUX1bWeb28+qLh
2HLX6dVS755dN6pDw9q1Wnwe2qDVgNWnE7Nzn+VROOmilkDUQeQ1JE3YmUGUkZF9CKXcOv39yHaf
1wpt0SC0YashP5xNysZJco7k/8shQnJdcc21fesDInJ829fQWdL/T22WIvlGo3HiN5MeEdHtI/9r
91ldP1nszpxJ+V+z+YNF8vNDmR+GzRpdzP6nh1Mk/w+ODxbJH4zNuUjORXIuknORnIvkXCTnIjkX
yblIzkVyLpJzkZyL5Fwk5yI5F8m5SM5Fci6Sc5Gci+RcJOci+X8LyQFdPpThySPZ94Ox2YdHsucH
Y3NBHskuH4zNJYCK+HAGV6rQ/AOymav06PYB2cx1QBxt+nBsrvD/ANIP5RKCEJXDAAAAAElFTkSu
QmCC
--f46d040a62c49bb1c804f027e8cc--"""
class PyzorPreDigestTest(PyzorTestBase):
# we don't need the pyzord server to test this
@classmethod
def setUpClass(cls):
pass
@classmethod
def tearDownClass(cls):
pass
def setUp(self):
# no argument necessary
self.client_args = {}
def test_predigest_email(self):
"""Test email removal in the predigest process"""
emails = ["t@abc.ro",
"t1@abc.ro",
"t+@abc.ro",
"t.@abc.ro",
]
message = "Test %s Test2"
expected = b"TestTest2\n"
for email in emails:
msg = message % email
res = self.check_pyzor("predigest", None, input=TEXT % msg)
self.assertEqual(res, expected)
def test_predigest_long(self):
"""Test long "words" removal in the predigest process"""
strings = ["0A2D3f%a#S",
"3sddkf9jdkd9",
"@@#@@@@@@@@@"]
message = "Test %s Test2"
expected = b"TestTest2\n"
for s in strings:
msg = message % s
res = self.check_pyzor("predigest", None, input=TEXT % msg)
self.assertEqual(res, expected)
def test_predigest_line_length(self):
"""Test small lines removal in the predigest process"""
msg = "This line is included\n"\
"not this\n"\
"This also"
expected = b"Thislineisincluded\nThisalso\n"
res = self.check_pyzor("predigest", None, input=TEXT % msg)
self.assertEqual(res, expected)
def test_predigest_atomic(self):
"""Test atomic messages (lines <= 4) in the predigest process"""
msg = "All this message\nShould be included\nIn the predigest"
expected = b"Allthismessage\nShouldbeincluded\nInthepredigest\n"
res = self.check_pyzor("predigest", None, input=TEXT % msg)
self.assertEqual(res, expected)
def test_predigest_pieced(self):
"""Test pieced messages (lines > 4) in the predigest process"""
msg = ""
for i in range(100):
msg += "Line%d test test test\n" % i
expected = b""
for i in [20, 21, 22, 60, 61, 62]:
expected += ("Line%dtesttesttest\n" % i).encode("utf8")
res = self.check_pyzor("predigest", None, input=TEXT % msg)
self.assertEqual(res, expected)
def test_predigest_html(self):
expected = """Emailspam,alsoknownasjunkemailorbulkemail,isasubset
ofspaminvolvingnearlyidenticalmessagessenttonumerous
byemail.Clickingonlinksinspamemailmaysendusersto
byemail.Clickingonlinksinspamemailmaysendusersto
phishingwebsitesorsitesthatarehostingmalware.
Emailspam.Emailspam,alsoknownasjunkemailorbulkemail,isasubsetofspaminvolvingnearlyidenticalmessagessenttonumerousbyemail.Clickingonlinksinspamemailmaysenduserstophishingwebsitesorsitesthatarehostingmalware.
""".encode("utf8")
res = self.check_pyzor("predigest", None, input=HTML_TEXT)
self.assertEqual(res, expected)
def test_predigest_attachemnt(self):
expected = b"Thisisatestmailing\n"
res = self.check_pyzor("predigest", None, input=TEXT_ATTACHMENT)
self.assertEqual(res, expected)
class PyzorDigestTest(PyzorTestBase):
# we don't need the pyzord server to test this
@classmethod
def setUpClass(cls):
pass
@classmethod
def tearDownClass(cls):
pass
def setUp(self):
# no argument necessary
self.client_args = {}
def test_digest_email(self):
"""Test email removal in the digest process"""
emails = ["t@abc.ro",
"t1@abc.ro",
"t+@abc.ro",
"t.@abc.ro",
]
message = "Test %s Test2"
expected = b"TestTest2"
for email in emails:
msg = message % email
res = self.check_pyzor("digest", None, input=TEXT % msg)
self.assertEqual(res.decode("utf8"),
hashlib.sha1(expected).hexdigest().lower() + "\n")
def test_digest_long(self):
"""Test long "words" removal in the digest process"""
strings = ["0A2D3f%a#S",
"3sddkf9jdkd9",
"@@#@@@@@@@@@"]
message = "Test %s Test2"
expected = b"TestTest2"
for s in strings:
msg = message % s
res = self.check_pyzor("digest", None, input=TEXT % msg)
self.assertEqual(res.decode("utf8"),
hashlib.sha1(expected).hexdigest().lower() + "\n")
def test_digest_line_length(self):
"""Test small lines removal in the digest process"""
msg = "This line is included\n"\
"not this\n"\
"This also"
expected = b"ThislineisincludedThisalso"
res = self.check_pyzor("digest", None, input=TEXT % msg)
self.assertEqual(res.decode("utf8"),
hashlib.sha1(expected).hexdigest().lower() + "\n")
def test_digest_atomic(self):
"""Test atomic messages (lines <= 4) in the digest process"""
msg = "All this message\nShould be included\nIn the digest"
expected = b"AllthismessageShouldbeincludedInthedigest"
res = self.check_pyzor("digest", None, input=TEXT % msg)
self.assertEqual(res.decode("utf8"),
hashlib.sha1(expected).hexdigest().lower() + "\n")
def test_digest_pieced(self):
"""Test pieced messages (lines > 4) in the digest process"""
msg = ""
for i in range(100):
msg += "Line%d test test test\n" % i
expected = b""
for i in [20, 21, 22, 60, 61, 62]:
expected += ("Line%dtesttesttest" % i).encode("utf8")
res = self.check_pyzor("digest", None, input=TEXT % msg)
self.assertEqual(res.decode("utf8"),
hashlib.sha1(expected).hexdigest().lower() + "\n")
def test_digest_html(self):
expected = """Emailspam,alsoknownasjunkemailorbulkemail,isasubset
ofspaminvolvingnearlyidenticalmessagessenttonumerous
byemail.Clickingonlinksinspamemailmaysendusersto
byemail.Clickingonlinksinspamemailmaysendusersto
phishingwebsitesorsitesthatarehostingmalware.
Emailspam.Emailspam,alsoknownasjunkemailorbulkemail,isasubsetofspaminvolvingnearlyidenticalmessagessenttonumerousbyemail.Clickingonlinksinspamemailmaysenduserstophishingwebsitesorsitesthatarehostingmalware.
""".replace("\n", "").encode("utf8")
res = self.check_pyzor("digest", None, input=HTML_TEXT)
self.assertEqual(res.decode("utf8"),
hashlib.sha1(expected).hexdigest().lower() + "\n")
def test_digest_attachemnt(self):
expected = b"Thisisatestmailing"
res = self.check_pyzor("digest", None, input=TEXT_ATTACHMENT)
self.assertEqual(res.decode("utf8"),
hashlib.sha1(expected).hexdigest().lower() + "\n")
ENCODING_TEST_EMAIL = """From nobody Tue Apr 1 13:18:54 2014
Content-Type: multipart/related;
boundary="===============0632694142025794937=="
MIME-Version: 1.0
This is a multi-part message in MIME format.
--===============0632694142025794937==
Content-Type: text/plain; charset="iso-8859-1"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Thist is a t=E9st
--===============0632694142025794937==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
VGhpcyBpcyBhIHRlc3Qg5r+A5YWJ6YCZ
--===============0632694142025794937==
MIME-Version: 1.0
Content-Type: text/plain; charset="cp1258"
Content-Transfer-Encoding: base64
VGhpcyBpcyBhIHTpc3Qg4qXG
--===============0632694142025794937==--
"""
BAD_ENCODING = """From nobody Tue Apr 1 13:18:54 2014
Content-Type: multipart/related;
boundary="===============0632694142025794937=="
MIME-Version: 1.0
This is a multi-part message in MIME format.
--===============0632694142025794937==
Content-Type: text/plain; charset=ISO-8859-1Content-Transfer-Encoding: quoted-printable
This is a test
--===============0632694142025794937==
Content-Type: text/plain; charset=us-asciia
Content-Transfer-Encoding: quoted-printable
This is a test
--===============0632694142025794937==
"""
class PyzorEncodingTest(PyzorTestBase):
# we don't need the pyzord server to test this
@classmethod
def setUpClass(cls):
pass
@classmethod
def tearDownClass(cls):
pass
def setUp(self):
# no argument necessary
self.client_args = {}
def test_encodings(self):
expected = "47a83cd0e5cc9bd2c64c06c00e3853f79e63014f\n"
res = self.check_pyzor("digest", None, input=ENCODING_TEST_EMAIL)
self.assertEqual(res.decode("utf8"), expected)
def test_bad_encoding(self):
expected = "2b4dbf2fb521edd21d997f3f04b1c7155ba91fff\n"
res = self.check_pyzor("digest", None, input=BAD_ENCODING)
self.assertEqual(res.decode("utf8"), expected)
def suite():
"""Gather all the tests from this module in a test suite."""
test_suite = unittest.TestSuite()
test_suite.addTest(unittest.makeSuite(PyzorDigestTest))
test_suite.addTest(unittest.makeSuite(PyzorPreDigestTest))
test_suite.addTest(unittest.makeSuite(PyzorEncodingTest))
return test_suite
if __name__ == '__main__':
unittest.main()

View File

@@ -0,0 +1,30 @@
import unittest
from tests.util import *
class GdbmPyzorTest(PyzorTest, PyzorTestBase):
"""Test the gdbm engine"""
dsn = "pyzord.db"
engine = "gdbm"
class ThreadsGdbmPyzorTest(GdbmPyzorTest):
"""Test the gdbm engine with threads activated."""
threads = "True"
max_threads = "0"
class MaxThreadsGdbmPyzorTest(GdbmPyzorTest):
"""Test the gdbm engine with with maximum threads."""
threads = "True"
max_threads = "10"
def suite():
"""Gather all the tests from this module in a test suite."""
test_suite = unittest.TestSuite()
test_suite.addTest(unittest.makeSuite(GdbmPyzorTest))
test_suite.addTest(unittest.makeSuite(ThreadsGdbmPyzorTest))
test_suite.addTest(unittest.makeSuite(MaxThreadsGdbmPyzorTest))
return test_suite
if __name__ == '__main__':
unittest.main()

View File

@@ -0,0 +1,108 @@
import unittest
import ConfigParser
from tests.util import *
try:
import MySQLdb
except ImportError:
MySQLdb = None
schema = """
CREATE TABLE IF NOT EXISTS `%s` (
`digest` char(40) default NULL,
`r_count` int(11) default NULL,
`wl_count` int(11) default NULL,
`r_entered` datetime default NULL,
`wl_entered` datetime default NULL,
`r_updated` datetime default NULL,
`wl_updated` datetime default NULL,
PRIMARY KEY (`digest`)
)
"""
@unittest.skipIf(not os.path.exists("./test.conf"),
"test.conf is not available")
@unittest.skipIf(MySQLdb == None, "MySQLdb library not available")
class MySQLdbPyzorTest(PyzorTest, PyzorTestBase):
"""Test the mysql engine."""
dsn = None
engine = "mysql"
@classmethod
def setUpClass(cls):
conf = ConfigParser.ConfigParser()
conf.read("./test.conf")
table = conf.get("test", "table")
db = MySQLdb.Connect(host=conf.get("test", "host"),
user=conf.get("test", "user"),
passwd=conf.get("test", "passwd"),
db=conf.get("test", "db"))
c = db.cursor()
c.execute(schema % table)
c.close()
db.close()
cls.dsn = "%s,%s,%s,%s,%s" % (conf.get("test", "host"),
conf.get("test", "user"),
conf.get("test", "passwd"),
conf.get("test", "db"),
conf.get("test", "table"))
super(MySQLdbPyzorTest, cls).setUpClass()
@classmethod
def tearDownClass(cls):
super(MySQLdbPyzorTest, cls).tearDownClass()
try:
conf = ConfigParser.ConfigParser()
conf.read("./test.conf")
table = conf.get("test", "table")
db = MySQLdb.Connect(host=conf.get("test", "host"),
user=conf.get("test", "user"),
passwd=conf.get("test", "passwd"),
db=conf.get("test", "db"))
c = db.cursor()
c.execute("DROP TABLE %s" % table)
c.close()
db.close()
except:
pass
class ThreadsMySQLdbPyzorTest(MySQLdbPyzorTest):
"""Test the mysql engine with threads activated."""
threads = "True"
max_threads = "0"
class BoundedThreadsMySQLdbPyzorTest(MySQLdbPyzorTest):
"""Test the mysql engine with threads and DBConnections set."""
threads = "True"
max_threads = "0"
db_connections = "10"
class MaxThreadsMySQLdbPyzorTest(MySQLdbPyzorTest):
"""Test the mysql engine with threads and MaxThreads set."""
threads = "True"
max_threads = "10"
class BoundedMaxThreadsMySQLdbPyzorTest(MySQLdbPyzorTest):
"""Test the mysql engine with threads, MaxThreads and DBConnections set."""
threads = "True"
max_threads = "10"
db_connections = "10"
class ProcessesMySQLdbPyzorTest(MySQLdbPyzorTest):
processes = "True"
max_processes = "10"
def suite():
"""Gather all the tests from this module in a test suite."""
test_suite = unittest.TestSuite()
test_suite.addTest(unittest.makeSuite(MySQLdbPyzorTest))
test_suite.addTest(unittest.makeSuite(ThreadsMySQLdbPyzorTest))
test_suite.addTest(unittest.makeSuite(BoundedThreadsMySQLdbPyzorTest))
test_suite.addTest(unittest.makeSuite(MaxThreadsMySQLdbPyzorTest))
test_suite.addTest(unittest.makeSuite(BoundedMaxThreadsMySQLdbPyzorTest))
test_suite.addTest(unittest.makeSuite(ProcessesMySQLdbPyzorTest))
return test_suite
if __name__ == '__main__':
unittest.main()

View File

@@ -0,0 +1,164 @@
import sys
import unittest
from tests.util import *
class PyzorScriptTest(PyzorTestBase):
password_file = None
access = """ALL : anonymous : allow
"""
def test_report_threshold(self):
input = "Test1 report threshold 1 Test2"
self.client_args["-r"] = "2"
self.check_pyzor("report", None, input=input, code=200, exit_code=0)
self.check_pyzor("check", None, input=input, code=200, exit_code=1,
counts=(1, 0))
self.check_pyzor("report", None, input=input, code=200, exit_code=0)
self.check_pyzor("check", None, input=input, code=200, exit_code=1,
counts=(2, 0))
# Exit code will be success now, since the report count exceeds the
# threshold
self.check_pyzor("report", None, input=input, code=200, exit_code=0)
self.check_pyzor("check", None, input=input, code=200, exit_code=0,
counts=(3, 0))
def test_whitelist_threshold(self):
input = "Test1 white list threshold 1 Test2"
self.client_args["-w"] = "2"
self.check_pyzor("report", None, input=input, code=200, exit_code=0)
self.check_pyzor("check", None, input=input, code=200, exit_code=0,
counts=(1, 0))
self.check_pyzor("whitelist", None, input=input, code=200, exit_code=0)
self.check_pyzor("check", None, input=input, code=200, exit_code=0,
counts=(1, 1))
self.check_pyzor("whitelist", None, input=input, code=200, exit_code=0)
self.check_pyzor("check", None, input=input, code=200, exit_code=0,
counts=(1, 2))
# Exit code will be failure now, since the whitelist count exceeds the
# threshold
self.check_pyzor("whitelist", None, input=input, code=200, exit_code=0)
self.check_pyzor("check", None, input=input, code=200, exit_code=1,
counts=(1, 3))
def test_report_whitelist_threshold(self):
input = "Test1 report white list threshold 1 Test2"
self.client_args["-w"] = "2"
self.client_args["-r"] = "1"
self.check_pyzor("report", None, input=input, code=200, exit_code=0)
self.check_pyzor("check", None, input=input, code=200, exit_code=1,
counts=(1, 0))
# Exit code will be success now, since the report count exceeds the
# threshold
self.check_pyzor("report", None, input=input, code=200, exit_code=0)
self.check_pyzor("check", None, input=input, code=200, exit_code=0,
counts=(2, 0))
self.check_pyzor("whitelist", None, input=input, code=200, exit_code=0)
self.check_pyzor("check", None, input=input, code=200, exit_code=0,
counts=(2, 1))
self.check_pyzor("whitelist", None, input=input, code=200, exit_code=0)
self.check_pyzor("check", None, input=input, code=200, exit_code=0,
counts=(2, 2))
# Exit code will be failure now, since the whitelist count exceeds the
# threshold
self.check_pyzor("whitelist", None, input=input, code=200, exit_code=0)
self.check_pyzor("check", None, input=input, code=200, exit_code=1,
counts=(2, 3))
def test_digest_style(self):
input = "da39a3ee5e6b4b0d3255bfef95601890afd80700"
self.client_args["-s"] = "digests"
self.check_pyzor("pong", None, input=input, code=200, exit_code=0,
counts=(sys.maxint, 0))
self.check_pyzor("check", None, input=input, code=200, exit_code=1,
counts=(0, 0))
self.check_pyzor("report", None, input=input, code=200, exit_code=0)
self.check_pyzor("check", None, input=input, code=200, exit_code=0,
counts=(1, 0))
self.check_pyzor("whitelist", None, input=input, code=200, exit_code=0)
self.check_pyzor("check", None, input=input, code=200, exit_code=1,
counts=(1, 1))
r = self.get_record(input, None)
self.assertEqual(r["Count"], "1")
self.assertEqual(r["WL-Count"], "1")
def test_digest_style_multiple(self):
input2 = "da39a3ee5e6b4b0d3255bfef95601890afd80705\n"\
"da39a3ee5e6b4b0d3255bfef95601890afd80706\n"
input3 = "da39a3ee5e6b4b0d3255bfef95601890afd80705\n"\
"da39a3ee5e6b4b0d3255bfef95601890afd80706\n"\
"da39a3ee5e6b4b0d3255bfef95601890afd80707\n"
self.client_args["-s"] = "digests"
self.check_pyzor_multiple("pong", None, input=input3, exit_code=0,
code=[200, 200, 200],
counts=[(sys.maxint, 0),
(sys.maxint, 0),
(sys.maxint, 0)])
self.check_pyzor_multiple("check", None, input=input3, exit_code=1,
code=[200, 200, 200],
counts=[(0, 0), (0, 0), (0, 0)])
self.check_pyzor_multiple("report", None, input=input2, exit_code=0)
self.check_pyzor_multiple("check", None, input=input3, exit_code=0,
code=[200, 200, 200],
counts=[(1, 0), (1, 0), (0, 0)])
self.check_pyzor_multiple("whitelist", None, input=input3, exit_code=0)
self.check_pyzor_multiple("check", None, input=input3, exit_code=1,
code=[200, 200, 200],
counts=[(1, 1), (1, 1), (0, 1)])
def test_mbox_style(self):
input = "From MAILER-DAEMON Mon Jan 6 15:13:33 2014\n\nTest1 message 0 Test2\n\n"
self.client_args["-s"] = "mbox"
self.check_pyzor("pong", None, input=input, code=200, exit_code=0,
counts=(sys.maxint, 0))
self.check_pyzor("check", None, input=input, code=200, exit_code=1,
counts=(0, 0))
self.check_pyzor("report", None, input=input, code=200, exit_code=0)
self.check_pyzor("check", None, input=input, code=200, exit_code=0,
counts=(1, 0))
self.check_pyzor("whitelist", None, input=input, code=200, exit_code=0)
self.check_pyzor("check", None, input=input, code=200, exit_code=1,
counts=(1, 1))
r = self.get_record(input, None)
self.assertEqual(r["Count"], "1")
self.assertEqual(r["WL-Count"], "1")
def test_mbox_style_multiple(self):
input2 = "From MAILER-DAEMON Mon Jan 6 15:08:02 2014\n\nTest1 message 1 Test2\n\n"\
"From MAILER-DAEMON Mon Jan 6 15:08:05 2014\n\nTest1 message 2 Test2\n\n"
input3 = "From MAILER-DAEMON Mon Jan 6 15:08:02 2014\n\nTest1 message 1 Test2\n\n"\
"From MAILER-DAEMON Mon Jan 6 15:08:05 2014\n\nTest1 message 2 Test2\n\n"\
"From MAILER-DAEMON Mon Jan 6 15:08:08 2014\n\nTest1 message 3 Test2\n\n"
self.client_args["-s"] = "mbox"
self.check_pyzor_multiple("pong", None, input=input3, exit_code=0,
code=[200, 200, 200],
counts=[(sys.maxint, 0),
(sys.maxint, 0),
(sys.maxint, 0)])
self.check_pyzor_multiple("check", None, input=input3, exit_code=1,
code=[200, 200, 200],
counts=[(0, 0), (0, 0), (0, 0)])
self.check_pyzor_multiple("report", None, input=input2, exit_code=0)
self.check_pyzor_multiple("check", None, input=input3, exit_code=0,
code=[200, 200, 200],
counts=[(1, 0), (1, 0), (0, 0)])
self.check_pyzor_multiple("whitelist", None, input=input3, exit_code=0)
self.check_pyzor_multiple("check", None, input=input3, exit_code=1,
code=[200, 200, 200],
counts=[(1, 1), (1, 1), (0, 1)])
def test_predigest(self):
out = self.check_pyzor("predigest", None, input=msg).strip()
self.assertEqual(out.decode("utf8"), "TestEmail")
def test_digest(self):
out = self.check_pyzor("digest", None, input=msg).strip()
self.assertEqual(out.decode("utf8"), digest)
def suite():
"""Gather all the tests from this module in a test suite."""
test_suite = unittest.TestSuite()
test_suite.addTest(unittest.makeSuite(PyzorScriptTest))
return test_suite
if __name__ == '__main__':
unittest.main()

View File

@@ -0,0 +1,40 @@
import unittest
try:
import redis
except ImportError:
redis = None
from tests.util import *
@unittest.skipIf(redis == None, "redis library not available")
class RedisPyzorTest(PyzorTest, PyzorTestBase):
"""Test the redis engine"""
dsn = "localhost,,,10"
engine = "redis"
@classmethod
def tearDownClass(cls):
super(RedisPyzorTest, cls).tearDownClass()
redis.StrictRedis(db=10).flushdb()
class ThreadsRedisPyzorTest(RedisPyzorTest):
"""Test the redis engine with threads activated."""
threads = "True"
class MaxThreadsRedisPyzorTest(RedisPyzorTest):
"""Test the gdbm engine with with maximum threads."""
threads = "True"
max_threads = "10"
def suite():
"""Gather all the tests from this module in a test suite."""
test_suite = unittest.TestSuite()
test_suite.addTest(unittest.makeSuite(RedisPyzorTest))
test_suite.addTest(unittest.makeSuite(ThreadsRedisPyzorTest))
test_suite.addTest(unittest.makeSuite(MaxThreadsRedisPyzorTest))
return test_suite
if __name__ == '__main__':
unittest.main()

View File

@@ -0,0 +1,32 @@
"""A suite of unit tests that verifies the correct behaviour of various
functions/methods in the pyzord code.
Note these tests the source of pyzor, not the version currently installed.
"""
import unittest
import test_gdbm
import test_mysql
import test_redis
import test_client
import test_digest
import test_server
import test_account
def suite():
"""Gather all the tests from this package in a test suite."""
test_suite = unittest.TestSuite()
test_suite.addTest(test_gdbm.suite())
test_suite.addTest(test_mysql.suite())
test_suite.addTest(test_redis.suite())
test_suite.addTest(test_client.suite())
test_suite.addTest(test_digest.suite())
test_suite.addTest(test_server.suite())
test_suite.addTest(test_account.suite())
return test_suite
if __name__ == '__main__':
unittest.main(defaultTest='suite')

View File

@@ -0,0 +1,139 @@
"""Test the pyzor.account module
"""
import os
import sys
import time
import email
import hashlib
import unittest
import StringIO
import pyzor
import pyzor.config
import pyzor.account
class AccountTest(unittest.TestCase):
def setUp(self):
unittest.TestCase.setUp(self)
self.timestamp = 1381219396
self.msg = email.message_from_string("")
self.msg["Op"] = "ping"
self.msg["Thread"] = "14941"
self.msg["PV"] = "2.1"
self.msg["User"] = "anonymous"
self.msg["Time"] = str(self.timestamp)
def tearDown(self):
unittest.TestCase.tearDown(self)
def test_sign_msg(self):
"""Test the sign message function"""
hashed_key = hashlib.sha1(b"test_key").hexdigest()
expected = "2ab1bad2aae6fd80c656a896c82eef0ec1ec38a0"
result = pyzor.account.sign_msg(hashed_key, self.timestamp, self.msg)
self.assertEqual(result, expected)
def test_hash_key(self):
"""Test the hash key function"""
user = "testuser"
key = "testkey"
expected = "0957bd79b58263657127a39762879098286d8477"
result = pyzor.account.hash_key(key, user)
self.assertEqual(result, expected)
def test_verify_signature(self):
"""Test the verify signature function"""
def mock_sm(h, t, m):
return "testsig"
real_sm = pyzor.account.sign_msg
pyzor.account.sign_msg = mock_sm
try:
self.msg["Sig"] = "testsig"
del self.msg["Time"]
self.msg["Time"] = str(int(time.time()))
pyzor.account.verify_signature(self.msg, "testkey")
finally:
pyzor.account.sign_msg = real_sm
def test_verify_signature_old_timestamp(self):
"""Test the verify signature with old timestamp"""
def mock_sm(h, t, m):
return "testsig"
real_sm = pyzor.account.sign_msg
pyzor.account.sign_msg = mock_sm
try:
self.msg["Sig"] = "testsig"
self.assertRaises(pyzor.SignatureError, pyzor.account.verify_signature, self.msg, "testkey")
finally:
pyzor.account.sign_msg = real_sm
def test_verify_signature_bad_signature(self):
"""Test the verify signature with invalid signature"""
def mock_sm(h, t, m):
return "testsig"
real_sm = pyzor.account.sign_msg
pyzor.account.sign_msg = mock_sm
try:
self.msg["Sig"] = "testsig-bad"
del self.msg["Time"]
self.msg["Time"] = str(int(time.time()))
self.assertRaises(pyzor.SignatureError, pyzor.account.verify_signature, self.msg, "testkey")
finally:
pyzor.account.sign_msg = real_sm
class LoadAccountTest(unittest.TestCase):
"""Tests for the load_accounts function"""
def setUp(self):
unittest.TestCase.setUp(self)
self.real_exists = os.path.exists
os.path.exists = lambda p: True
self.mock_file = StringIO.StringIO()
self.real_open = pyzor.account.__builtins__["open"]
def mock_open(path, mode="r", buffering=-1):
if path == "test_file":
self.mock_file.seek(0)
return self.mock_file
else:
return self.real_open(path, mode, buffering)
pyzor.account.__builtins__["open"] = mock_open
def tearDown(self):
unittest.TestCase.tearDown(self)
os.path.exists = self.real_exists
pyzor.account.__builtins__["open"] = self.real_open
def test_load_accounts(self):
"""Test loading the account file"""
self.mock_file.write("public.pyzor.org : 24441 : test : 123abc,cba321\n"
"public2.pyzor.org : 24441 : test2 : 123abc,cba321")
result = pyzor.config.load_accounts("test_file")
self.assertIn(("public.pyzor.org", 24441), result)
self.assertIn(("public2.pyzor.org", 24441), result)
account = result[("public.pyzor.org", 24441)]
self.assertEqual((account.username, account.salt, account.key),
("test", "123abc", "cba321"))
account = result[("public2.pyzor.org", 24441)]
self.assertEqual((account.username, account.salt, account.key),
("test2", "123abc", "cba321"))
def test_load_accounts_comment(self):
"""Test skipping commented lines"""
self.mock_file.write("#public1.pyzor.org : 24441 : test : 123abc,cba321")
result = pyzor.config.load_accounts("test_file")
self.assertNotIn(("public.pyzor.org", 24441), result)
self.assertFalse(result)
def suite():
"""Gather all the tests from this module in a test suite."""
test_suite = unittest.TestSuite()
test_suite.addTest(unittest.makeSuite(AccountTest))
test_suite.addTest(unittest.makeSuite(LoadAccountTest))
return test_suite
if __name__ == '__main__':
unittest.main()

View File

@@ -0,0 +1,168 @@
import sys
import time
import socket
import unittest
import pyzor
import pyzor.client
import pyzor.account
import pyzor.message
def make_MockSocket(response, request):
"""Create a MockSocket class that will append requests to
the specified `request` list and return the specified `response`
"""
class MockSocket():
def __init__(self, *args, **kwargs):
pass
def settimeout(self, timeout):
pass
def recvfrom(self, packetsize):
return response, ("127.0.0.1", 24441)
def sendto(self, data, flag, address):
request.append(data)
return MockSocket
def make_MockThreadId(thread):
"""Creates a MockThreadId class that will generate
the specified thread number.
"""
class MockThreadId(int):
def __new__(cls, i):
return int.__new__(cls, i)
@classmethod
def generate(cls):
return thread
def in_ok_range(self):
return True
return MockThreadId
def mock_sign_msg(hash_key, timestamp, msg):
return "TestSig"
def mock_hash_key(user_key, user):
return None
class ClientTest(unittest.TestCase):
def setUp(self):
unittest.TestCase.setUp(self)
self.real_sg = pyzor.account.sign_msg
pyzor.account.sign_msg = mock_sign_msg
self.real_hk = pyzor.account.hash_key
pyzor.account.hash_key = mock_hash_key
self.thread = 33715
# the response the mock socket will send
self.response = "Code: 200\nDiag: OK\nPV: 2.1\nThread: 33715\n\n"
# the requests send by the client will be stored here
self.request = []
# the expected request that the client should send
self.expected = {"Thread": str(self.thread),
"PV": str(pyzor.proto_version),
"User": "anonymous",
"Time": str(int(time.time())),
"Sig": "TestSig"}
def tearDown(self):
unittest.TestCase.tearDown(self)
pyzor.account.sign_msg = self.real_sg
pyzor.account.hash_key = self.real_hk
def check_request(self, request):
"""Check if the request sent by the client is equal
to the expected one.
"""
req = {}
request = request.decode("utf8").replace("\n\n", "\n")
for line in request.splitlines():
key = line.split(":")[0].strip()
value = line.split(":")[1].strip()
req[key] = value
self.assertEqual(req, self.expected)
def check_client(self, accounts, method, *args, **kwargs):
"""Tests if the request and response are sent
and read correctly by the client.
"""
real_socket = socket.socket
socket.socket = make_MockSocket(self.response.encode("utf8"),
self.request)
real_ThreadId = pyzor.message.ThreadId
pyzor.message.ThreadId = make_MockThreadId(self.thread)
client = pyzor.client.Client(accounts)
try:
response = getattr(client, method)(*args, **kwargs)
self.assertEqual(str(response), self.response)
self.check_request(self.request[0])
finally:
socket.socket = real_socket
pyzor.message.ThreadId = real_ThreadId
return client
def test_ping(self):
"""Test the client ping request"""
self.expected["Op"] = "ping"
self.check_client(None, "ping")
def test_pong(self):
"""Test the client pong request"""
digest = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
self.expected["Op"] = "pong"
self.expected["Op-Digest"] = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
self.check_client(None, "pong", digest)
def test_check(self):
"""Test the client check request"""
digest = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
self.expected["Op"] = "check"
self.expected["Op-Digest"] = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
self.check_client(None, "check", digest)
def test_info(self):
"""Test the client info request"""
digest = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
self.expected["Op"] = "info"
self.expected["Op-Digest"] = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
self.check_client(None, "info", digest)
def test_report(self):
"""Test the client report request"""
digest = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
self.expected["Op"] = "report"
self.expected["Op-Digest"] = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
self.expected["Op-Spec"] = "20,3,60,3"
self.check_client(None, "report", digest)
def test_whitelist(self):
"""Test the client whitelist request"""
digest = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
self.expected["Op"] = "whitelist"
self.expected["Op-Digest"] = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
self.expected["Op-Spec"] = "20,3,60,3"
self.check_client(None, "whitelist", digest)
def test_handle_account(self):
"""Test client handling accounts"""
test_account = pyzor.account.Account("TestUser", "TestKey", "TestSalt")
self.expected["Op"] = "ping"
self.expected["User"] = "TestUser"
self.check_client({("public.pyzor.org", 24441): test_account}, "ping")
def test_handle_invalid_thread(self):
"""Test invalid thread id"""
self.thread += 20
self.expected["Op"] = "ping"
self.assertRaises(pyzor.ProtocolError, self.check_client, None, "ping")
def suite():
"""Gather all the tests from this module in a test suite."""
test_suite = unittest.TestSuite()
test_suite.addTest(unittest.makeSuite(ClientTest))
return test_suite
if __name__ == '__main__':
unittest.main()

View File

@@ -0,0 +1,200 @@
"""The the pyzor.digest module
"""
import sys
import hashlib
import unittest
from pyzor.digest import *
HTML_TEXT = """<html><head><title>Email spam</title></head><body>
<p><b>Email spam</b>, also known as <b>junk email</b>
or <b>unsolicited bulk email</b> (<i>UBE</i>), is a subset of
<a href="/wiki/Spam_(electronic)" title="Spam (electronic)">electronic spam</a>
involving nearly identical messages sent to numerous recipients by <a href="/wiki/Email" title="Email">
email</a>. Clicking on <a href="/wiki/Html_email#Security_vulnerabilities" title="Html email" class="mw-redirect">
links in spam email</a> may send users to <a href="/wiki/Phishing" title="Phishing">phishing</a>
web sites or sites that are hosting <a href="/wiki/Malware" title="Malware">malware</a>.</body></html>"""
HTML_TEXT_STRIPED = 'Email spam Email spam , also known as junk email or unsolicited bulk email ( UBE ),'\
' is a subset of electronic spam involving nearly identical messages sent to numerous recipients by email'\
' . Clicking on links in spam email may send users to phishing web sites or sites that are hosting malware .'
class HTMLStripperTests(unittest.TestCase):
def setUp(self):
unittest.TestCase.setUp(self)
self.data = []
def tearDown(self):
unittest.TestCase.tearDown(self)
def test_HTMLStripper(self):
stripper = HTMLStripper(self.data)
stripper.feed(HTML_TEXT)
res = " ".join(self.data)
self.assertEqual(res, HTML_TEXT_STRIPED)
class PreDigestTests(unittest.TestCase):
def setUp(self):
unittest.TestCase.setUp(self)
self.lines = []
def mock_digest_paylods(c, message):
yield message.decode("utf8")
def mock_handle_line(s, line):
self.lines.append(line.decode("utf8"))
self.real_digest_payloads = DataDigester.digest_payloads
self.real_handle_line = DataDigester.handle_line
DataDigester.digest_payloads = mock_digest_paylods
DataDigester.handle_line = mock_handle_line
def tearDown(self):
unittest.TestCase.tearDown(self)
DataDigester.digest_payloads = self.real_digest_payloads
DataDigester.handle_line = self.real_handle_line
def test_predigest_emails(self):
"""Test email removal in the predigest process"""
real_longstr = DataDigester.longstr_ptrn
DataDigester.longstr_ptrn = re.compile(r'\S{100,}')
emails = ["test@example.com",
"test123@example.com",
"test+abc@example.com",
"test.test2@example.com",
"test.test2+abc@example.com", ]
message = "Test %s Test2"
expected = "TestTest2"
try:
for email in emails:
self.lines = []
DataDigester((message % email).encode("utf8"))
self.assertEqual(self.lines[0], expected)
finally:
DataDigester.longstr_ptrn = real_longstr
# XXX This fails
# def test_predigest_emails_whitespace(self):
# real_longstr = DataDigester.longstr_ptrn
# DataDigester.longstr_ptrn = re.compile(r'\S{100,}')
# emails = ["chirila@example. com",
# "chirila@example . com",
# "chirila @example. com",
# "chirila@ example. com",
# "chirila @example . com",
# "chirila @ example. com",
# "chirila @ example . com",]
# message = "Test %s Test2"
# expected = "TestTest2"
# try:
# for email in emails:
# self.lines = []
# DataDigester(message % email)
# self.assertEqual(self.lines[0], expected)
# finally:
# DataDigester.longstr_ptrn = real_longstr
def test_predigest_urls(self):
"""Test url removal in the predigest process"""
real_longstr = DataDigester.longstr_ptrn
DataDigester.longstr_ptrn = re.compile(r'\S{100,}')
urls = ["http://www.example.com",
# "www.example.com", # XXX This also fail
"http://example.com",
# "example.com", # XXX This also fails
"http://www.example.com/test/"
"http://www.example.com/test/test2", ]
message = "Test %s Test2"
expected = "TestTest2"
try:
for url in urls:
self.lines = []
DataDigester((message % url).encode("utf8"))
self.assertEqual(self.lines[0], expected)
finally:
DataDigester.longstr_ptrn = real_longstr
def test_predigest_long(self):
"""Test long "words" removal in the predigest process"""
strings = ["0A2D3f%a#S",
"3sddkf9jdkd9",
"@@#@@@@@@@@@"]
message = "Test %s Test2"
expected = "TestTest2"
for string in strings:
self.lines = []
DataDigester((message % string).encode("utf8"))
self.assertEqual(self.lines[0], expected)
def test_predigest_min_line_lenght(self):
"""Test small lines removal in the predigest process"""
message = "This line is included\n"\
"not this\n"\
"This also"
expected = ["Thislineisincluded", "Thisalso"]
DataDigester(message.encode("utf8"))
self.assertEqual(self.lines, expected)
def test_predigest_atomic(self):
"""Test atomic messages (lines <= 4) in the predigest process"""
message = "All this message\nShould be included\nIn the predigest"
expected = ["Allthismessage", "Shouldbeincluded", "Inthepredigest"]
DataDigester(message.encode("utf8"))
self.assertEqual(self.lines, expected)
def test_predigest_pieced(self):
"""Test pieced messages (lines > 4) in the predigest process"""
message = ""
for i in range(100):
message += "Line%d test test test\n" % i
expected = []
for i in [20, 21, 22, 60, 61, 62]:
expected.append("Line%dtesttesttest" % i)
DataDigester(message.encode("utf8"))
self.assertEqual(self.lines, expected)
class DigestTests(unittest.TestCase):
def setUp(self):
unittest.TestCase.setUp(self)
self.lines = []
def mock_digest_paylods(c, message):
yield message.decode("utf8")
self.real_digest_payloads = DataDigester.digest_payloads
DataDigester.digest_payloads = mock_digest_paylods
def tearDown(self):
unittest.TestCase.tearDown(self)
DataDigester.digest_payloads = self.real_digest_payloads
def test_digest(self):
message = b"That's some good ham right there"
predigested = b"That'ssomegoodhamrightthere"
digest = hashlib.sha1()
digest.update(predigested)
expected = digest.hexdigest()
result = DataDigester(message).value
self.assertEqual(result, expected)
def suite():
"""Gather all the tests from this module in a test suite."""
test_suite = unittest.TestSuite()
test_suite.addTest(unittest.makeSuite(HTMLStripperTests))
test_suite.addTest(unittest.makeSuite(PreDigestTests))
test_suite.addTest(unittest.makeSuite(DigestTests))
return test_suite
if __name__ == '__main__':
unittest.main()

View File

@@ -0,0 +1,153 @@
"""Test the pyzor.engines.gdbm_ module."""
import gdbm
import unittest
import threading
from datetime import datetime, timedelta
import pyzor.engines
import pyzor.engines.gdbm_
import pyzor.engines.common
class MockTimer():
def __init__(self, *args, **kwargs):
pass
def start(self):
pass
def setDaemon(self, daemon):
pass
class MockGdbm(dict):
"""Mock a gdbm database"""
def firstkey(self):
if not self.keys():
return None
self.key_index = 1
return self.keys()[0]
def nextkey(self, key):
if len(self.keys()) <= self.key_index:
return None
else:
self.key_index += 1
return self.keys()[self.key_index]
def sync(self):
pass
def reorganize(self):
pass
class GdbmTest(unittest.TestCase):
"""Test the GdbmDBHandle class"""
max_age = 60 * 60 * 24 * 30 * 4
r_count = 24
wl_count = 42
entered = datetime.now() - timedelta(days=10)
updated = datetime.now() - timedelta(days=2)
wl_entered = datetime.now() - timedelta(days=20)
wl_updated = datetime.now() - timedelta(days=3)
def setUp(self):
unittest.TestCase.setUp(self)
self.real_timer = threading.Timer
threading.Timer = MockTimer
self.db = MockGdbm()
def mock_open(fn, mode):
return self.db
self.real_open = gdbm.open
gdbm.open = mock_open
self.record = pyzor.engines.common.Record(self.r_count, self.wl_count,
self.entered, self.updated,
self.wl_entered, self.wl_updated)
def tearDown(self):
unittest.TestCase.tearDown(self)
threading.Timer = self.real_timer
gdbm.open = self.real_open
def record_as_str(self, record=None):
if not record:
record = self.record
return ("1,%s,%s,%s,%s,%s,%s" % (record.r_count, record.r_entered,
record.r_updated, record.wl_count,
record.wl_entered, record.wl_updated)).encode("utf8")
def test_set_item(self):
"""Test GdbmDBHandle.__setitem__"""
digest = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
handle = pyzor.engines.gdbm_.GdbmDBHandle(None, None,
max_age=self.max_age)
handle[digest] = self.record
self.assertEqual(self.db[digest], self.record_as_str().decode("utf8"))
def test_get_item(self):
"""Test GdbmDBHandle.__getitem__"""
digest = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
handle = pyzor.engines.gdbm_.GdbmDBHandle(None, None,
max_age=self.max_age)
self.db[digest] = self.record_as_str()
result = handle[digest]
self.assertEqual(self.record_as_str(result), self.record_as_str())
def test_del_item(self):
"""Test GdbmDBHandle.__delitem__"""
digest = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
handle = pyzor.engines.gdbm_.GdbmDBHandle(None, None,
max_age=self.max_age)
self.db[digest] = self.record_as_str()
del handle[digest]
self.assertFalse(self.db.get(digest))
def test_reorganize_older(self):
"""Test GdbmDBHandle.start_reorganizing with older records"""
digest = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
self.db[digest] = self.record_as_str()
handle = pyzor.engines.gdbm_.GdbmDBHandle(None, None,
max_age=3600 * 24)
self.assertFalse(self.db.get(digest))
def test_reorganize_older_no_max_age(self):
"""Test GdbmDBHandle.start_reorganizing with older records, but no
max_age set.
"""
digest = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
self.db[digest] = self.record_as_str()
handle = pyzor.engines.gdbm_.GdbmDBHandle(None, None,
max_age=None)
self.assertEqual(self.db[digest], self.record_as_str())
def test_reorganize_fresh(self):
"""Test GdbmDBHandle.start_reorganizing with newer records"""
digest = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
self.db[digest] = self.record_as_str()
handle = pyzor.engines.gdbm_.GdbmDBHandle(None, None,
max_age=3600 * 24 * 3)
self.assertEqual(self.db[digest], self.record_as_str())
def suite():
"""Gather all the tests from this module in a test suite."""
test_suite = unittest.TestSuite()
test_suite.addTest(unittest.makeSuite(GdbmTest))
return test_suite
if __name__ == '__main__':
unittest.main()

View File

@@ -0,0 +1,156 @@
"""Test the pyzor.engines.mysql module."""
import unittest
import threading
from datetime import datetime, timedelta
import pyzor.engines
import pyzor.engines.mysql
import pyzor.engines.common
class MockTimer():
def __init__(self, *args, **kwargs):
pass
def start(self):
pass
def setDaemon(self, daemon):
pass
def make_MockMySQL(result, queries):
class MockCursor():
def __init__(self):
pass
def fetchone(self):
return result
def fetchall(self):
return [result]
def execute(self, query, args=None):
queries.append((query, args))
def close(self):
pass
class MockDB():
def cursor(self):
return MockCursor()
def close(self):
pass
def commit(self):
pass
def autocommit(self, value):
pass
class MockMysql():
@staticmethod
def connect(*args, **kwargs):
return MockDB()
class Error(Exception):
pass
return MockMysql
class MySQLTest(unittest.TestCase):
"""Test the GdbmDBHandle class"""
max_age = 60 * 60 * 24 * 30 * 4
r_count = 24
wl_count = 42
entered = datetime.now() - timedelta(days=10)
updated = datetime.now() - timedelta(days=2)
wl_entered = datetime.now() - timedelta(days=20)
wl_updated = datetime.now() - timedelta(days=3)
def setUp(self):
unittest.TestCase.setUp(self)
self.real_timer = threading.Timer
threading.Timer = MockTimer
self.record = pyzor.engines.common.Record(self.r_count, self.wl_count,
self.entered, self.updated,
self.wl_entered, self.wl_updated)
self.response = self.record_unpack()
self.queries = []
mock_MySQL = make_MockMySQL(self.response, self.queries)
self.real_mysql = pyzor.engines.mysql.MySQLdb
pyzor.engines.mysql.MySQLdb = mock_MySQL
def tearDown(self):
unittest.TestCase.tearDown(self)
threading.Timer = self.real_timer
pyzor.engines.mysql.MySQLdb = self.real_mysql
def record_unpack(self, record=None):
if not record:
record = self.record
return (record.r_count, record.wl_count,
record.r_entered, record.r_updated,
record.wl_entered, record.wl_updated)
def test_reconnect(self):
"""Test MySQLDBHandle.__init__"""
expected = "DELETE FROM testtable WHERE r_updated<%s"
pyzor.engines.mysql.MySQLDBHandle("testhost,testuser,testpass,testdb,testtable",
None, max_age=self.max_age)
self.assertEqual(self.queries[0][0], expected)
def test_no_reorganize(self):
pyzor.engines.mysql.MySQLDBHandle("testhost,testuser,testpass,testdb,testtable",
None, max_age=None)
self.assertFalse(self.queries)
def test_set_item(self):
"""Test MySQLDBHandle.__setitem__"""
digest = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
expected = ("INSERT INTO testtable (digest, r_count, wl_count, "
"r_entered, r_updated, wl_entered, wl_updated) "
"VALUES (%s, %s, %s, %s, %s, %s, %s) ON "
"DUPLICATE KEY UPDATE r_count=%s, wl_count=%s, "
"r_entered=%s, r_updated=%s, wl_entered=%s, "
"wl_updated=%s",
(digest, self.r_count, self.wl_count, self.entered,
self.updated, self.wl_entered, self.wl_updated,
self.r_count, self.wl_count, self.entered,
self.updated, self.wl_entered, self.wl_updated))
handle = pyzor.engines.mysql.MySQLDBHandle("testhost,testuser,testpass,testdb,testtable",
None, max_age=self.max_age)
handle[digest] = self.record
self.assertEqual(self.queries[1], expected)
def test_get_item(self):
"""Test MySQLDBHandle.__getitem__"""
digest = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
expected = ("SELECT r_count, wl_count, r_entered, r_updated, "
"wl_entered, wl_updated FROM testtable WHERE digest=%s",
(digest,))
handle = pyzor.engines.mysql.MySQLDBHandle("testhost,testuser,testpass,testdb,testtable",
None, max_age=self.max_age)
result = handle[digest]
self.assertEqual(self.queries[1], expected)
self.assertEqual(self.record_unpack(result), self.record_unpack())
def test_del_item(self):
"""Test MySQLDBHandle.__detitem__"""
digest = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
expected = ("DELETE FROM testtable WHERE digest=%s", (digest,))
handle = pyzor.engines.mysql.MySQLDBHandle("testhost,testuser,testpass,testdb,testtable",
None, max_age=self.max_age)
del handle[digest]
self.assertEqual(self.queries[1], expected)
def suite():
"""Gather all the tests from this module in a test suite."""
test_suite = unittest.TestSuite()
test_suite.addTest(unittest.makeSuite(MySQLTest))
return test_suite
if __name__ == '__main__':
unittest.main()

View File

@@ -0,0 +1,188 @@
"""Test the pyzor.engines.gdbm_ module."""
import unittest
from datetime import datetime, timedelta
import pyzor.engines
import pyzor.engines.redis_
import pyzor.engines.common
class EncodingRedisTest(unittest.TestCase):
"""Test the RedisDBHandle class"""
r_count = 24
wl_count = 42
entered = datetime(2014, 4, 23, 15, 41, 30)
updated = datetime(2014, 4, 25, 17, 22, 25)
wl_entered = datetime(2014, 2, 12, 11, 10, 55)
wl_updated = datetime(2014, 3, 25, 5, 1, 50)
def setUp(self):
unittest.TestCase.setUp(self)
self.record = pyzor.engines.common.Record(self.r_count, self.wl_count,
self.entered, self.updated,
self.wl_entered, self.wl_updated)
def compare_records(self, r1, r2):
attrs = ("r_count", "r_entered", "r_updated",
"wl_count", "wl_entered", "wl_updated")
self.assertTrue(all(getattr(r1, attr) == getattr(r2, attr)
for attr in attrs))
def tearDown(self):
unittest.TestCase.tearDown(self)
def test_encode_record(self):
expected = ("24,2014-04-23 15:41:30,2014-04-25 17:22:25,"
"42,2014-02-12 11:10:55,2014-03-25 05:01:50").encode()
result = pyzor.engines.redis_.RedisDBHandle._encode_record(self.record)
self.assertEqual(result, expected)
def test_encode_record_no_date(self):
expected = ("24,2014-04-23 15:41:30,,"
"42,2014-02-12 11:10:55,2014-03-25 05:01:50").encode()
self.record.r_updated = None
result = pyzor.engines.redis_.RedisDBHandle._encode_record(self.record)
self.assertEqual(result, expected)
def test_encode_record_no_white(self):
expected = ("24,2014-04-23 15:41:30,2014-04-25 17:22:25,"
"0,,").encode()
self.record.wl_count = 0
self.record.wl_entered = None
self.record.wl_updated = None
result = pyzor.engines.redis_.RedisDBHandle._encode_record(self.record)
self.assertEqual(result, expected)
def test_decode_record(self):
encoded = ("24,2014-04-23 15:41:30,2014-04-25 17:22:25,"
"42,2014-02-12 11:10:55,2014-03-25 05:01:50").encode()
result = pyzor.engines.redis_.RedisDBHandle._decode_record(encoded)
self.compare_records(result, self.record)
def test_decode_record_no_date(self):
encoded = ("24,2014-04-23 15:41:30,,"
"42,2014-02-12 11:10:55,2014-03-25 05:01:50").encode()
result = pyzor.engines.redis_.RedisDBHandle._decode_record(encoded)
self.record.r_updated = None
self.compare_records(result, self.record)
def test_decode_record_no_white(self):
encoded = ("24,2014-04-23 15:41:30,2014-04-25 17:22:25,"
"0,,").encode()
result = pyzor.engines.redis_.RedisDBHandle._decode_record(encoded)
self.record.wl_count = 0
self.record.wl_entered = None
self.record.wl_updated = None
self.compare_records(result, self.record)
def make_MockRedis(commands):
class MockRedis():
def __init__(self, *args, **kwargs):
commands.append(("init", args, kwargs))
def set(self, *args, **kwargs):
commands.append(("set", args, kwargs))
def setex(self, *args, **kwargs):
commands.append(("setex", args, kwargs))
def get(self, *args, **kwargs):
commands.append(("get", args, kwargs))
def delete(self, *args, **kwargs):
commands.append(("delete", args, kwargs))
return MockRedis
mock_encode_record = lambda s, x: x
mock_decode_record = lambda s, x: x
class RedisTest(unittest.TestCase):
max_age = 60 * 60
def setUp(self):
unittest.TestCase.setUp(self)
self.commands = []
self.real_redis = pyzor.engines.redis_.redis.StrictRedis
self.real_encode = pyzor.engines.redis_.RedisDBHandle._encode_record
self.real_decode = pyzor.engines.redis_.RedisDBHandle._decode_record
pyzor.engines.redis_.redis.StrictRedis = make_MockRedis(self.commands)
pyzor.engines.redis_.RedisDBHandle._encode_record = mock_encode_record
pyzor.engines.redis_.RedisDBHandle._decode_record = mock_decode_record
def tearDown(self):
unittest.TestCase.tearDown(self)
pyzor.engines.redis_.redis.StrictRedis = self.real_redis
pyzor.engines.redis_.RedisDBHandle._encode_record = self.real_encode
pyzor.engines.redis_.RedisDBHandle._decode_record = self.real_decode
def test_init(self):
expected = {"host": "example.com",
"port": 6387,
"password": "passwd",
"db": 5,
}
db = pyzor.engines.redis_.RedisDBHandle("example.com,6387,passwd,5",
None)
self.assertEqual(self.commands[0], ("init", (), expected))
def test_init_defaults(self):
expected = {"host": "localhost",
"port": 6379,
"password": None,
"db": 0,
}
db = pyzor.engines.redis_.RedisDBHandle(",,,", None)
self.assertEqual(self.commands[0], ("init", (), expected))
def test_set(self):
digest = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
value = "record test"
db = pyzor.engines.redis_.RedisDBHandle(",,,", None)
db[digest] = value
expected = ("pyzord.digest.%s" % digest, value)
self.assertEqual(self.commands[1], ("set", expected, {}))
def test_set_max_age(self):
digest = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
value = "record test"
db = pyzor.engines.redis_.RedisDBHandle(",,,", None,
max_age=self.max_age)
db[digest] = value
expected = ("pyzord.digest.%s" % digest, self.max_age, value)
self.assertEqual(self.commands[1], ("setex", expected, {}))
def test_get(self):
digest = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
db = pyzor.engines.redis_.RedisDBHandle(",,,", None)
result = db[digest]
expected = ("pyzord.digest.%s" % digest,)
self.assertEqual(self.commands[1], ("get", expected, {}))
def test_delete(self):
digest = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
db = pyzor.engines.redis_.RedisDBHandle(",,,", None)
del db[digest]
expected = ("pyzord.digest.%s" % digest,)
self.assertEqual(self.commands[1], ("delete", expected, {}))
def suite():
"""Gather all the tests from this module in a test suite."""
test_suite = unittest.TestSuite()
test_suite.addTest(unittest.makeSuite(EncodingRedisTest))
test_suite.addTest(unittest.makeSuite(RedisTest))
return test_suite
if __name__ == '__main__':
unittest.main()

View File

@@ -0,0 +1,341 @@
"""Test the pyzor.server module
"""
import io
import sys
import time
import logging
import hashlib
import unittest
import SocketServer
from datetime import datetime, timedelta
import pyzor
import pyzor.server
import pyzor.engines.common
class MockServer():
"""Mocks the pyzor.server.Server class"""
def __init__(self):
self.log = logging.getLogger("pyzord")
self.usage_log = logging.getLogger("pyzord-usage")
self.log.addHandler(logging.NullHandler())
self.usage_log.addHandler(logging.NullHandler())
class MockDatagramRequestHandler():
""" Mock the SocketServer.DatagramRequestHand."""
def __init__(self, headers, database=None, acl=None, accounts=None):
"""Initiates an request handler and set's the data in `headers` as
the request. Also set's the database, acl and accounts for the
MockServer.
This will be set as base class for RequestHandler.
"""
self.rfile = io.BytesIO()
self.wfile = io.BytesIO()
for i, j in headers.iteritems():
self.rfile.write(("%s: %s\n" % (i, j)).encode("utf8"))
self.rfile.seek(0)
self.packet = None
self.client_address = ["127.0.0.1"]
# Setup MockServer data
self.server = MockServer()
self.server.database = database
if acl:
self.server.acl = acl
else:
self.server.acl = {pyzor.anonymous_user: ("check", "report", "ping", "info", "whitelist",)}
self.server.accounts = accounts
self.handle()
def handle(self):
pass
class RequestHandlerTest(unittest.TestCase):
def setUp(self):
unittest.TestCase.setUp(self)
self.real_drh = SocketServer.DatagramRequestHandler
SocketServer.DatagramRequestHandler = MockDatagramRequestHandler
pyzor.server.RequestHandler.__bases__ = (MockDatagramRequestHandler,)
# setup the basic values for request and response
self.request = {"User": pyzor.anonymous_user,
"Time": str(int(time.time())),
"PV": str(pyzor.proto_version),
"Thread": "3597"}
self.expected_response = {"Code": "200",
"Diag": "OK",
"PV": str(pyzor.proto_version),
"Thread": "3597"}
def tearDown(self):
unittest.TestCase.tearDown(self)
SocketServer.DatagramRequestHandler = self.real_drh
pyzor.server.RequestHandler.__bases__ = (self.real_drh,)
def check_response(self, handler):
"""Checks if the response from the handler is equal to
the expected response.
"""
handler.wfile.seek(0)
response = handler.wfile.read()
response = response.decode("utf8").replace("\n\n", "\n")
result = {}
for line in response.splitlines():
key = line.split(":", 1)[0].strip()
value = line.split(":")[1].strip()
result[key] = value
self.assertEqual(result, self.expected_response)
def timestamp(self, time_obj):
if not time_obj:
return 0
else:
return str(int(time.mktime(time_obj.timetuple())))
def test_ping(self):
"""Tests the ping command handler"""
self.request["Op"] = "ping"
handler = pyzor.server.RequestHandler(self.request)
self.check_response(handler)
def test_pong(self):
"""Tests the pong command handler"""
digest = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
database = {digest: pyzor.engines.common.Record(24, 42)}
self.request["Op"] = "pong"
self.request["Op-Digest"] = digest
handler = pyzor.server.RequestHandler(self.request, database)
self.expected_response["Count"] = str(sys.maxint)
self.expected_response["WL-Count"] = "0"
def test_check(self):
"""Tests the check command handler"""
digest = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
database = {digest: pyzor.engines.common.Record(24, 42)}
self.request["Op"] = "check"
self.request["Op-Digest"] = digest
handler = pyzor.server.RequestHandler(self.request, database)
self.expected_response["Count"] = "24"
self.expected_response["WL-Count"] = "42"
self.check_response(handler)
def test_check_new(self):
"""Tests the check command handler with a new record"""
digest = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
database = {}
self.request["Op"] = "check"
self.request["Op-Digest"] = digest
handler = pyzor.server.RequestHandler(self.request, database)
self.expected_response["Count"] = "0"
self.expected_response["WL-Count"] = "0"
self.check_response(handler)
def test_info(self):
"""Tests the info command handler"""
entered = datetime.now() - timedelta(days=10)
updated = datetime.now()
wl_entered = datetime.now() - timedelta(days=20)
wl_updated = datetime.now() - timedelta(days=2)
digest = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
database = {digest: pyzor.engines.common.Record(24, 42, entered, updated,
wl_entered, wl_updated)}
self.request["Op"] = "info"
self.request["Op-Digest"] = digest
handler = pyzor.server.RequestHandler(self.request, database)
self.expected_response["Count"] = "24"
self.expected_response["WL-Count"] = "42"
self.expected_response["Entered"] = self.timestamp(entered)
self.expected_response["Updated"] = self.timestamp(updated)
self.expected_response["WL-Entered"] = self.timestamp(wl_entered)
self.expected_response["WL-Updated"] = self.timestamp(wl_updated)
self.check_response(handler)
def test_info_new(self):
"""Tests the info command handler with a new record"""
digest = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
database = {}
self.request["Op"] = "info"
self.request["Op-Digest"] = digest
handler = pyzor.server.RequestHandler(self.request, database)
self.expected_response["Count"] = "0"
self.expected_response["WL-Count"] = "0"
self.expected_response["Entered"] = "0"
self.expected_response["Updated"] = "0"
self.expected_response["WL-Entered"] = "0"
self.expected_response["WL-Updated"] = "0"
self.check_response(handler)
def test_report(self):
"""Tests the report command handler"""
digest = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
database = {digest: pyzor.engines.common.Record(24, 42)}
self.request["Op"] = "report"
self.request["Op-Digest"] = digest
handler = pyzor.server.RequestHandler(self.request, database)
self.check_response(handler)
self.assertEqual(database[digest].r_count, 25)
def test_report_new(self):
"""Tests the report command handler with a new record"""
digest = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
database = {}
self.request["Op"] = "report"
self.request["Op-Digest"] = digest
handler = pyzor.server.RequestHandler(self.request, database)
self.check_response(handler)
self.assertEqual(database[digest].r_count, 1)
def test_whitelist(self):
"""Tests the whitelist command handler"""
digest = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
database = {digest: pyzor.engines.common.Record(24, 42)}
self.request["Op"] = "whitelist"
self.request["Op-Digest"] = digest
handler = pyzor.server.RequestHandler(self.request, database)
self.check_response(handler)
self.assertEqual(database[digest].wl_count, 43)
def test_whitelist_new(self):
"""Tests the whitelist command handler with a new record"""
digest = "2aedaac999d71421c9ee49b9d81f627a7bc570aa"
database = {}
self.request["Op"] = "whitelist"
self.request["Op-Digest"] = digest
handler = pyzor.server.RequestHandler(self.request, database)
self.check_response(handler)
self.assertEqual(database[digest].wl_count, 1)
def test_handle_no_version(self):
"""Tests handling an request with no version specified"""
self.request["Op"] = "ping"
del self.request["PV"]
handler = pyzor.server.RequestHandler(self.request)
self.expected_response["Code"] = "400"
self.expected_response["Diag"] = "Bad request"
self.check_response(handler)
def test_handle_unsupported_version(self):
"""Tests handling an request with an unsupported version specified"""
self.request["Op"] = "ping"
self.request["PV"] = str(pyzor.proto_version + 2)
handler = pyzor.server.RequestHandler(self.request)
self.expected_response["Code"] = "505"
self.expected_response["Diag"] = "Version Not Supported"
self.check_response(handler)
def test_handle_not_implemented(self):
"""Tests handling an request with an unimplemented command"""
self.request["Op"] = "notimplemented"
acl = {pyzor.anonymous_user: "notimplemented"}
handler = pyzor.server.RequestHandler(self.request, acl=acl)
self.expected_response["Code"] = "501"
self.expected_response["Diag"] = "Not implemented"
self.check_response(handler)
def test_handle_unauthorized(self):
"""Tests handling an request with an unauthorized command"""
self.request["Op"] = "report"
acl = {pyzor.anonymous_user: ("ping", "check")}
handler = pyzor.server.RequestHandler(self.request, acl=acl)
self.expected_response["Code"] = "403"
self.expected_response["Diag"] = "Forbidden"
self.check_response(handler)
def test_handle_account(self):
"""Tests handling an request where user is not anonymous"""
self.request["Op"] = "ping"
self.request["User"] = "testuser"
acl = {"testuser": ("ping", "check")}
accounts = {"testuser": "testkey"}
mock_vs = lambda x, y: None
real_vs = pyzor.account.verify_signature
pyzor.account.verify_signature = mock_vs
try:
handler = pyzor.server.RequestHandler(self.request, acl=acl,
accounts=accounts)
self.check_response(handler)
finally:
pyzor.account.verify_signature = real_vs
def test_handle_unknown_account(self):
"""Tests handling an request where user is unkwown"""
self.request["Op"] = "ping"
self.request["User"] = "testuser"
acl = {"testuser": ("ping", "check")}
accounts = {}
self.expected_response["Code"] = "401"
self.expected_response["Diag"] = "Unauthorized"
def mock_vs(x, y):
pass
real_vs = pyzor.account.verify_signature
pyzor.account.verify_signature = mock_vs
try:
handler = pyzor.server.RequestHandler(self.request, acl=acl,
accounts=accounts)
self.check_response(handler)
finally:
pyzor.account.verify_signature = real_vs
def test_handle_invalid_signature(self):
"""Tests handling an request where user key is invalid"""
self.request["Op"] = "ping"
self.request["User"] = "testuser"
acl = {"testuser": ("ping", "check")}
accounts = {"testuser": ("ping", "check")}
self.expected_response["Code"] = "401"
self.expected_response["Diag"] = "Unauthorized"
def mock_vs(x, y):
raise pyzor.SignatureError("Invalid signature.")
real_vs = pyzor.account.verify_signature
pyzor.account.verify_signature = mock_vs
try:
handler = pyzor.server.RequestHandler(self.request, acl=acl,
accounts=accounts)
self.check_response(handler)
finally:
pyzor.account.verify_signature = real_vs
def suite():
"""Gather all the tests from this module in a test suite."""
test_suite = unittest.TestSuite()
test_suite.addTest(unittest.makeSuite(RequestHandlerTest))
return test_suite
if __name__ == '__main__':
unittest.main()

View File

@@ -0,0 +1,341 @@
"""This package contains various utilities use in the pyzor tests."""
import os
import sys
import time
import shutil
import unittest
import subprocess
import ConfigParser
from datetime import datetime, timedelta
msg = """Newsgroups:
Date: Wed, 10 Apr 2002 22:23:51 -0400 (EDT)
From: Frank Tobin <ftobin@neverending.org>
Fcc: sent-mail
Message-ID: <20020410222350.E16178@palanthas.neverending.org>
X-Our-Headers: X-Bogus,Anon-To
X-Bogus: aaron7@neverending.org
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Test Email
"""
digest = "7421216f915a87e02da034cc483f5c876e1a1338"
_dt_decode = lambda x: None if x == 'None' else datetime.strptime(x, "%a %b %d %H:%M:%S %Y")
class PyzorTestBase(unittest.TestCase):
"""Test base that starts the pyzord daemon in setUpClass with specified
arguments. The daemon is killed in tearDownClass. This also create the
necessary files and the homedir.
"""
pyzord = None
_args = {"homedir": "--homedir",
"engine": "-e",
"dsn": "--dsn",
"address": "-a",
"port": "-p",
"threads": "--threads",
"max_threads": "--max-threads",
"processes": "--processes",
"max_processes": "--max-processes",
"db_connections": "--db-connections",
"password_file": "--password-file",
"access_file": "--access-file",
"cleanup_age": "--cleanup-age",
"log_file": "--log-file",
}
homedir = "./pyzor-test/"
address = "127.0.0.1"
port = "9999"
threads = "False"
access_file = "pyzord.access"
password_file = "pyzord.passwd"
log_file = "pyzord-test.log"
access = """check report ping pong info whitelist : alice : deny
check report ping pong info whitelist : bob : allow
ALL : dan : allow
pong info whitelist : dan : deny
"""
passwd = """alice : fc7f1cad729b5f3862b2ef192e2d9e0d0d4bd515
bob : cf88277c5d4abdc0a3f56f416011966d04a3f462
dan : c1a50281fc43e860fe78c16c73b9618ada59f959
"""
servers = """127.0.0.1:9999
"""
accounts_alice = """127.0.0.1 : 9999 : alice : d28f86151e80a9accba4a4eba81c460532384cd6,fc7f1cad729b5f3862b2ef192e2d9e0d0d4bd515
"""
accounts_bob = """127.0.0.1 : 9999 : bob : de6ef568787256bf5f55909dc0c398e49b5c9808,cf88277c5d4abdc0a3f56f416011966d04a3f462
"""
accounts_chuck = """127.0.0.1 : 9999 : bob : de6ef568787256bf5f55909dc0c398e49b5c9808,af88277c5d4abdc0a3f56f416011966d04a3f462
"""
accounts_dan = """127.0.0.1 : 9999 : dan : 1cc2efa77d8833d83556e0cc4fa617c64eebc7fb,c1a50281fc43e860fe78c16c73b9618ada59f959
"""
@classmethod
def write_homedir_file(cls, name, content):
if not name or not content:
return
with open(os.path.join(cls.homedir, name), "w") as f:
f.write(content)
@classmethod
def setUpClass(cls):
super(PyzorTestBase, cls).setUpClass()
try:
os.mkdir(cls.homedir)
except OSError:
pass
cls.write_homedir_file(cls.access_file, cls.access)
cls.write_homedir_file(cls.password_file, cls.passwd)
cls.write_homedir_file(cls.password_file, cls.passwd)
cls.write_homedir_file("servers", cls.servers)
cls.write_homedir_file("alice", cls.accounts_alice)
cls.write_homedir_file("bob", cls.accounts_bob)
cls.write_homedir_file("chuck", cls.accounts_chuck)
cls.write_homedir_file("dan", cls.accounts_dan)
args = ["pyzord"]
for key, value in cls._args.iteritems():
option = getattr(cls, key, None)
if option:
args.append(value)
args.append(option)
cls.pyzord = subprocess.Popen(args)
time.sleep(0.3) # allow time to initialize server
def setUp(self):
unittest.TestCase.setUp(self)
self.client_args = {"--homedir": self.homedir,
"--servers-file": "servers",
"-t": None, # timeout
"-r": None, # report threshold
"-w": None, # whitelist threshold
"-s": None, # style
}
def tearDown(self):
unittest.TestCase.tearDown(self)
@classmethod
def tearDownClass(cls):
super(PyzorTestBase, cls).tearDownClass()
cls.pyzord.kill()
shutil.rmtree(cls.homedir, True)
def check_pyzor(self, cmd, user, input=None,
code=None, exit_code=None, counts=()):
"""Call the pyzor client with the specified args from self.client_args
and verifies the response.
"""
args = ["pyzor"]
if user:
args.append("--accounts-file")
args.append(user)
for key, value in self.client_args.iteritems():
if value:
args.append(key)
args.append(value)
args.append(cmd)
pyzor = subprocess.Popen(args,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
if input:
stdout, stderr = pyzor.communicate(input.encode("utf8"))
else:
stdout, stderr = pyzor.communicate()
if stderr:
self.fail(stderr)
if code is not None:
try:
stdout = stdout.decode("utf8")
results = stdout.strip().split("\t")
status = eval(results[1])
except Exception as e:
self.fail("Parsing error: %s of %r" % (e, stdout))
self.assertEqual(status[0], code, status)
if counts:
self.assertEqual(counts, (int(results[2]), int(results[3])))
if exit_code is not None:
self.assertEqual(exit_code, pyzor.returncode)
return stdout
def check_pyzor_multiple(self, cmd, user, input=None,
code=None, exit_code=None, counts=()):
"""Call the pyzor client with the specified args from self.client_args
and verifies the response.
"""
args = ["pyzor"]
if user:
args.append("--accounts-file")
args.append(user)
for key, value in self.client_args.iteritems():
if value:
args.append(key)
args.append(value)
args.append(cmd)
pyzor = subprocess.Popen(args,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
if input:
stdout, stderr = pyzor.communicate(input.encode("utf8"))
else:
stdout, stderr = pyzor.communicate()
if stderr:
self.fail(stderr)
stdout = stdout.decode("utf8")
for i, line in enumerate(stdout.splitlines()):
try:
line = line.strip()
if not line:
continue
results = line.strip().split("\t")
except Exception as e:
self.fail("Parsing error: %s of %r" % (e, stdout))
if code is not None:
try:
status = eval(results[1])
except Exception as e:
self.fail("Parsing error: %s of %r" % (e, stdout))
self.assertEqual(status[0], code[i], status)
if counts:
self.assertEqual((int(results[2]), int(results[3])),
counts[i])
if exit_code is not None:
self.assertEqual(exit_code, pyzor.returncode)
return stdout
def get_record(self, input, user="bob"):
"""Uses `pyzor info` to get the record data."""
stdout = self.check_pyzor("info", user, input, code=200, exit_code=0)
info = stdout.splitlines()[1:]
record = {}
try:
for line in info:
line = line.strip()
if not line:
continue
key, value = line.split(":", 1)
record[key.strip()] = value.strip()
except Exception as e:
self.fail("Error parsing %r: %s" % (info, e))
return record
def check_fuzzy_date(self, date1, date2=None, seconds=5):
"""Check if the given date is almost equal to now."""
date1 = _dt_decode(date1)
if not date2:
date2 = datetime.now()
delta = abs((date2 - date1).total_seconds())
if delta > seconds:
self.fail("Delta %s is too big: %s, %s" % (delta , date1, date2))
class PyzorTest(object):
"""MixIn class for PyzorTestBase that performs a series of basic tests."""
def test_ping(self):
self.check_pyzor("ping", "bob")
def test_pong(self):
input = "Test1 pong1 Test2"
self.check_pyzor("pong", "bob", input=input, code=200, exit_code=0,
counts=(sys.maxint, 0))
def test_check(self):
input = "Test1 check1 Test2"
self.check_pyzor("check", "bob", input=input, code=200, exit_code=1,
counts=(0, 0))
r = self.get_record(input)
self.assertEqual(r["Count"], "0")
def test_report(self):
input = "Test1 report1 Test2"
self.check_pyzor("report", "bob", input=input, code=200, exit_code=0)
self.check_pyzor("check", "bob", input=input, code=200, exit_code=0,
counts=(1, 0))
r = self.get_record(input)
self.assertEqual(r["Count"], "1")
self.check_fuzzy_date(r["Entered"])
def test_report_update(self):
input = "Test1 report update1 Test2"
self.check_pyzor("report", "bob", input=input, code=200, exit_code=0)
self.check_pyzor("check", "bob", input=input, code=200, exit_code=0,
counts=(1, 0))
time.sleep(1)
self.check_pyzor("report", "bob", input=input, code=200, exit_code=0)
self.check_pyzor("check", "bob", input=input, code=200, exit_code=0,
counts=(2, 0))
r = self.get_record(input)
self.assertEqual(r["Count"], "2")
self.assertNotEqual(r["Entered"], r["Updated"])
self.check_fuzzy_date(r["Updated"])
def test_whitelist(self):
input = "Test1 white list1 Test2"
self.check_pyzor("whitelist", "bob", input=input, code=200, exit_code=0)
self.check_pyzor("check", "bob", input=input, code=200, exit_code=1,
counts=(0, 1))
r = self.get_record(input)
self.assertEqual(r["WL-Count"], "1")
self.check_fuzzy_date(r["WL-Entered"])
def test_whitelist_update(self):
input = "Test1 white list update1 Test2"
self.check_pyzor("whitelist", "bob", input=input, code=200, exit_code=0)
self.check_pyzor("check", "bob", input=input, code=200, exit_code=1,
counts=(0, 1))
time.sleep(1)
self.check_pyzor("whitelist", "bob", input=input, code=200, exit_code=0)
self.check_pyzor("check", "bob", input=input, code=200, exit_code=1,
counts=(0, 2))
r = self.get_record(input)
self.assertEqual(r["WL-Count"], "2")
self.assertNotEqual(r["WL-Entered"], r["WL-Updated"])
self.check_fuzzy_date(r["WL-Updated"])
def test_report_whitelist(self):
input = "Test1 white list report1 Test2"
self.check_pyzor("whitelist", "bob", input=input, code=200, exit_code=0)
self.check_pyzor("report", "bob", input=input, code=200, exit_code=0)
self.check_pyzor("check", "bob", input=input, code=200, exit_code=1,
counts=(1, 1))
r = self.get_record(input)
self.assertEqual(r["Count"], "1")
self.check_fuzzy_date(r["Entered"])
self.assertEqual(r["WL-Count"], "1")
self.check_fuzzy_date(r["WL-Entered"])
def test_report_whitelist_update(self):
input = "Test1 white list report update1 Test2"
self.check_pyzor("whitelist", "bob", input=input, code=200, exit_code=0)
self.check_pyzor("report", "bob", input=input, code=200, exit_code=0)
self.check_pyzor("check", "bob", input=input, code=200, exit_code=1,
counts=(1, 1))
time.sleep(1)
self.check_pyzor("whitelist", "bob", input=input, code=200, exit_code=0)
self.check_pyzor("report", "bob", input=input, code=200, exit_code=0)
self.check_pyzor("check", "bob", input=input, code=200, exit_code=1,
counts=(2, 2))
r = self.get_record(input)
self.assertEqual(r["Count"], "2")
self.assertNotEqual(r["Entered"], r["Updated"])
self.check_fuzzy_date(r["Updated"])
self.assertEqual(r["WL-Count"], "2")
self.assertNotEqual(r["WL-Entered"], r["WL-Updated"])
self.check_fuzzy_date(r["WL-Updated"])