On Wed, Jan 21, 2009 at 12:48:31PM +1100, Stephen Vaughan wrote:Hi Stephen,
> Hi Simon,
>
> Basically we have a webmail service, which connects to 4 different IMAP
> servers via perdition. As I said there is 13,000+ entries in popmap.re,
> using this format: (.*)@domain.com: 123.123.123.123
>
> We have a script which reloads perdition every hour as changes are made to
> popmap.re frequently. The problem is when a user first attempts to login
> they experience a delay in authenticating. Each time this happens the
> perdition.imap4 process consumes 100% cpu for about 60 seconds, after about
> 30 seconds there are 3 perdition.imap4 processes all consuming cpu usage.
>
> I suspect that because the contents of popmap.re is so large, perdition is
> taking longer to process the request.
I'm a little surprised that perdition is slow in the way that you describe,
but I'm not surprised that it is slow in some way with your configuration.
When I wrote the popre code (which was rather a long time ago) I wasn't
thinking in terms of it scaling to 13,000+ entries. The implementation
is rather simple and works in two parts as follows:
on load or reload:
Run through the popmap.re file and for each entry compile it as a regex
All tresultsand store the result in an array
on lookup:
Run through the array in order, checking each regex in turn.
Return when a match is fine.
So basically both load/reload and lookup is O(n), where n=13,000+.
Clearly not sastifactory.
Do you actually need the regex? If you are just mapping
domain names to IP addresses, then I think that you can use
the MySQL, PostgreSQL, ODBC, BDB, GDBD, LDAP or even NIS module.
The trick is to set the query key to so that only the domain
name supplied by the username is passed to the db module to
perform the lookup.
This is done by setting query_key in the perdition configuration file
or on the command line:
query_key=\d
--
Simon Horman
VA Linux Systems Japan K.K., Sydney, Australia Satellite Office
H: www.vergenet.net/~horms/ W: www.valinux.co.jp/en