NAME
calamaris - generate statistic out of log files from
Proxy-Cache-Servers
SYNOPSIS
cat log | calamaris [ calamaris_options ]
DESCRIPTION
Calamaris is used to produce statistical
output from Squid, NetCache, Inktomi Traffic
Server, Oops! proxy server, Compaq Tasksmart,
Cisco Content Engines or related Proxy log files. The
resulting output can be ascii or html. It is possible
to cache calculated data in a file to use them in later runs.
This manual page describes the options of Calamaris and
gives a few examples.
OPTIONS
Reports
- -a
- all: extracts all reports available, -a equals -D 10
-d 20 -P 60 -r 20 -s -t 20.
- -d n
- domain report: switches the top level and the second level
report on. The data is derived from the URL. The output is limited
by n. (-1 is unlimited)
- -p type
- peak report: Measures the peaks of the Proxy usage in requests
per second, per minute and per hour. It reports peaks for TCP, UDP
and ALL requests. If set to 'old' these values were
calculated with the old slow method, if set to 'new' the new
faster (but still slow) method is used.
- -P n
- performance report: Shows the throughput of TCP requests for
every n minutes.
- -r n
- requester report: Switches the UDP and TCP requester reports
on. The output is limited by n. (-1 is unlimited)
- -R n
- Requested report: adds to each line of the requester report the
requested URLs. The output is limited by n. (-1 is unlimited, and
can result in very very long reports.)
Warning: Using this option breaks the privacy of your
users! (see PRIVACY-Section below)
- -s
- status report: alters the default behaviour of Calamaris
and makes the status reports more verbose.
- -t n
- type report: switches the content type and the file extension
report on. The output is limited by n. (-1 is unlimited)
- -D n
- Distribution Histogram: shows size-based distribution of
requested objects, smaller numbers result in more verbose reports.
(choose 2, 10 or 100 for useful output.)
Allocation report: shows how different sized objects perform on
your Proxy.
Input Format
- -f type
- format: sets the type of input logfiles. If set to
'auto' Calamaris tries to guess the input file
format. This is the Default.
Note: If the first line of your input file is corrupted,
Calamaris will stop with an error.
'squid' Calamaris expects native logfile derived
from Squid V1.1.beta26-V2.x or OOPS.
'squid-extended' Calamaris expects native logfile
derived from Squid V1.1.alpha1-V2.x with
log_mime_hdrs enabled or Squid with
Smartfilter-Patch. or squid-style logfiles out of Cisco
Content Engines. (This only enables parsing of these kind of
logfile, the additional data will be ignored.)
(Logging of MIME-Headers breaks the privacy of your users! (see
PRIVACY-Section below)
'squid-old' Calamaris expects native logfile
derived from Squid V1.1.alpha1-V1.1.beta25.
'nc' Calamaris expects Logfiles from NetCache
up to V4.x. (Please see the README on this.)
'its' Calamaris expects Logfiles from Inktomi
Traffic Server.
'elff' Calamaris expects Logfiles in Extended
Logfile Format (i.e. from Compaq Tasksmart, Novell
Internet Caching System or NetCache V5.x )
'nse' Calamaris expects Logfiles in Netscape
Extended-1 or Netscape Extended-2 Logfile Format (from
Netscape/ipoint Proxy-Server )
Output Format
Standard output format is plain ascii with 80
chars width.
- -F type[,type[,type[,...]]]
- Format: sets the format of the output-report. If set to
'mail' adds a subject header to the beginning of the
report.
'html' all output is given in html with tables. Can be
combined with 'mail' to send html mails.
'html-embed' all output is given in html with tables
without HTML-Headers. Useful for Server-Side-Includes.
'unformatted' gives out the raw numbers seperated by
spaces. Useful for re-using the output in other scripts. If you use
this along with -U, the byte values are calculated in the
given Unit, and displayed without indication along with the
numbers. the indication moves up to the header of the report.
- -M string
- Meta: adds a custom string into the <HEAD> of a
HTML-Report. Useful if you want to add Stylesheets or something to
the Report.
Note: -M works only in combination with -F
html.
- -l string
- logo: add a custom string to a HTML-Report. It'll be added to a
table on the top of the output. -l '<A HREF=http://cord.de/><IMG BORDER=0
SRC=http://cord.de/Images/cord/cordlog2n.gif
ALT=Cord></A>' will add my logo with a link to the
Report.
Note: -l works only in combination with -F
html.
- -U string
- Unit: You can define this to K(ilo), M(ega), G(iga) or T(era)
for the Byte-units.
- -S n[,n[,n[,...]]]
- Show: Shows only the defined reports in the specified order.
Default is to display the reports as they are defined through the
report-switches above. The following numbers are defined:
- 0 Summary
- 1 Incoming request peak per protocol
- 2 Incoming transfer volume per protocol
- 3 Incoming requests by method
- 4 Incoming UDP-requests by status
- 5 Incoming TCP-requests by status
- 6 Outgoing requests by status
- 7 Outgoing requests by destination
- 8 Request-destinations by 2ndlevel-domain
- 9 Request-destinations by toplevel-domain
- 10 TCP-Request-protocol
- 11 Requested content-type
- 12 Requested extensions
- 13 Incoming UDP-requests by host
- 14 Incoming TCP-requests by host
- 15 Distribution Histogram
- 16 Performance in minute steps
Note: Using this doesn't make Calamaris any
faster, the internal calculations will be done as the
report-switches were set (see above).
Caching
- -i file
- input file: You can reuse a cache file generated with -o
file to add old data to a new report. Several files can be
seperated with a ':'.
Note: if you use more than one cache file, make sure they
are chronologicaly ordered (oldest first).
Note: if you reuse cache-files, which were not created
with -d -1 -r -1 -t -1 -R -1 the number of 'others' is
likely wrong everywhere.
- -o file
- output file: Calamaris stores a summary of the computed
information in file and you can reuse it at a later time
with -i.
Note: The output file can be the same as the input file:
it is simply overwritten after reading the data. It is not
recommended to change the options between different runs if you
include older data as this can result in strange
measurements.
Misc
- -H name
- Host name: The name for the title or subject of the output. If
- Host name: The name for the title or subject of the output. If
set to 'lookup' Calamaris looks up the host name of
the system its been run on.
- -n
- no lookup: Switches the IP lookup for the requesters off.
- -O
- Order: Changes the sort order in the reports to request size,
default is sorting by number of requests.
- -I t-t
- Interval: defines which time-interval should be parsed. t has
to be the format yyyymmddhhmmss
Note: omitting the beginning or ending date is allowed.
- -N
- N-level: All URL-Host reports will be switched from 2nd-level
to N-level-reports. (-1 shows a full urlhost-report)
Note: This option is only useful with activated
domain-report.
- -T n
- Time: Adjust the Performancereport in minutes for non
GMT-Timezoners.
- -c
- case: Switch to case-insensitive. This is useful for the
'Requested extensions' report.
- -u
- user: Switches the usage of eventually available ident
information for requester reports on.
Warning: This breaks the privacy of your users! (see
PRIVACY-Section below)
- -z
- zero: Switches reading from standard input off. You can use
this to merge many cache files to one (see -i and -o)
or to generate a report out of cache files.
- -b n
- benchmark: A switch for the impatient as it prints a '#' for
every n parsed lines.
- -v
- verbose: print more information about what is Calamaris is
doing and believing.
- -L
- Loop: prints the internal loop to STDERR. (for Debugging)
- -C
- Copyright: Prints the copyright information of
Calamaris.
- -h
- help: Prints a brief description of the command line options.
- -V
- Version: Prints out the Version-Number.
EXAMPLES
- This example mails the daily statistics to root:
- cat /var/log/squid/access.log | nice -39 calamaris -aH
'daily worf' -F mail | mail root
- This one only caches a summary for later use:
- cat /var/log/squid/access.log | calamaris -a -o
daily.`date +"%w"` > /dev/null
- You can then use the caches to have weekly statistics:
-
if [ $DAYOFWEEK = "0" ]; then calamaris -a -i
daily.1:daily.2:daily.3:daily.4:daily.5:daily.6:daily.0 -z -F mail
-H "weekly worf" | mail root ; fi
BUGS
If you have a problem with Calamaris, please
make sure that you use the recent version of Calamaris (see
VERSION below). Also check that your proxy works correctly and
doesn't produce invalid Logfiles. (see the README for buglist and
pointers.)
If you're sure that you've encountered a bug in Calamaris,
please report it to Calamaris-bug@cord.de. This also
applies if Calamaris itself says 'please report this'.
PRIVACY
Calamaris can be (mis-)used to track what users are
requesting.
So please read the following and think about it, before using
Calamaris to be the Big Brother.
- -
- If you don't trust your users than there is something more
wrong than the loss of productivity.
- -
- Squid has some nice acl-mechanisms. If you think that your
users don't use the net properly, don't let them use it. (You can
also open the net at specific times or to specific sites, if you
want.)
- -
- If you still want to use Calamaris that way, let your
vict^Wusers know, that they'll be monitored. (in Germany you have
to let them know!)
SEE ALSO
AUTHOR
Cord Beermann <Cord@Wunder-Nett.org>. There
are also a lot of people who contributed code, gave ideas or
requested features. Look them up in the executable.
This man page was written by Philipp Frauenfelder <pfrauenf@debian.org>,
maintainer of the Debian package. Maintenance is now taken over by
Cord Beermann.
VERSION
Version of this manpage: $Id: calamaris.1,v 1.28 2003/05/14
21:22:37 cord Exp $
It describes the usage of Calamaris V2.58 and later.
Information about new releases, mailing lists, and other related
issues can be found from the Calamaris home page at http://Calamaris.Cord.de/
WARRANTY
Calamaris comes with "absolutely no
warranty".
COPYRIGHT
Copyright (C) 1997, 1998, 1999, 2000, 2001, 2002,
2003 Cord Beermann
This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License as
published by the Free Software Foundation; either version 2 of the
License, or (at your option) any later version.
(If you modify and want to publish it under the name
Calamaris, please ask me. I don't want to confuse the
'audience' with many different versions of the same name and/or
Version number. (This is not part of the license, it is only a
favour i asked of you.))
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
General Public License for more details.
You should have received a copy of the GNU General Public
License along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA
02111-1307, USA.