Access log with logback, Jetty and Tomcat

Authors: Ceki Gülcü, Sébastien Pennec
Creative Commons License

Copyright © 2000-2006, QOS.ch

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 License .

Introduction

Logback was designed as a modular framework from the start. Being able to use logback's internal core in many situations, without heavy coding or complex specific configuration was one of our goals.

Lobgack access integrates with Servlet containers such as Jetty and Tomcat to provide HTTP-access log functionality.

Logback Access and Tomcat

To use logback-access with Tomcat, after downlading the logback distribution, place the files logback-core-VERSION.jar and logback-access-VERSION.jar under $TOMCAT_HOME/server/lib directory, where $TOMCAT_HOME is the folder where you have installed Tomcat. We have tested logback-access module with Tomcat version 5.5.20.

LogbackValve

The ch.qos.logback.access.tomcat.LogbackValve class extends Tomcat's ValveBase class. This class is used to allow external components to be integrated into Tomcat.

To configure Tomcat in order to use LogbackValve, add the following lines to the tomcat server configuration file, namely $TOMCAT_HOME/conf/server.xml:

<Valve className="ch.qos.logback.access.tomcat.LogbackValve"/>

This line need to be nested in an Engine element.

By default, LogbackValve looks for a logback configuration file called logback-access.xml, in the same folder where server.xml is located, that is in $TOMCAT_HOME/conf/. This configuration file contains directives for configuring logback components. Among others, you can specify the appenders where the logging requests will be sent, and their format. Please refer to the description below about logback access configuration for examples.

Logback Access and Jetty

To use logback-access with Jetty, after downlading the logback distribution, place the files logback-core-VERSION.jar and logback-access-VERSION.jar under $JETTY_HOME/lib directory, where $JETTY_HOME is the folder where you have installed Jetty. We have tested logback-access module with Jetty version 6.0.1.

Logback's RequestLog implementation

The ch.qos.logback.access.jetty.RequestLogImpl class implements jetty's RequestLog interface. This interface is used by Jetty to allow external components to manage request logging.

In logback, logging destinations are called Appenders. These classes can be attached directly to RequestLogImpl.

To configure jetty in order to use logback's RequestLogImpl, add the following lines to the jetty configuration file, namely $JETTY_HOME/etc/jetty.xml:

<Ref id="requestLog">
  <Set name="requestLog">
    <New id="requestLogImpl"
      class="ch.qos.logback.access.jetty.RequestLogImpl">
    </New>
  </Set>
</Ref>

These lines reference the requestLog functionnality of Jetty, setting the actual class that will be called at each logging request.

By default, RequestLogImpl looks for a logback configuration file called logback-access.xml, in the same folder where jetty.xml is located. This configuration file contains directives for configuring logback components. Among others, you can specify the appenders where the logging requests will be sent, and their format.

As long the path is specified, you can place the logback configuration file in any location. Here is another example of jetty configuration file, with a path to the logback access configuration file.

<Ref id="requestLog">
  <Set name="requestLog">
    <New id="requestLogImpl"
      class="ch.qos.logback.access.jetty.RequestLogImpl">
    </New>
    <Set name="fileName">path/to/myaccess.xml</Set>
  </Set>
</Ref>

Logback Access Configuration

Altough similar, the logback-access.xml file is slightly different than the usual logback classic configuration file. Appenders and Layouts are declared the exact same way. However, in the access module there is no notion of loggers and consequently loggers elements are disallowed in configuraiton files for logback-access.

Example 1: basic logback-access configuration

Here is a sample logback-access.xml file that you can immediately put to use:

<configuration>
  <appender name="STDOUT"
    class="ch.qos.logback.core.ConsoleAppender">
    <layout
      class="ch.qos.logback.access.PatternLayout">
      <Pattern>%h %l %u %user %date "%r" %s %b</Pattern>
    </layout>
  </appender>

  <appender-ref ref="STDOUT" />
</configuration>

It declares a ConsoleAppender which directs its output at the console. The ConsoleAppender contains a PatternLayout format the output. The log format is specied using the %h %l %u %user %date "%r" %s %b" pattern which is the Commong Log Format (CLF). This format is recognized by log analysers such as Analog or AWStats.

Instead of specifying the complete pattern, the word "common" or "clf" can be used as a shorthand. Thus, the following are all equivalent

<Pattern>%h %l %u %user %date "%r" %s %b</Pattern>
<Pattern>common</Pattern>
<Pattern>clf</Pattern>

The so called "combined" format is also widely recognized. It is defined as the '%h %l %u %t "%r" %s %b "%i{Referer}" "%i{User-Agent}"' pattern. As a facilitator, you can use the "combined" as a shorthand. Thus, the following directive

<layout class="ch.qos.logback.access.PatternLayout">
  <Pattern>%h %l %u %t "%r" %s %b "%i{Referer}" "%i{User-Agent}"</Pattern>
</layout>

is equivalent to:

<layout class="ch.qos.logback.access.PatternLayout">
  <Pattern>combined</Pattern>
</layout>

Example 2: RollingFileAppender

Another configuration file, using logback' RollingFileAppender, could be:

<configuration>
  <appender name="FILE"
    class="ch.qos.logback.core.rolling.RollingFileAppender">
    <File>access.log"</File>
    <rollingPolicy
      class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
      <FileNamePattern>access.%d{yyyy-MM-dd}.log.zip</FileNamePattern>
    </rollingPolicy>

    <layout class="ch.qos.logback.access.PatternLayout">
      <Pattern">combined</Pattern">
    </layout>
  </appender>
 
  <appender-ref ref="FILE" />
</configuration>

Here, there is no output to the console. Instead, logback access logs to the file named access.log. This file will be rolled over every 24 hours. We specify in the configuration the name of the file where the actual logging is added, and the pattern that the archived files must match. The newly archived file will be automatically compressed.

These two configuration examples should give you an idea of the possibilities offered by the logback-access module. In principle, most of the things that you can do with logback-classic module are equally possible with logback-access.

PatternLayout

An http-specific implementation of PatternLayout is included in the access module. The ch.qos.logback.access.PatternLayout provides a way to format the logging output that is just as easy and flexible as the PatternLayout found in logback classic.

For detailled instructions on how to use the PatternLayout for logback access, please refer to the corresponding chapter of the logback manual.

JMX Components

Logback access easily integrates with JMX servers to publish useful information about some of its components.

Both RequestLogImpl and LogbackValve can be published and modified via JMX. A special filter, that we will cover further down this document, publishes statistical views of access logs.

Configuring Tomcat

Accessing JMX components with Tomcat requires to add the following lines to the $TOMCAT_HOME/bin/catalina.sh configuration file:

CATALINA_OPTS="-Dcom.sun.management.jmxremote"
CATALINA_OPTS="$CATALINA_OPTS -Dcom.sun.management.jmxremote.ssl=false"
CATALINA_OPTS="$CATALINA_OPTS -Dcom.sun.management.jmxremote.authenticate=false"

Once started with these options, Tomcat's JMX compoenents can be accessed with JConsole by issuing the following command in a shell:

jconsole &

The user might prefer to access her components via a web-based solution using MX4J. In that case, here are the required steps:

First, download MX4J. Place the mx4j-impl.jar file in the $TOMCAT_HOME/bin/ directory, and the mx4j-tools.jar in the $TOMCAT_HOME/common/lib/ directory.

Then, add the following lines to the $TOMCAT_HOME/bin/catalina.sh configuration file:

<!-- at the beginning of the file -->
CATALINA_OPTS="-Dcom.sun.management.jmxremote"
CATALINA_OPTS="$CATALINA_OPTS -Djavax.management.builder.initial=mx4j.server.MX4JMBeanServerBuilder"

<!-- in the "Add on extra jar files to CLASSPATH" section -->
CLASSPATH="$CLASSPATH":"$CATALINA_HOME"/bin/mx4j-impl.jar

Finally, declare a new Connector in the $TOMCAT_HOME/conf/server.xml file:

<Connector port="8050" 
  handler.list="mx"
  mx.enabled="true" 
  mx.httpHost="localhost" 
  mx.httpPort="8082" 
  protocol="AJP/1.3" />

Once Tomcat is started, you should be ableo to reach the JMX components by pointing a browser to the following URL:

http://host_name:8082/

Configuring Jetty

Configuring Jetty to publish JMX components requires a few modifications to the $JETTY_HOME/etc/jetty.xml configuration file. Here are the elements that need to be added:

<Call id="MBeanServer" class="java.lang.management.ManagementFactory" name="getPlatformMBeanServer"/>
<!-- initialize the Jetty MBean container -->
<Get id="Container" name="container">
  <Call name="addEventListener">
    <Arg>
      <New class="org.mortbay.management.MBeanContainer">
        <Arg><Ref id="MBeanServer"/></Arg>
        <Set name="managementPort">8082</Set>
        <Call name="start" />
      </New>
    </Arg>
  </Call>
</Get>

Once Jetty is started with this configuration, all available components can be reviewed at this address:

http://host_name:8082/

Logback's RequestLogImpl is available, and its start() and stop() method can be called.

CountingFilter

Logback can provide a statistical view of the accesses to the server. This is done by using the CountingFilter class.

To use the CountingFilter, one just needs to declare it, like any other filter:

<configuration>
  <filter class="ch.qos.logback.access.filter.CountingFilter">
    <name>countingFilter</name>
  </filter>

  <appender name="STDOUT"
    class="ch.qos.logback.core.ConsoleAppender">
    <layout class="ch.qos.logback.access.PatternLayout">
      <Pattern>%h %l %u %t \"%r\" %s %b</Pattern>
    </layout>
  </appender>

  <appender-ref ref="STDOUT" />
</configuration>

This component registers itself to the JMX server and publishes the following statistical figures: