Spring Boot Logback Async Logging
This guide demonstrates configuring asynchronous logging with Logback in a Spring Boot application. This approach provides improved performance, reduced latency, scalability, and fault tolerance.
By leveraging existing Spring logging configuration properties, we can customize the logging behavior according to your application’s requirements seamlessly based on the environment variables declared in Spring Boot’s default Logback configuration.
1. Overview
Asynchronous logging is a powerful technique that offloads the logging operations to a separate thread, preventing the main application threads from blocking on I/O operations. This is particularly beneficial for high-throughput applications where logging can become a bottleneck.
The AsyncAppender in Logback wraps both console and file appenders to enable asynchronous logging, enhancing performance and fault tolerance. It uses a blocking queue to buffer log events and processes them asynchronously.
Contrary to the default Spring Boot configuration, file logging (ASYNC_FILE) is enabled in this setup even if |
2. Synchronous vs Asynchronous Logging
Understanding the fundamental difference between synchronous and asynchronous logging is crucial for making the right choice for your application.
2.1. Key Differences
| Aspect | Synchronous Logging | Asynchronous Logging |
|---|---|---|
Thread Behavior | Application thread blocks until log is written | Application thread returns immediately |
Performance Impact | Direct impact on request processing time | Minimal impact on request processing |
Latency | Higher latency due to I/O blocking | Lower latency, I/O handled separately |
Throughput | Limited by I/O operations | Higher throughput, decoupled from I/O |
Complexity | Simple, straightforward | More complex, requires queue management |
Use Case | Low-volume logging, simple applications | High-volume logging, performance-critical apps |
3. Benefits of Async Logging
3.1. Improved Performance
Asynchronous logging reduces the time the application thread spends on logging operations by delegating them to a separate thread.
3.2. Reduced Latency
Main application threads are not blocked waiting for log operations to complete, resulting in lower response times.
3.3. Scalability
Better resource utilization allows the application to handle more concurrent requests.
3.4. Fault Tolerance
If the logging system experiences temporary issues, the application continues to function normally while log events are queued.
4. AsyncAppender Configuration Parameters
Logback’s AsyncAppender provides several configuration parameters to fine-tune its behavior:
4.1. queueSize
Default Value:
256Description: Maximum capacity of the blocking queue that buffers log events.
Usage: Increase for high-volume logging scenarios, but be mindful of memory consumption.
4.2. discardingThreshold
Default Value:
20%of queue sizeDescription: By default, drops events of level TRACE, DEBUG, and INFO when the queue has 20% capacity remaining. Set to
0to keep all events.Usage: Prevents queue overflow by discarding low-priority events when the queue is nearly full.
4.3. includeCallerData
Default Value:
falseDescription: Controls whether to extract caller data (class name, method name, line number).
Usage: Set to
trueonly if you need caller information, as extracting this data is expensive and impacts performance.
4.4. maxFlushTime
Default Value: Not set
Description: Maximum time in milliseconds the AsyncAppender waits for the queue to flush during application shutdown.
Usage: Ensures log events are not lost during graceful shutdown.
4.5. neverBlock
Default Value:
falseDescription: When
false, the appender blocks on a full queue. Whentrue, it drops the message instead.Usage: Set to
trueif you prefer dropping log events over blocking application threads.
5. Project Setup
5.1. Requirements
6. Configuration
6.1. Application Properties
Configure basic logging properties in application.yml:
1
2
3
4
5
logging:
file:
name: tutorial
pattern:
console: "%d{yyyy-MM-dd HH:mm:ss.SSSZZ} %magenta([%thread]) [%logger{36}] %highlight(%level) %cyan([%class{0}.%method:%line]) - %message%n%xException}"
This configuration:
Sets the log file name to
tutorialCustomizes the console log pattern with colors and detailed information including timestamp, thread, logger name, log level, class/method/line, and message
6.2. Logback Configuration
Create logback-spring.xml in src/main/resources:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
<?xml version="1.0" encoding="UTF-8"?>
<configuration debug="true" scan="true" >
<statusListener class="ch.qos.logback.core.status.OnConsoleStatusListener" />
<include resource="org/springframework/boot/logging/logback/defaults.xml" /> (1)
<property name="LOG_FILE" value="${LOG_FILE:-${LOG_PATH:-${LOG_TEMP:-${java.io.tmpdir:-/tmp}}}/spring.log}"/> (2)
<include resource="org/springframework/boot/logging/logback/console-appender.xml" /> (3)
<include resource="org/springframework/boot/logging/logback/file-appender.xml" /> (4)
<appender name="ASYNC_CONSOLE" class="ch.qos.logback.classic.AsyncAppender"> (5)
<appender-ref ref="CONSOLE" />
<queueSize>256</queueSize>
<includeCallerData>false</includeCallerData>
<neverBlock>false</neverBlock>
</appender>
<appender name="ASYNC_FILE" class="ch.qos.logback.classic.AsyncAppender"> (6)
<appender-ref ref="FILE"/>
</appender>
<root level="INFO"> (7)
<appender-ref ref="ASYNC_CONSOLE" />
<appender-ref ref="ASYNC_FILE" />
</root>
</configuration>
| 1 | Include Spring Boot’s default Logback configurations, providing access to predefined properties and converters. |
| 2 | Define the log file location with fallback values: LOG_FILE → LOG_PATH → LOG_TEMP → java.io.tmpdir → /tmp/spring.log. |
| 3 | Include Spring Boot’s default console appender configuration. |
| 4 | Include Spring Boot’s default file appender configuration. |
| 5 | Configure the async console appender wrapping the standard CONSOLE appender with all available parameters shown. |
| 6 | Configure the async file appender wrapping the standard FILE appender using default parameter values. |
| 7 | Set the root logger level to INFO and attach both async appenders. |
The
|
7. Understanding Spring Boot Logging Defaults
Spring Boot provides a well-structured default Logback configuration that you can leverage and extend. The key files include:
7.1. defaults.xml
Contains conversion rules, patterns, and property definitions used by Spring Boot. Available at:
7.2. base.xml
Provides the base configuration including console and file appenders. Available at:
7.3. DefaultLogbackConfiguration.java
The Java source that programmatically configures Logback when no custom configuration is provided. Available at:
8. Use Cases and Recommendations
8.1. Development Environment
For development, you might want to disable async logging or use only console logging for immediate feedback:
<root level="DEBUG">
<appender-ref ref="CONSOLE" />
</root>8.2. Production Environment
In production, enable both async console and file logging with appropriate queue sizes:
<appender name="ASYNC_CONSOLE" class="ch.qos.logback.classic.AsyncAppender">
<appender-ref ref="CONSOLE" />
<queueSize>512</queueSize>
<neverBlock>true</neverBlock>
</appender>
<appender name="ASYNC_FILE" class="ch.qos.logback.classic.AsyncAppender">
<appender-ref ref="FILE"/>
<queueSize>512</queueSize>
<includeCallerData>false</includeCallerData>
</appender>
<root level="INFO">
<appender-ref ref="ASYNC_CONSOLE" />
<appender-ref ref="ASYNC_FILE" />
</root>8.3. High-Throughput Applications
For applications with very high logging volume:
Increase
queueSizeto 1024 or higherSet
neverBlocktotrueto prevent application threads from blockingConsider setting
discardingThresholdto a higher value to drop low-priority logs earlierKeep
includeCallerDataasfalsefor maximum performance
9. Best Practices
9.1. Monitor Queue Size
Monitor the async appender queue to ensure it’s not frequently reaching capacity. If it does, consider:
Increasing the queue size
Reducing log volume by adjusting log levels
Optimizing the underlying appender performance
9.2. Graceful Shutdown
Ensure your application allows time for the async appender to flush remaining log events during shutdown. Configure maxFlushTime appropriately:
<appender name="ASYNC_FILE" class="ch.qos.logback.classic.AsyncAppender">
<appender-ref ref="FILE"/>
<maxFlushTime>5000</maxFlushTime>
</appender>9.3. Avoid includeCallerData in Production
Extracting caller data is expensive. Only enable it if absolutely necessary:
<includeCallerData>false</includeCallerData>9.4. Use Appropriate Log Levels
Set proper log levels for different packages to avoid overwhelming the async queue:
<logger name="com.myapp.verbose.package" level="WARN"/>
<logger name="org.springframework" level="INFO"/>
<logger name="org.hibernate" level="WARN"/>10. Troubleshooting
10.1. Logs Not Appearing
If logs are not appearing, check:
The
queueSizemight be too small and events are being discardedSet
debug="true"in the configuration to see Logback’s internal statusVerify the log file path is writable
10.2. Performance Issues
If async logging doesn’t improve performance:
Ensure
includeCallerDataisfalseIncrease
queueSizeif the queue is frequently fullConsider using
neverBlock="true"to prevent thread blockingCheck if the underlying appender (file I/O) is the bottleneck
10.3. Lost Log Events
If you’re losing log events:
Increase
queueSizeSet
discardingThreshold="0"to keep all eventsConfigure
maxFlushTimeto ensure events are flushed during shutdownConsider using
neverBlock="false"to block rather than drop events
11. Conclusion
Asynchronous logging with Logback in Spring Boot provides significant performance benefits for high-throughput applications. By properly configuring the AsyncAppender, you can:
Reduce application latency by offloading logging to separate threads
Improve throughput by preventing main threads from blocking on I/O
Maintain fault tolerance with configurable queue management
Customize behavior based on environment-specific requirements
The configuration demonstrated here leverages Spring Boot’s default Logback setup while adding async capabilities, making it easy to integrate into existing applications.
The complete source code is available on GitHub Gist.