The logging module in Python provides a standardized way to record diagnostic and runtime information from your programs. The key benefits of logging are the ability to:

Log messages with different verbosity levels (debug, info, warning, error)
Flexibly configure how and where log messages are outputted
Format log message output
Filter log messages
Log exceptions and stack traces

In this article, we’ll cover the basic and advanced features of the logging module.

Logging Levels

The logging module provides the following severity levels to indicate the importance of log messages:

DEBUG - Debugging information, typically only enabled during development
INFO - General information about program operation
WARNING - Warning messages about possible issues
ERROR - Error messages when something has gone wrong
CRITICAL - Very severe error events

By default, the logging module captures all messages at WARNING and above.

Basic Logging Configuration

To start logging from Python:

import logging

logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO) 

logger.info("Started")

Import the logging module
Get a logger instance using getLogger()
Set the log level on the logger using setLevel()
Call log methods on the logger to generate log messages

This outputs a log message like:

INFO:__main__:Started

The log message contains the level (INFO), logger name (main), and message (Started).

Log Message Formatting

To configure how log messages are formatted:

formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')

handler = logging.StreamHandler()
handler.setFormatter(formatter)

logger.addHandler(handler)

This formats each message as: timestamp - logger_name - level - message

Logging to a File

To log messages to a file instead of the console:

handler = logging.FileHandler('app.log')
handler.setFormatter(formatter)

logger.addHandler(handler)

This will write log messages to app.log.

Advanced Configuration

Here are some common advanced configuration needs:

Configure separate loggers for different modules/components
Log to multiple destinations simultaneously
Set different log levels for different loggers
Custom logging handlers for syslog, HTTP, databases, etc
Handler filters to only capture certain log messages
Rotation of log files based on size

All this is achieved by working directly with the objects from the logging module - loggers, handlers, formatters, filters etc.

Best Practices

Some key best practices for working with logging in Python:

Configure logging as early as possible in app initialization
Use INFO for overall operational logging and DEBUG for diagnostics
Always log exceptions to get full tracebacks
Use meaningful logger names based on application/module
Avoid logging confidential information

This helps ensure logging is setup properly and provides the most useful diagnostic information.

Conclusion

Python’s logging module provides a full-featured and flexible logging system. It takes some initial configuration, but once setup allows comprehensive logging and debugging with control over how much information is captured and where it is sent. Following the standard patterns and best practices will ensure you get the most benefit.

Using Python logging