Association Rule Mining in System Event Logs to Discover Patterns
This research paper provides an overview of the current state of logs analysis in IT systems. Initial part covers some fundamental theory and summarises basic goals and techniques about system logs. The current software systems have been drastically evolving which are increasing in scale and complexity of software systems, that leads to a flood of logs. The traditional manual log inspection and analysis became impractical and almost impossible. As logs are unstructured in nature, the first important step is to parse the text log messages into structured and meaningful data for further processing and analysis. Correlation of diverse data and uncovering patterns and relationships in the data is a backbone of Artificial intelligence for IT operations (AIOps) field.
In this research paper, we present a comprehensive evaluation study on log events and discovering best association rules in logs to better understand and get more insight of logs events. More specifically, we evaluate more than a hundred log events spanning across distributed IT systems, hosts, customised services and application servers. We report the pattern discovery results in terms of association rules which gives practical importance when investigating and troubleshoot system issues.