By Liisa Tallinn and Raido Karro
This article is a step-by-step guide on pointing SpectX at your IIS logs, applying one of the built-in IIS parsers to your data and customizing the schema if needed. Once the parser matches the raw data, SpectX makes it quick and easy to run queries on a large number of log files from their current location. This is especially handy if the volumes are large and the time is limited.
IIS logs seem fairly easy to parse - the structure is well standardized and available in the header of the files. At the same time, there is no pattern to rule them all because logging use cases are unique, storage is limited and not all IIS servers have every field and option turned on in their configuration. The situation gets even trickier when looking at multiple servers with multiple configurations. The solution we've drawn up at SpectX is a full IIS log pattern (or schema) to cover all fields potentially available at IIS versions 8.5 and up. Fields not present in the data can be commented out in the pattern.
To get started, download and install SpectX. The Desktop edition is free of charge with unlimited data volumes. If you happen to get stuck anywhere, feel free to join our Slack community to ask for advice.
#drive#:\inetpub\logs\LogFilesThe fastest option for reading, parsing and analyzing these files with SpectX is to create an AD security group for users running SpectX in their machines and give them access to the IIS log folder. This will let the user map the log folder to the machine running SpectX and running complex queries without first importing or copying the data from the log folder. If the IIS server runs in Azure and the logs sit at an Azure blob, SpectX can read the files directly from that blob. The third option is centralized logging. SpectX can read files from the central file storage via ssh or much faster if adding the SpectX Source Agent to the server.
If the IIS log folder is mapped to the machine running SpectX, you can simply open the Input Data browser and navigate to one of the IIS log files. If the logs are stored in the cloud, create a New>Datastore and configure SpectX access to the storage. Having clicked on a log file in the data browser, follow these steps:
1. Click 'Prepare pattern'. SpectX will automatically apply one of the three IIS patterns included in the Desktop edition to your data. When the pattern matches the data, fields in the parse preview light up in yellow and blue:
2. If you're not happy with the pattern, click on the shared > pattern folder in the resource tree and drag-drop another IIS pattern to the pattern editing window.
file:/C:/inetpub/logs/u_ex181029.logcan be expanded with a wildcard to look at the whole month
file:/C:/inetpub/logs/u_ex1810*.logHere's how to look at the full year
@src = PARSE(pattern:$pattern, src: [‘file:/C:/inetpub/logs/u_ex18*.log', 'file:/D:/inetpub/logs/u_ex1810*.log', 'file:/E:/inetpub/logs/u_ex181029.log']);