By Liisa Tallinn and Raido Karro
This article is a step-by-step guide on pointing SpectX at your IIS logs, applying one of the built-in IIS parsers to your data and customizing the schema if needed. Once the parser matches the raw data, SpectX makes it quick and easy to run queries on a large number of log files from their current location. This is especially handy if the volumes are large and the time is limited.
IIS logs seem fairly easy to parse - the structure is well standardized and available in the header of the files. At the same time, there is no pattern to rule them all because logging use cases are unique, storage is limited and not all IIS servers have every field and option turned on in their configuration. The situation gets even trickier when looking at multiple servers with multiple configurations. The solution we've drawn up at SpectX is a full IIS log pattern (or schema) to cover all fields potentially available at IIS versions 8.5 and up. Fields not present in the data can be commented out in the pattern.
To get started, download and install SpectX. The Desktop edition is free of charge with unlimited data volumes. If you happen to get stuck anywhere, feel free to join our Slack community to ask for advice.
#drive#:\inetpub\logs\LogFilesThe fastest option for reading, parsing and analyzing these files with SpectX is to create an AD security group for users running SpectX in their machines and give them access to the IIS log folder. This will let the user map the log folder to the machine running SpectX and running complex queries without first importing or copying the data from the log folder. If the IIS server runs in Azure and the logs sit at an Azure blob, SpectX can read the files directly from that blob. The third option is centralized logging. SpectX can read files from the central file storage via ssh or much faster if adding the SpectX Source Agent to the server.
If the IIS log folder is mapped to the machine running SpectX, you can simply open the Input Data browser and navigate to one of the IIS log files. If the logs are stored in the cloud, create a New>Datastore and configure SpectX access to the storage. Having clicked on a log file in the data browser, follow these steps:
1. Click 'Prepare pattern'. SpectX will automatically apply one of the three IIS patterns included in the Desktop edition to your data. When the pattern matches the data, fields in the parse preview light up in yellow and blue:
2. If you're not happy with the pattern, click on the shared > pattern folder in the resource tree and drag-drop another IIS pattern to the pattern editing window.
file:/C:/inetpub/logs/u_ex181029.logcan be expanded with a wildcard to look at the whole month
file:/C:/inetpub/logs/u_ex1810*.logHere's how to look at the full year
file:/C:/inetpub/logs/u_ex18*.log
@src = PARSE(pattern:$pattern, src: [‘file:/C:/inetpub/logs/u_ex18*.log', 'file:/D:/inetpub/logs/u_ex1810*.log', 'file:/E:/inetpub/logs/u_ex181029.log']);
Introducing SpectX Business
SpectX log parser and query engine for instant log investigation and query automation is now available for smaller teams and organizations. Starting at 79€/month, the Business license introduces several opt-in features for analyzing raw log files stored in file servers, S3, Azure, Google Storage, Elastic clusters and databases.
Collecting and Analyzing Google Workspace Audit Logs
What's the most accessed publicly shared document in your Drive? How many documents per user were downloaded during the past month? Where are all those calendar (spam) invites coming from? If your business relies on Gmail, Docs, Drive, Calendar, Meet and other products from Google, the answers to these questions lie in Google Workspace (formerly G Suite) audit logs.
What to Search from your Office 365 Audit Logs?
Here’s a guide on quickly accessing and running advanced queries on your Office365 audit logs with your desktop computer and the free edition of SpectX. Whether you need to know more about logins, accessing, editing, deleting files - O365 logs are the data to go to. The query examples we've prepared will give you a broad overview of user activities and zoom into the suspicious. If you're in the incident response phase, you can use the queries to learn the details.
Analyze Cloudfront Logs in S3
SpectX gets you instant insights into your web resources served by Cloudfront. Download the free desktop edition to your computer to parse and query the S3 bucket storing your log files. Follow the steps below and run the query examples to get a general overview or specific questions answered in minutes. The example queries are also included in the SpectX Github query pack.
