Apologies for the lag between posts, dear reader. I’m in the midst of a doctoral dissertation, almost finished my second chapter, and it doesn’t leave a lot of room for additional writing. Treat this entry as a stop gap, courtesy of Raido, from SpectX, the subject of our last toolsmith #143 on SpectX4DFIR. Herein, Raido provides us with a SpectX query to count hits from IPs during different time intervals.
This can be one way of detecting possible bots and automated queries. Running the query below will tell you:
- On how many different days do we have hits from a particular IP (column ‘days’)?
- On how many different days did we see this IP every hour, 24 hours in a row (column ‘full_days’)?
- During how many different hours did we get hits from this IP? (column ‘hours’)?
I ran the query below, slightly modified from Raido’s original, against the April 2020 log file for holisticinfosec.io. You can run this on any log file that contains timestamps and IP-addresses, just change the path, pattern and field names accordingly.
LIST('file:/C:/logs/holisticinfosec.io-ssl_log-Apr-2020')
| parse(pattern:$[/user/patterns/apache/apacheLog.sxp])
| select(hour:timestamp[1 hour], clientIp)
| group(hour, clientIp)
| select(day:hour[24 hour], clientIp, hours:count(*))
| group(day, clientIp)
| select(clientIp, days:count(*), full_days:count(hours = 24), hours:sum(hours))
| group(clientIp)
| sort(full_days desc)
The results as seen in Figure 1 provided immediate insights.
Figure 1: SpectX IP hitcount query result
As promised, these IPs as noted in the results per Figure 1 are all making constant calls to my site, all day, every day. Each are calling my index.xml file, some appear to be RSS readers or scrapers, which is fairly routine. Seems like a lot of needless connect and compute cycles for a low traffic, static site such as mine.
Some of these IPs are definitely of ill repute however.
173.212.239.212, originating out of Nuremberg, Bavaria scored a near perfect 99 of 100 for fraudulent behavior and malicious activity based on recent actions according to IPQS.
Figure 2 bears this out.
This is useful little query to quickly detect possible bots and automated queries. Hopefully you’ve already downloaded SpectX and given a try after a last post. Load it back up and feed a log. If you want a copy of the log as utilized for this post, let me know via socials or email.
Cheers…until next time.