Threat intelligence is a much-bandied phrase these days, with security analysts saying CISOs need it to keep ahead of attackers. So it’s interesting to read that researchers at Arizona State University have created a system that gathers data from hacker marketplaces and forums to identify emerging cyber threats. It could add another useful weapon in the fight against threat actors.

Meanwhile a chief strategy officer at a security vendor has reminded CISOs that threat intelligence also includes looking in their own organizations for holes they are leaving open for exploitation.

First the research paper: The authors say their system — which involves a crawler, data mining and machine learning — collects on average 305 high-quality cyber threat warnings each week, including information on newly-developed malware and exploits that have not yet been deployed.  “With the use of machine learning models, we are able to recall 92 per cent of products in marketplaces and 80 per cent of discussions on forums relating to malicious hacking with high precision,” say the authors.

The system has three components: Crawlers that hunt for and retrieves HTML documents and forum discussions from darknet; a parser for each crawler that extracts specific information from marketplaces on the sale of malware/exploits and hacker forums discussing services and threats. This structured information is stored in two relational databases. The parser also communicates a list of relevant Web pages to the crawler, which are re-crawled to get time-varying data; and a classifier that uses machine learning techniques to detect relevant products.

In tests the researchers found 16 new zero-day exploits over a 4 week period, which could help CISOs decide what systems of their own to patch or replace. Researchers were also able to construct a social network of likely hackers from the data gathered on people participating in multiple malicious hacker forums or marketplaces.

The project is considered successful enough that the researchers hope to commercialize it. IT will be interesting to see who picks it up, how much the system adds to threat knowledge and how much customers will have to pay for it.

Advance warning of what’s coming is helpful to CISOs, but Adam Meyer, chief security strategist at SurfWatch Labs. reminds infosec pros in a column this week that one way to use threat intelligence is to evaluate the risks in their environments.

A threat actor could be preparing an exploit of an application the organization has, but you may not have to worry about it because you’re prepared. He suggests infosec pros think of whether an actor has the capability, opportunity and the intent to cause harm. So asking questions like does the actor have the capability to cause an event, have they been known to do it in the past, are they active in communications forums and do they have the opportunity — meaning are there vulnerabilities in your defences.

“Look at breach history in various sectors and look at your own internal incident information,” he writes. “How much was due to organizations opening the door and giving the adversary the opportunity? How much of it was due poor maintenance, poor oversight, and/or poor cyber hygiene?” Threat intelligence, he says, helps answer those questions.