This security vendor’s goal was to provide real-time threat analytics by ingesting threat intelligence data streams and correlating them with its own indexed metadata. The current product architecture collected data in a Postgres database, which did not lend itself well to analyzing streaming machine data. The team looked at a variety of solutions but cost or lack of in-house development resources were deterrents. X15 Enterprise was implemented and provided the following benefits:
- Enriched analytics with real-time security intelligence
- Provided real-time search capability
- Provided historical reporting capability
- Scaled security analytics for Fortune 500 customers
- Accelerated time to market
“Breach detection requires large volumes of intelligence and powerful analytics to pull insight out of the data. With X15 as our integration partner, our product has a robust architecture that is capable of real-time data ingestion from various external data feeds, authoritative indexing of network data, and powerful querying and reporting capabilities.”
VP Products, Leading Security Vendor
This leading security vendor indexes a network to identify and map every IP connected device, as well as to uncover network segmentation violations and cybersecurity anomalies. Its first-generation product focused on point-in-time analytics and collected IP scan data in a Postgres database. In order to provide real-time network visibility, the product needed to ingest, search and correlate a variety of streaming data including Netflow, DNS, and DHCP.
Schema-based architecture could not ingest semi-structured machine data
The Postgres database architecture did not lend itself well to analyzing streaming machine data. The sheer volume of the data immediately created scalability issues. More importantly, the unknown structure of the streaming data made it impossible to retrofit it within an architecture that relies on a rigid schema.
Alternatives such as Splunk and ELK were not a good fit
The team looked at a variety of options but none of them were a good fit. Splunk was evaluated but the cost was a deterrent, in particular because the solution would need to scale across the vendor’s entire customer base, which includes large global enterprises. The ELK stack was also considered but the upfront capital expense and lack of in-house development resources were deterrents. The development team also considered working with a partner who had the data management expertise to build a solution, but the time to market was too long.
The X15 Solution
X15 Enterprise seamlessly embeds within the security product
The security vendor ultimately selected X15 Enterprise. In the new architecture, X15 ingests threat intelligence data streams and correlates them with the security product’s indexed metadata. This provides real-time threat analytics such as botnet activity, nefarious TCP/UDP port usage by known malware exploits, and much more. The X15 user interface is seamlessly embedded within the product providing powerful visualization, search, and analysis functionality.
Enriched analytics with real-time security intelligence
The X15 platform discovers data structure upon ingestion and dynamically adjusts it as the data continues to stream, eliminating the need to specify rigid schemas upfront. This is critical because the structure of external streaming data can vary greatly. The security vendor’s product can now ingest NetFlow data and threat intelligence feeds (from open source or commercial providers) to correlate with network architecture and segmentation information, providing real-time breach detection analytics.
Provided real-time search capability
The X15 platform stores raw data along with parsed data. This approach enables the product’s users to rapidly search the data to find the needle in the haystack during investigations.
Provided historical reporting capability
The X15 platform optimizes redundant data by applying intelligent compression, which results in significant storage savings. This enables the security vendor to cost effectively retain large volumes of ingested data and provide a “scroll back in time” forensic view for up to one year.
Scaled security analytics for Fortune 500 customers
The X15 platform can be deployed as a single node or a distributed cluster where every node in the cluster indexes, stores, and queries data to enable fault tolerance and true linear scalability as nodes are added. The vendor can scale real-time security analytics to meet the needs of its largest customer deployments, which includes Fortune 500 organizations.
Accelerated time to market
The X15 platform is extensible by design and provides numerous integration options. The security product queries the platform via the Postgres interface to produce analytics. In addition, the X15 platform’s powerful querying and reporting capabilities are available to customers via the security product’s user interface. The seamless integration has dramatically accelerated the product’s time to market.