splunk> Think Crazy Big Data

Yes, Splunk can eat all your data so you can search, report, and alert on it. You can then make reports that answer traditional, really useful things, like "How many web users were there per country this month?". But we want you to think bigger.

What should you know about your business or customers that you don't? Think crazy big. You have an amazing amount of data in your company, all over the place. If its got a plug and isn't making coffee, there's a good chance its spewing out data. And that's just the devices physically around you. Think bigger -- data in the cloud, on SaaS systems, hypervisors, and global data centers -- generated by every service you can think off -- servers, network devices, mobile devices, applications, call logs, RSS feeds, social media comments, weather data, GPS, etc.

Now, go ask the wacky questions using data from everything in your arsenal:

• Do purple sweaters make people leave your site? Looking at web access logs, is there a correlation between product colors and whether a purchase was made? Can you predict the best items to show a visitor based on his web browser information and location?
• When is the best time to have a meeting on Tuesdays? Use your cardkey logs from your office. Which employees work on the weekends? Have former employees tried to get in the office?
• Do your customers like the Family Guy? Use a free twitter data feed and see how your customers feel about your new advertisement by location. What are their favorite football teams and TV show so we can target them better?
• Should your boss buy more hardware? Use performance metrics logs and correlate slowness with lost sales.
• What's the least delayed flight I should take between Seattle to Chicago? Use free FAA data and do a search.
• Which departments are screwing around? Use wifi data to see which physical parts of your office are using Facebook the most.
• Did anyone sneak into the office? Use HID, VPN, security event logs, and see who logged into a computer at the office but didn't swipe their security badge to get in the building.

This isn't just talk. NPR initially used Splunk to monitor and troubleshoot their content deliver to customers, but soon expanded it to include critical business metrics, like program popularity, views by device, reconcile royalty payments for digital rights, and measure abandonment rates. To take another example, Expedia initially used Splunk to avoid website outages, but soon expanded it to include monitoring 98% of their infrastructure, including online bookings, performance of air-travel coupons and optimizing SEM.

Another way to thinks about it is by thinking of your goals, and some possible uses:

• Goal: Sleep Easy
  • Keep your machines up 24x7 by proactive monitoring and alerting on problems.
  • Prevent cyber-security risks and fraudulent activity.
  • Present reports that even business can understand.
  • Verify that you organization is meeting compliance requirements.
  • Track capacity and digital assets.
  • Resolve complex problems in less time.
• Goal: Grow your business
  • Understand your customers better with real-time data feeds
  • Identify trends and analyze sentiment by monitoring social media networks.
  • Understand customer location and behavior by analyze mobile data.
  • Find trends and anomalies by combining live and historical data feeds.

Still having trouble thinking about what data I should give Splunk? Fair enough. Here are some common categories to think about: Application logs; Call Detail Records; Clickstream Data; Configuration Files; Database Audit Logs and Tables; Filesystem Audit Logs; Management and Logging APIs; Message Queues; OS Metrics, Status, and Diagnostic commands; Packet Data; Security Infrastructure; Syslog; WMI; Web Access Logs; Web Proxy Logs

Now get out there, and think big, crazy big.