ELK ( Elastic Search, Logstash, Kibana) for Windows and Linux logs for easy data mining

Recently I have been testing ELK (Elasticsearch, Logstash, Kibana) for the visualisation and data mining with Syslog, Windows Event Viewer, Apache, SQUID and IIS logs.

First of all, what is ELK? 

ELK is a set of aplications:

ElasticSearch --> Search server based on Lucene, distributed and full text search engine.
It is developed on JAVA and is open source, it runs over Linux and Windows, however it is better documented over Linux.
We use it to store all the information provided by the logs.

Logstash / Logstashforwarder --> It is a useful tool for harvesting and managing the logs. It is developed in jRUBY and requires a JVM; it works over Windows, Linux and OSX.
We use it to receive logs from remote machines and store them into ElasticSearch.

Kibana --> This tool is used to visualize and make the dashboards taking data from the Elasticsearch servers.

Additionally, for ELK to work on Windows, we need to setup the following software:

Nxlog --> It is required to forward data from Windows servers to Logstash server. NXlog is similar to logstash-forwarder.

ElasticHQ --> It is an Elastic Search plugin required to monitor Elastic Search infrastructure.

Now I´m going to link some pages because all of them have very good documentation. There's no need for rewriting it again! However, if any of the links is broken, report it and I will write about it.

READ ALL OF THEM CAREFULLY BEFORE YOU START DOING THE DEPLOYMENT.

https://www.digitalocean.com/community/tutorials/how-to-use-logstash-and-kibana-to-centralize-and-visualize-logs-on-ubuntu-14-04 <-- UBUNTU Server deployment
https://www.digitalocean.com/community/tutorials/how-to-use-logstash-and-kibana-to-centralize-logs-on-centos-6 <-- CENTOS Server deployment
http://everythingshouldbevirtual.com/highly-available-elk-elasticsearch-logstash-kibana-setup <-- Distrubuted deployment with Windows NXlog
http://www.ragingcomputer.com/2014/02/logstash-elasticsearch-kibana-for-windows-event-logs <-- Another distrubuted deployment with Windows NXlog
http://jacob.ludriks.com/iis-logging-to-the-elk-stack/  <-- Management of IIS LOGS

If you have a large amount of data, perhaps you will have to think about integration with Hadoop:

http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/hive.html
http://azure.microsoft.com/es-es/documentation/articles/hdinsight-get-started-emulator/
http://es.slideshare.net/hortonworks/hortonworks-elastic-searchfinal