GET Monitoring for the poor or SAR + MySQL + Gnuplot / Sudo Null IT News FREE
Why exactly SAR?
I've been monitoring for quite some time. Therefore, aside the nature of my activenes, I often encounter unusual situations when I undergo to come up with various "bicycles" in arrange to monitor the host. For example, we will deliberate the state of affairs when we have a host (virtual machines surgery VDS), which is very limited in resources.
There are some good monitoring systems like Zabbix, Nagios, Cacti, etc. Only for all of our state of affairs, they are not suitable, for obvious reasons - they themselves consume resources, which we don't have and so many another. The question immediately arises, what to do? And here SAR hurries to help us.
Install and configure SAR
SAR (System Activity Report) is a very powerful inferior for collection statistical information about system performance. Included in sysstat package. Therefore, if you do not yet have the sysstat package on the server, install information technology. For example, for Debian:
# appropriate-bewilder install sysstat Then we need to configure the SAR itself. We include sysstat, for this we correct the charge:
# vim /etc/default option/sysstat And change the line ENABLED = "false" to ENABLED = "right". Then we need to fix the task in cron.
# vim /etc/cron.d/sysstat I collect information all minute, so on my organization it looks look-alike this (you power take in a slightly incompatible one):
# Natural process reports every 1 proceedings quotidian * * * * * root word command -v debian-sa1 > /dev/null && debian-sa1 1 1 # Additive run at 23:59 to revolve the statistics file 59 23 * * * root command -v debian-sa1 > /dev/null && debian-sa1 60 2 Restarting the sysstat inspection and repair:
# service sysstat re-start This completes the frame-up.
Exploitation SAR
I will not describe in detail how to employ SAR. To do this, at that place are many articles along the Net, and the SAR itself provides quite extensive documentation. All you need is a human beings sar, and there, I think, you'll project information technology unconscious.
Learn SAR to upload reports in MySQL
Yes, yes, you take IT right field. Of track, we can expend the SAR for its premeditated purpose, shout out it from the console and pick up reports in this form:
$ sar -s 07:00:00 -e 07:10:00 Linux 3.2.0-4-amd64 (take care-x) 02.03.2015 _x86_64_ (1 CPU) 07:00:01 CPU %user %nice %system %iowait %steal %idle 07:01:01 all 0,64 0,00 0,15 0,10 0,00 99,11 07:02:01 all 0,03 0,00 0,02 0,00 0,00 99,95 07:03:01 all 0,03 0,00 0,02 0,00 0,00 99,95 07:04:01 all 0,03 0,00 0,02 0,02 0,00 99,93 07:05:01 all 0,05 0,00 0,03 0,00 0,00 99,92 07:06:01 all 0,63 0,00 0,17 0,10 0,00 99,11 07:07:01 every last 0,03 0,00 0,02 0,00 0,00 99,95 07:08:01 completely 0,02 0,00 0,02 0,00 0,00 99,97 07:09:01 all 0,03 0,00 0,02 0,00 0,00 99,95 Среднее: all 0,17 0,00 0,05 0,02 0,00 99,76 But you essential admit, I assume't feel like looking these numbers subsequently a tall functioning twenty-four hour period, my eyes hurt, etc. What to do? How to work with much important information?
I admit, I spent the gross evening cerebration just about this job. All decisions seemed strange to me. Mayhap the solution I came functioning with is likewise strange, but why did I want to lay down friends with SAR MySQL? I will try to explain further.
If you carefully read the SAR tuning part, then you probably noticed that the SAR rotates its logs later on midnight. This is done specifically to split up magazines by day. I sought to receive data continuously (because we will fles graphs on them later) so that I can only select the column I need, for a certain time, etc. And and then I thought, wherefore non upload all these reports to the database? Since the server already had MySQL, the choice became obvious.
Thusly, friends of SAR and MySQL
I searched the whole Internet about this, but thither is practically no entropy, just few articles, the result of recitation which was a script. But ahead we dismantle it, let's gear up the base.
Create the sysstat database:
CREATE DATABASE `sysstat`; We create the user sysstat (you can give this drug user some other rights).
Produce USER 'sysstat'@'localhost' IDENTIFIED BY 'some_pass'; GRANT ALL PRIVILEGES ON sysstat.* TO 'sysstat'@'localhost'; We create tables, information technology looks something like this for Pine Tree State:
CREATE Shelve `host_health_cpu` ( `datetime` datetime NOT NULL Default option '0000-00-00 00:00:00', `pct_user` decimal(10,2) DEFAULT NULL, `pct_nice` decimal(10,2) DEFAULT Zilch, `pct_system` decimal fraction(10,2) Default option NULL, `pct_iowait` decimal fraction(10,2) DEFAULT NULL, `pct_steal` decimal(10,2) Default option NULL, `pct_idle` quantitative(10,2) Default option NULL ) ENGINE=InnoDB Nonpayment CHARSET=utf8; CREATE TABLE `host_health_memory` ( `datetime` datetime NOT NULL DEFAULT '0000-00-00 00:00:00', `kbmemfree` int(11) DEFAULT Zip, `kbmemused` int(11) DEFAULT NULL, `per_memused` denary(10,2) DEFAULT NULL, `kbbuffers` int(11) Nonpayment NULL, `kbcached` int(11) Nonremittal NULL, `kbcommit` int(11) Nonremittal Nix, `per_commit` decimal(10,2) DEFAULT Nil, `kbactive` int(11) DEFAULT Zilch, `kbinact` int(11) Default option NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8; Make TABLE `host_health_la` ( `datetime` datetime NOT NULL DEFAULT '0000-00-00 00:00:00', `runq_sz` int(11) Default on NULL, `plist_sz` int(11) Default option NULL, `ldavg_1` decimal fraction(10,2) DEFAULT NULL, `ldavg_5` decimal(10,2) Default on NULL, `ldavg_15` quantitative(10,2) DEFAULT NULL ) Locomotive engine=InnoDB Default option CHARSET=utf8; CREATE TABLE `host_health_net` ( `datetime` datetime NOT NULL DEFAULT '0000-00-00 00:00:00', `iface` varchar(7) DEFAULT NULL, `rxpck_persec` decimal(10,2) DEFAULT NULL, `txpck_persec` decimal(10,2) DEFAULT Zero, `rxbyt_persec` decimal(10,2) DEFAULT NULL, `txbyt_persec` denary(10,2) DEFAULT NULL, `rxcmp_persec` decimal fraction(10,2) Default on NULL, `txcmp_persec` decimal(10,2) DEFAULT NULL, `rxcst_persec` decimal(10,2) Nonremittal NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8; Everything, now everything is ready, in order to upload reports to MySQL.
Parse the script
In the script, we will use the sadf public utility , which can reveal information collected by SAR in various formats, e.g., in CSV, JSON, XML. We will penury a CSV. The script itself is not very boastfully and not difficult to understand, I volition only add that I remove the first 2 columns from the sadf yield, they bear information about the host and the interval between data readings. There is only one host in our case, and I already know the separation, thusly I do non need this information.
#!/bin/bash #Инициализируем переменные WORKDIR=/var/cache/sar FMATDIR=/volt-ampere/cache/sar_data SADF=`which sadf` Question=`which head` AWK=`which awk` #Копируем последний журнал в рабочую директорию cd ${WORKDIR} COPY=`which cp` SAR_LOG=/var/log/sysstat candle ${SAR_LOG} LATEST_DATA=`ls -tlr ${SAR_LOG} | rear end -1 | awk '{print $9}'` ${Replicate} ${LATEST_DATA} ${WORKDIR} for file in *; do #Выбираем репорты за последние 6 минут. Для этого создаем переменную $TIME UNIXTIME=$((`date +%s`-300)) TIME=`date -d@${UNIXTIME} +%H:%M:%S` #Статистика по CPU ${SADF} -d ${file} -s ${TIME} -t | cut -d ';' -f3,5- > "${FMATDIR}"/"${file}"-host_health_cpu.csv #Сетевая статистика ${SADF} -d ${file in} -s ${TIME} -t -- -n DEV | cut -d ';' -f3- > "${FMATDIR}"/"${file}"-host_health_net.csv #Размеры очередей и LA ${SADF} -d ${file} -s ${TIME} -t -- -q | cut -d ';' -f3- > "${FMATDIR}"/"${file}"-host_health_la.csv #Статистика по использованию памяти ${SADF} -d ${file} -s ${TIME} -t -- -r | cut -d ';' -f3- > "${FMATDIR}"/"${file}"-host_health_memory.csv done #Заливаем данные в MySQL cd ${FMATDIR} MYSQL=`which mysql` Exploiter='user' PASS='some_pass' HOST='localhost' DB='sysstat' for data file in *.csv; do ${MYSQL} -u${USER} -p${PASS} -h${HOST} -D${DB} -e "LOAD DATA LOCAL INFILE '${FMATDIR}/${file}' INTO TABLE `echo ${file} | sed 's/.csv//g' | awk -F'-' '{print $2}'` William Claude Dukenfield Concluded BY ';' IGNORE 1 LINES;" done #Удаляем временные файлы rm -fr ${FMATDIR}/*.csv rm -fr ${WORKDIR}/sa* In the main, much a script. In decree for the filling to commenc, contribute it to cron. We will call once all 5 proceedings.
*/5 * * * * bash /usr/local/sbin/sar.sh An interesting feature of SAR is that when we call information technology as sar -s, indicating the start up time, this time itself is not included in the report card, indeed when we call the sar script in the last 6 minutes, we get the report in 5 minutes in the output. This must be taken into accounting. Otherwise there testament be "holes" Beaver State duplicates in MySQL.
As a event, you will get something like the following:
mysql> SELECT * FROM host_health_memory WHERE datetime > Right away() - Time interval 10 MINUTE; +---------------------+-----------+-----------+-------------+-----------+----------+----------+------------+----------+---------+ | datetime | kbmemfree | kbmemused | per_memused | kbbuffers | kbcached | kbcommit | per_commit | kbactive | kbinact | +---------------------+-----------+-----------+-------------+-----------+----------+----------+------------+----------+---------+ | 2015-03-02 08:36:01 | 1381896 | 679396 | 32.00 | 104044 | 155520 | 803420 | 38.00 | 484244 | 142688 | | 2015-03-02 08:37:01 | 1377476 | 683816 | 33.00 | 104068 | 155668 | 810284 | 39.00 | 487632 | 142808 | | 2015-03-02 08:38:01 | 1377476 | 683816 | 33.00 | 104096 | 155672 | 810284 | 39.00 | 487668 | 142804 | | 2015-03-02 08:39:01 | 1377476 | 683816 | 33.00 | 104120 | 155680 | 810524 | 39.00 | 487832 | 142804 | | 2015-03-02 08:40:01 | 1372416 | 688876 | 33.00 | 104160 | 155684 | 819104 | 39.00 | 490708 | 142816 | | 2015-03-02 08:41:01 | 1377104 | 684188 | 33.00 | 104276 | 155700 | 810524 | 39.00 | 488008 | 142808 | | 2015-03-02 08:42:01 | 1379228 | 682064 | 33.00 | 104288 | 155708 | 816640 | 39.00 | 486392 | 142632 | | 2015-03-02 08:43:01 | 1378980 | 682312 | 33.00 | 104328 | 155708 | 816744 | 39.00 | 486680 | 142628 | | 2015-03-02 08:44:01 | 1378608 | 682684 | 33.00 | 104356 | 155716 | 816932 | 39.00 | 486936 | 142636 | | 2015-03-02 08:45:01 | 1371564 | 689728 | 33.00 | 104392 | 155720 | 827704 | 40.00 | 491912 | 142648 | +---------------------+-----------+-----------+-------------+-----------+----------+----------+------------+----------+---------+ What to do with this information?
You can do anything with them. I wrote a dinky higher that founded on this information we bequeath draw graphs. This can represent through with in any way possible. It all depends on your imagination.
Gnuplot
My choice fell on Gnuplot . This is a jolly convenient tool for building graphs, charts, etc. There is a lot of documentation connected Gnuplot connected his home plate site , thusly we will skip the part about its features and how to usance it.
I wrote this script:
#!/bin/bash sleep 10 # Параметры по умолчанию width="640" utmost="480" outfile="graph.svg" format="svg" NO_ARGS=0 E_OPTERROR=65 if [ $# -eq "$NO_ARGS" ] # Сценарий вызван без аргументов? past echo "Скрипт запущен без параметров! Для правильной работы используйте параметры: -o Указывает имя файла графика (по умолчанию \"graph.svg\") -f Указывает формат файла графика (по умолчанию \"svg\") -w Ширина файла (по умолчанию \"640\") -h Высота файла (по умолчанию \"480\") -t Тип графика Повторите вашу попытку." exit $E_OPTERROR fi #Выбираем необходимые параметры while getopts ":h:w:t:f:o:" Option do subject $Option in h) high="$OPTARG";; w) width="$OPTARG";; f) format="$OPTARG";; o) outfile="$OPTARG";; t) type="$OPTARG";; esac through with budge $(($OPTIND - 1)) #Записываем результат во временный файл /tmp/datatmp if [ ${type} == 'cpu' ] then #Создаем заголовок графика title="Распределение нагрузки на CPU, %" Question="select time(datetime), pct_idle equally idle, pct_iowait equally iowait, pct_system as system, pct_user as user from host_health_cpu where now() - interval 1 hour < datetime;" format_y='set format y "%.0f%%";' elif [ ${type} == 'mem' ] then title="Использование памяти, B" Inquiry="select time(datetime),(kbmemfree * 1024) every bit Unrestrained,(kbmemused * 1024) atomic number 3 Utilised,(kbbuffers * 1024) as Buffers,(kbcached * 1024) as Cached from host_health_memory where instantly() - interval 1 hour < datetime;" format_y="localise format y '%.1s%cB'" elif [ ${type} == "net" ] then deed of conveyance="Входящий и исходящий трафик, B/s" QUERY='pick out time(datetime), (rxbyt_persec * 1024) as Rx, (txbyt_persec * 1024) as Tx from host_health_net where forthwith() - interval 1 hour < datetime and iface="eth0";' format_y="set format y '%.1s%cB/s'" elif [ ${type} == "la" ] then title="Load Mean" Question="prime time(datetime),ldavg_1 as LoadAvg1, ldavg_5 LoadAvg5,ldavg_15 as LoadAvg15 from host_health_la where now() - interval 1 hour < datetime;" format_y="set arrange y '%.2f'" fi #Делаем выборку из базы и сохраняем во временный файл MYSQL=`which mysql` Exploiter="user" PASS="some_pass" Dubnium="sysstat" ${MYSQL} -u${USER} -p${Passing} -D${DB} -B -r -e "${Enquiry}" >> /tmp/datatmp_${type} #Подсчитываем количество столбцов в файле. cols=`awk '{print NF}' /tmp/datatmp_${type} | sort -nu | shadower -n 1` #Определяем временные промежутки hour_ago=`${MYSQL} -u${USER} -p${PASS} -D${DB} -N -e "select time(datetime) from host_health_cpu where now() - interval 1 hour < datetime limit 1;"` directly=`${MYSQL} -u${Drug user} -p${PASS} -D${DB} -N -e "select time(datetime) from host_health_cpu where today() - interval 1 hour < datetime order by datetime desc terminal point 1;"` #Рисуем график gnuplot << EOP #Указываем формат файла и его размер set terminal ${format} size ${width},${high} #Указываем выходной файл set production ${outfile} #Рисуем легенды band Francis Scott Key autotitle columnhead set headstone right center behind hard key crosswise #Рисуем заголовок set elan fill transparent hard 0.5 noborder set title "${title}" #Делаем ось Х в формате отображения дат set xdata clock time set timefmt "%H:%M:%S" set xrange ["${hour_ago}":"${now}"] set xtics format "%H:%M" #Указываем имена осей set xlabel "Время" put away ylabel "${title}" set grid set yrange [0:*] ${format_y} #Получаем конечный результат. plat for [i=2:${cols}] "/tmp/datatmp_${type}" exploitation 1:i smooth unique with filledcurve x1 EOP #Удаляем временный файл. rm /tmp/datatmp_${typecast} echo "Picture spell to \"${outfile}\"" issue 0; Run it like this:
$ sar-patch.sh -t mem -o /path/to/memory.svg The result is such a schedule in 1 hour.
If you add this playscript to cron and call information technology, for example, once all 5 minutes, then we testament forever have fresh graphs. For good example, I did it like-minded this:
*/5 * * * * belt /usr/local/sbin/sar-plot.sh -t cpu -o /path/to/images/processor.svg */5 * * * * bash /usr/local/sbin/sar-plot.sh -t mem -o /course/to/images/memory.svg */5 * * * * bash /usr/local/sbin/sar-plot.sh -t net -o /path/to/images/network.svg */5 * * * * bang /usr/local/sbin/sar-plot.sh -t la -o /path/to/images/la.svg Promote, these images give the axe be added to the HTML page, to ready it automatically updated. Thus, we can monitor host performance in near real time (with a delay of 5 minutes). What is enough for us in rule.
Conclusion
SAR is a very convenient creature, indispensable in the work of the system administrator. The monitoring method described by me is well suited for those who are same limited in resources, do not want to ascent Zabbix on the server surgery for respective other reasons.
You stern develop this result for yourself:
- Add other metrics that are not described in the article, for example, you can add statistics along the activity of the harrow subsystem.
- You can draw graphics in any other way, for example, using Javascript libraries.
- You tail modify the system sol that when whatever critical values are reached, alerts are displayed connected your World Wide Web page, and, for example, messages are sent to your mail.
It every last depends on your desire. Luckiness
DOWNLOAD HERE
GET Monitoring for the poor or SAR + MySQL + Gnuplot / Sudo Null IT News FREE
Posted by: weirnonsts.blogspot.com
0 Response to "GET Monitoring for the poor or SAR + MySQL + Gnuplot / Sudo Null IT News FREE"
Post a Comment