{"id":1181,"date":"2015-05-19T17:43:09","date_gmt":"2015-05-19T22:43:09","guid":{"rendered":"http:\/\/www.nathanhunstad.com\/blog\/?p=1181"},"modified":"2015-05-19T17:45:50","modified_gmt":"2015-05-19T22:45:50","slug":"monitoring-my-ups-with-splunk","status":"publish","type":"post","link":"https:\/\/www.nathanhunstad.com\/blog\/2015\/05\/monitoring-my-ups-with-splunk\/","title":{"rendered":"Monitoring my UPS with Splunk"},"content":{"rendered":"<p><a href=\"http:\/\/www.nathanhunstad.com\/blog\/2015\/05\/installing-nut-on-centos\/\">Last time<\/a>, I had set up my UPS monitoring software on my CentOS logging server. But I wanted more: what good is having a UPS if I can\u2019t monitor things like voltage, battery charge, and load on an ongoing basis? Of course, the answer to this is to log to Splunk, which is what I ended up doing.<\/p>\n<p><!--more--><\/p>\n<p>To do the logging, I used <a href=\"http:\/\/www.networkupstools.org\/docs\/man\/upslog.html\" target=\"_blank\">UPSLOG<\/a>, which as you may be able to tell from the name, is a logger of UPS data. It took a few steps to get it set up properly, though, for long-term logging.<\/p>\n<p>First, I created a new user to run upslog as, since I didn\u2019t want it running as root for security reasons. That allows me to run upslog with the \u2013u option to setuid to the new user. The log file itself is located at <span style=\"font-family: 'Courier New';\">\/var\/log\/ups.log<\/span>. In order for Splunk to read it, I ran chmod of the log file to set the group to the splunk group, granting Splunk read access. So far, so good.<\/p>\n<p>Eventually, the log is going to get huge, and so I want to rotate it properly. To do this, I created a new file in <span style=\"font-family: 'Courier New';\">\/etc\/logrotate.d<\/span> in order to manage the log rotation. Never having created a logrotate configuration file before, I went with this:<\/p>\n<p><span style=\"font-family: 'Courier New';\">\/var\/log\/ups.log {<br \/>\nweekly<br \/>\nrotate 14<br \/>\ncompress<br \/>\ndelaycompress<br \/>\nmissingok<br \/>\nnotifempty<br \/>\ncreate 644 upslog splunk<br \/>\n}<\/span><\/p>\n<p>Will it work? We\u2019ll see! The important things are to create the new log with owner and group of upslog and splunk respectively. If that works, we should be in good shape.<\/p>\n<p>Finally, I wanted to create a init.d script for starting and stopping the service on boot and shutdown automatically. UPSLOG didn\u2019t have an init script for some reason, so I copied the ups script and changed it accordingly:<\/p>\n<p><span style=\"font-family: 'Courier New';\">#! \/bin\/bash<br \/>\n#<br \/>\n# upslog: Starts the UPS logging service<br \/>\n#<br \/>\n# chkconfig: &#8211; 26 74<br \/>\n# description: Starts the UPS logging service to log UPS statistics to \/var\/log\/ups.log<br \/>\n# processname: upslog<br \/>\n#<br \/>\n### BEGIN INIT INFO<br \/>\n# Provides: upslog<br \/>\n# Required-Start: $syslog $network $named<br \/>\n# Required-Stop: $local_fs<br \/>\n# Default-Stop: 0 1 6<br \/>\n# Short-Description: Starts the UPS logging service<br \/>\n# Description: Stars the UPS logging service to log UPS statistics to \/var\/log\/ups.log<br \/>\n### END INIT INFO<\/span><\/p>\n<p><span style=\"font-family: 'Courier New';\"># Source function library.<br \/>\nif [ -f \/etc\/init.d\/functions ]; then<br \/>\n. \/etc\/init.d\/functions<br \/>\nelif [ -f \/etc\/rc.d\/init.d\/functions ]; then<br \/>\n. \/etc\/rc.d\/init.d\/functions<br \/>\nelse<br \/>\nexit 0<br \/>\nfi<\/span><\/p>\n<p><span style=\"font-family: 'Courier New';\">pidfile=\/var\/run\/nut\/upslog.pid<br \/>\nbin=\/usr\/bin\/upslog<\/span><\/p>\n<p><span style=\"font-family: 'Courier New';\">start() {<br \/>\nprintf &#8220;Starting upslog&#8230;\\n&#8221;<br \/>\nupslog -s apc@localhost -l \/var\/log\/ups.log -u upslog<br \/>\nsleep 1<br \/>\nif [ -f $pidfile ]; then<br \/>\nprintf &#8220;OK\\n&#8221;<br \/>\nelse<br \/>\nprintf &#8220;Fail\\n&#8221;<br \/>\nfi<br \/>\n}<\/span><\/p>\n<p><span style=\"font-family: 'Courier New';\">stop() {<br \/>\nprintf &#8220;Stopping upslog&#8230;&#8221;<br \/>\nif [ -f $pidfile ]; then<br \/>\npid=`cat $pidfile`<br \/>\nkill $pid<br \/>\nprintf &#8220;upslog stopped\\n&#8221;<br \/>\nrm -f $pidfile<br \/>\nelse<br \/>\nprintf &#8220;pidfile not found\\n&#8221;<br \/>\nfi<br \/>\n}<\/span><\/p>\n<p><span style=\"font-family: 'Courier New';\">restart() {<br \/>\nstop<br \/>\nstart<br \/>\n}<\/span><\/p>\n<p><span style=\"font-family: 'Courier New';\"># See how we are called.<br \/>\ncase &#8220;$1&#8221; in<br \/>\nstart)<br \/>\nstart ;;<\/span><\/p>\n<p><span style=\"font-family: 'Courier New';\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 stop)<br \/>\nstop ;;<\/span><\/p>\n<p><span style=\"font-family: 'Courier New';\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 restart)<br \/>\nrestart ;;<\/span><\/p>\n<p><span style=\"font-family: 'Courier New';\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 status)<br \/>\nif [ &#8220;$SERVER&#8221; = &#8220;yes&#8221; ]; then<br \/>\nstatus upsd<br \/>\nfi<br \/>\nstatus upsmon<br \/>\n;;<\/span><\/p>\n<p><span style=\"font-family: 'Courier New';\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 *)<br \/>\necho $&#8221;Usage: $0 {start|stop|restart|status}&#8221;<br \/>\nRETVAL=3<br \/>\nesac<\/span><\/p>\n<p><span style=\"font-family: 'Courier New';\">exit $RETVAL<\/span><\/p>\n<p>Will this work? Well, I can stop and start it well enough, and with a UPS hopefully I won\u2019t need to power this down anytime soon. After saving this, and a chkconfig upslog on later, it said it would run on boot, so that looks good enough.<\/p>\n<p>With all of that set up, getting the data into Splunk was a simple as setting up a new file data source. I created a new index called \u201cups\u201d to hold the data, and soon I was getting events every 30 seconds:<\/p>\n<p><a href=\"http:\/\/www.nathanhunstad.com\/blog\/wp-content\/uploads\/2015\/05\/image.png\"><img loading=\"lazy\" decoding=\"async\" style=\"background-image: none; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border: 0px;\" title=\"image\" src=\"http:\/\/www.nathanhunstad.com\/blog\/wp-content\/uploads\/2015\/05\/image_thumb.png\" alt=\"image\" width=\"627\" height=\"121\" border=\"0\" \/><\/a><\/p>\n<p>In order to get the volts, battery charge, and load, I used Splunk\u2019s field extraction tool. With those fields extracted, my dashboards pretty much made themselves:<\/p>\n<p><a href=\"http:\/\/www.nathanhunstad.com\/blog\/wp-content\/uploads\/2015\/05\/image1.png\"><img loading=\"lazy\" decoding=\"async\" style=\"background-image: none; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border: 0px;\" title=\"image\" src=\"http:\/\/www.nathanhunstad.com\/blog\/wp-content\/uploads\/2015\/05\/image_thumb1.png\" alt=\"image\" width=\"695\" height=\"433\" border=\"0\" \/><\/a><\/p>\n<p>That, in a nutshell, is how easy it is to get UPS data into Splunk!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Last time, I had set up my UPS monitoring software on my CentOS logging server. But I wanted more: what good is having a UPS if I can\u2019t monitor things like voltage, battery charge, and load on an ongoing basis? Of course, the answer to this is to log to Splunk, which is what I&hellip; <a class=\"more-link\" href=\"https:\/\/www.nathanhunstad.com\/blog\/2015\/05\/monitoring-my-ups-with-splunk\/\">Continue reading <span class=\"screen-reader-text\">Monitoring my UPS with Splunk<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[19],"tags":[247,269,252],"class_list":["post-1181","post","type-post","status-publish","format-standard","hentry","category-tech-2","tag-centos","tag-nut","tag-splunk","entry"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.nathanhunstad.com\/blog\/wp-json\/wp\/v2\/posts\/1181","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.nathanhunstad.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.nathanhunstad.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.nathanhunstad.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.nathanhunstad.com\/blog\/wp-json\/wp\/v2\/comments?post=1181"}],"version-history":[{"count":3,"href":"https:\/\/www.nathanhunstad.com\/blog\/wp-json\/wp\/v2\/posts\/1181\/revisions"}],"predecessor-version":[{"id":1185,"href":"https:\/\/www.nathanhunstad.com\/blog\/wp-json\/wp\/v2\/posts\/1181\/revisions\/1185"}],"wp:attachment":[{"href":"https:\/\/www.nathanhunstad.com\/blog\/wp-json\/wp\/v2\/media?parent=1181"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.nathanhunstad.com\/blog\/wp-json\/wp\/v2\/categories?post=1181"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.nathanhunstad.com\/blog\/wp-json\/wp\/v2\/tags?post=1181"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}