2/15/2007

Log Monitor : 簡便的分散式 log 管理工具

JavaEye 的 CVU 版友最近發表了一篇 Log Monitor - 把分散在各台server上的log集中起来monitor 的程式,主要功能是可以將分散在各個機器的 Rails log 給集合起來。主要是使用 net-ssh 連到各個 server,然後將指定好的 log 給 tail -f 起來。經過試用後發現程式簡單方便修改,但是輸出目前為止又很堪用,實在感謝他。License 是 CC 2.5 姓名,相同方式


config 寫法
config出現在第六行裡面的 modules.conf ,Modules.conf 寫法如下

log 所在的 server ip or domain name:
__username:ssh 的使用者名稱
__password:ssh user 的 password
__logs:
____/log/所在/目錄/log檔名:
______name: [在 log monitor 顯示的名稱]
______greps:
________error.log:
__________- ERROR
________info.log:
__________- INFO

程式寫法
### license: http://creativecommons.org/licenses/by-sa/2.5/deed.zh
### by: caoweiyuan#gmail.com
require 'net/ssh'

require 'yaml'
modules = YAML.load(File.open('modules.conf'))
### sample config
# engine-1:
# username: admin
# password: ******
# logs:
# /opt/apps/production/tomcat/logs/app.log:
# name: [module-1]
# greps:
# error.log:
# - ERROR
# info.log:
# - INFO

def do_grep(name, line, greps)
log = "[#{name}] - #{line}"

# default output to screen
puts log

# grep patterns to sepecific files
for log_file, patterns in greps
pattern = (patterns.class == Array) ? patterns.join("|") : patterns
File.open('log/' + log_file, 'a') {|f| f.puts log} if line =~ /#{pattern}/
# TODO: auto mail for errors
# TODO: horizontal log for different products, and delete after each process over.
end
end

def do_tail( session, log_file, log_conf )
session.open_channel do |channel|
channel.on_data do |ch, data|
data.each_line do |line|
do_grep log_conf['name'], line, log_conf['greps']
end
end
channel.exec "tail -1f #{log_file}"
end
end

for key, value in modules
t = Thread.new(key, value) do |host, host_conf|
Net::SSH.start( host, host_conf['username'], host_conf['password'] ) do |session|
puts "#{host} logged in."
for log_file, log_conf in host_conf['logs']
puts "start watching #{log_file}"
# open channels for multiple log files
do_tail session, log_file, log_conf
end # log
session.loop # important: loop to keep reading channel outputs
end # session
end # thread
end # host

# TODO: implement a formal deamon
loop do
# deamonize
sleep 1
end

2 則留言:

cvu 提到...

thegiive兄弟客气了,很高兴程序对你有用。我从你的blog里也学到不少东西。互相学习。

匿名 提到...

I like watching [url=http://www.finsfootball.com]football[/url] on [url=http://www.finsfootball.com/bbc-football]bbc football[/url] chanel. How about you people?